ratelimit-bucketbased 0.0.1
Sign up to get free protection for your applications and to get access to all the features.
- data.tar.gz.sig +4 -0
- data/.gitignore +17 -0
- data/Gemfile +4 -0
- data/LICENSE.txt +22 -0
- data/README.md +119 -0
- data/Rakefile +12 -0
- data/lib/ratelimit/bucketbased.rb +264 -0
- data/lib/ratelimit/bucketbased/version.rb +5 -0
- data/ratelimit-bucketbased.gemspec +31 -0
- data/test/helper.rb +5 -0
- data/test/test.db +0 -0
- data/test/test_ratelimit-bucketbased.rb +155 -0
- metadata +177 -0
- metadata.gz.sig +0 -0
data.tar.gz.sig
ADDED
data/.gitignore
ADDED
data/Gemfile
ADDED
data/LICENSE.txt
ADDED
@@ -0,0 +1,22 @@
|
|
1
|
+
Copyright (c) 2013 chrislee35
|
2
|
+
|
3
|
+
MIT License
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
6
|
+
a copy of this software and associated documentation files (the
|
7
|
+
"Software"), to deal in the Software without restriction, including
|
8
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
9
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
10
|
+
permit persons to whom the Software is furnished to do so, subject to
|
11
|
+
the following conditions:
|
12
|
+
|
13
|
+
The above copyright notice and this permission notice shall be
|
14
|
+
included in all copies or substantial portions of the Software.
|
15
|
+
|
16
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
17
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
18
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
19
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
20
|
+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
21
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
22
|
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,119 @@
|
|
1
|
+
# RateLimit::BucketBased
|
2
|
+
|
3
|
+
This is a very simple rate limiting gem useful for regulating the speed at which service is provided and denying service when that rate has been exceeded. This is very similar to other gems like ratelimit, rack-ratelimiter, rate-limiting, and a ton of gems with the name "throttle" in them. Mine is different in that it (a) isn't tied to a particular storage, (b) isn't tied to a particular framework, and (c) was written by me. I guess there might be merits to my approach, but there's a lot of sharp coders out there.
|
4
|
+
|
5
|
+
To track the rates of various transactions Memory, MySQL, SQLite3, MemCache, and Redis storage options are supported, but the appropriateness of one over another for various workloads is an exercise left to the reader. I have not added any crafty speed-ups for the SQL databases (e.g., dynamically sized bloom filters), so use those primarily for durability and scalability, not high transaction rates. Probably the only solution that can scale in the number of items it can track, handle high-transaction loads, and have some durability is Redis. (it's too bad that MemCache can't save to disk)
|
6
|
+
|
7
|
+
Tested in ruby-1.8.7-p371, ruby-1.9.3-p392, and ruby-2.0.0-p0.
|
8
|
+
The Memcache gem fails to compile in ruby-2.0.
|
9
|
+
|
10
|
+
### The Concept
|
11
|
+
|
12
|
+
Imagine a set of buckets, one for each item you want to rate limit such as a user, an apikey, or answers to queries, with a number of balls (credits) in each bucket. Whenever a service is used, balls (credits) are removed from the user's bucket. When the user has no more credits left, the service is denied. Credits are added back to the bucket over time, up to a maximum. It is also possible for someone to be denied service, but since they keep asking for it anyway, they can go into debt up to a minimum (a negative number), thus they will have to wait some time to let the debt be paid.
|
13
|
+
|
14
|
+
Thus, each bucket has the following properties:
|
15
|
+
* +name+: the name of the item that you allocated the bucket for
|
16
|
+
* +current+: the current credit limit (which may be larger than max, or more negative than min)
|
17
|
+
* +max+: the maximum that the bucket will be filled by the regeneration process
|
18
|
+
* +min+: the minimum that a bucket can go, must be <= 0.
|
19
|
+
* +refill_amount+: the amount that will be added to the bucket every refill_epoch seconds.
|
20
|
+
* +refill_epoch+: the number of seconds before the bucket is credited by the regeneration process
|
21
|
+
* +last_refill+: the timestamp of the last refill
|
22
|
+
* +total_used+: the total, cumulative credits used for service (i.e., refusals don't count)
|
23
|
+
|
24
|
+
To set these parameters easily, you can create named configurations to create buckets using those templates. The configurations have the following fields:
|
25
|
+
* +name+: the name of the configuration
|
26
|
+
* +start+: the credits that a new bucket starts with
|
27
|
+
* +max+: the maximum that the bucket will be filled by the regeneration process
|
28
|
+
* +min+: the minimum that a bucket can go, must be <= 0.
|
29
|
+
* +refill_amount+: the amount that will be added to the bucket every refill_epoch seconds.
|
30
|
+
* +refill_epoch+: the number of seconds before the bucket is credited by the regeneration process
|
31
|
+
|
32
|
+
|
33
|
+
## Installation
|
34
|
+
|
35
|
+
Add this line to your application's Gemfile:
|
36
|
+
|
37
|
+
gem 'ratelimiter-bucketbased'
|
38
|
+
|
39
|
+
And then execute:
|
40
|
+
|
41
|
+
$ bundle
|
42
|
+
|
43
|
+
Or install it yourself as:
|
44
|
+
|
45
|
+
$ gem install ratelimiter-bucketbased
|
46
|
+
|
47
|
+
## Usage
|
48
|
+
|
49
|
+
The steps to use the rate limiter are the following:
|
50
|
+
1. set up the configurations for accounting,
|
51
|
+
1. create a store, (currently, Memory, MySQL, SQLite3, MemCache, and Redis storage options are supported)
|
52
|
+
1. create the rate limiter,
|
53
|
+
1. add non-default items,
|
54
|
+
1. and provide service.
|
55
|
+
|
56
|
+
For each of the examples below, use the following template:
|
57
|
+
|
58
|
+
require 'ratelimit-bucketbased'
|
59
|
+
|
60
|
+
# set up the configs
|
61
|
+
start = max = 10
|
62
|
+
min = -10
|
63
|
+
cost = refill_amount = refill_epoch = 2
|
64
|
+
configs = {
|
65
|
+
'default' => RateLimit::Config.new('default', start, max, min, cost, refill_amount, refill_epoch),
|
66
|
+
'power' => RateLimit::Config.new('default', 20, 20, min, cost, refill_amount, 1)
|
67
|
+
}
|
68
|
+
|
69
|
+
# create a store
|
70
|
+
*storage creation code here, see sub-sections below*
|
71
|
+
|
72
|
+
# create the rate limiter, setting the storage, the configurations, and the name of the default configuration
|
73
|
+
rl = RateLimit::BucketBased.new(storage, configs, 'default')
|
74
|
+
# add a bucket named "admin", using a non-default configuration
|
75
|
+
rl.create_bucket('admin', 'power')
|
76
|
+
|
77
|
+
def provide_service(username)
|
78
|
+
if rl.use(username)
|
79
|
+
// perform service
|
80
|
+
end
|
81
|
+
end
|
82
|
+
|
83
|
+
### Memory-based Store
|
84
|
+
|
85
|
+
# create a Memory-based storage
|
86
|
+
storage = RateLimit::Memory.new
|
87
|
+
|
88
|
+
### SQLite3-based Store
|
89
|
+
|
90
|
+
require 'sqlite3'
|
91
|
+
|
92
|
+
# attach to a SQLite3-based storage
|
93
|
+
dbh = SQLite3::Database.new( "test/test.db" )
|
94
|
+
custom_fields = ["username","credits","max_credits","min_credits","cost_per_transaction","refill_credits","refill_seconds","last_refill_time","total_used_credits"]
|
95
|
+
storage = RateLimit::SQLite3.new(dbh,'users_table',custom_fields)
|
96
|
+
|
97
|
+
### Memcache-based Store
|
98
|
+
|
99
|
+
require 'memcache'
|
100
|
+
|
101
|
+
# create a MemCache-based storage
|
102
|
+
memcache = Memcache.new(:server => 'localhost:11211')
|
103
|
+
storage = RateLimit::Memcache.new(memcache)
|
104
|
+
|
105
|
+
### Redis-based Store
|
106
|
+
|
107
|
+
require 'redis'
|
108
|
+
|
109
|
+
# create a Redis-based storage
|
110
|
+
redis = Redis.new(:server => 'localhost', :port => 6379)
|
111
|
+
storage = RateLimit::Redis.new(redis)
|
112
|
+
|
113
|
+
## Contributing
|
114
|
+
|
115
|
+
1. Fork it
|
116
|
+
2. Create your feature branch (`git checkout -b my-new-feature`)
|
117
|
+
3. Commit your changes (`git commit -am 'Add some feature'`)
|
118
|
+
4. Push to the branch (`git push origin my-new-feature`)
|
119
|
+
5. Create new Pull Request
|
data/Rakefile
ADDED
@@ -0,0 +1,264 @@
|
|
1
|
+
require "ratelimit/bucketbased/version"
|
2
|
+
|
3
|
+
module RateLimit
|
4
|
+
# Bucket tracks the credits for each item.
|
5
|
+
# Each bucket has the following parameters:
|
6
|
+
# * *Args* :
|
7
|
+
# - +name+ -> the name of the item that you allocated the bucket for
|
8
|
+
# - +current+ -> the current credit limit (which may be larger than max, or more negative than min)
|
9
|
+
# - +max+ -> the maximum that the bucket will be filled by the regeneration process
|
10
|
+
# - +min+ -> the minimum that a bucket can go, must be <= 0.
|
11
|
+
# - +refill_amount+ -> the amount that will be added to the bucket every refill_epoch seconds.
|
12
|
+
# - +refill_epoch+ -> the number of seconds before the bucket is credited by the regeneration process
|
13
|
+
# - +last_refill+ -> the timestamp of the last refill
|
14
|
+
# - +total_used+ -> the total, cumulative credits used for service (so refusals don't count)
|
15
|
+
# * *Returns* :
|
16
|
+
# - a Bucket
|
17
|
+
|
18
|
+
class Bucket < Struct.new(:name, :current, :max, :min, :cost, :refill_amount, :refill_epoch, :last_refill, :total_used); end
|
19
|
+
# To set bucket parameters easily, you can create named configurations to create buckets using those templates.
|
20
|
+
# The configurations have the following parameters:
|
21
|
+
# * <tt>name</tt>:: the name of the configuration
|
22
|
+
# * <tt>start</tt>:: the credits that a new bucket starts with
|
23
|
+
# * <tt>max</tt>:: the maximum that the bucket will be filled by the regeneration process
|
24
|
+
# * <tt>min</tt>:: the minimum that a bucket can go, must be <= 0.
|
25
|
+
# * <tt>refill_amount</tt>:: the amount that will be added to the bucket every refill_epoch seconds.
|
26
|
+
# * <tt>refill_epoch</tt>:: the number of seconds before the bucket is credited by the regeneration process
|
27
|
+
class Config < Struct.new(:name, :start, :max, :min, :cost, :refill_amount, :refill_epoch); end
|
28
|
+
|
29
|
+
# "Storage" is a jerk class that throws exceptions if you forgot to implement a critical function, welcome to Ruby (no interface)
|
30
|
+
# Bucket tracks the credits for each item.
|
31
|
+
# Each bucket has the following parameters:
|
32
|
+
# * *Args* :
|
33
|
+
# - +name+ -> the name of the item that you allocated the bucket for
|
34
|
+
# - +current+ -> the current credit limit (which may be larger than max, or more negative than min)
|
35
|
+
# - +max+ -> the maximum that the bucket will be filled by the regeneration process
|
36
|
+
# - +min+ -> the minimum that a bucket can go, must be <= 0.
|
37
|
+
# - +refill_amount+ -> the amount that will be added to the bucket every refill_epoch seconds.
|
38
|
+
# - +refill_epoch+ -> the number of seconds before the bucket is credited by the regeneration process
|
39
|
+
# - +last_refill+ -> the timestamp of the last refill
|
40
|
+
# - +total_used+ -> the total, cumulative credits used for service (so refusals don't count)
|
41
|
+
# * *Returns* :
|
42
|
+
# - a Bucket
|
43
|
+
# * *Raises* :
|
44
|
+
# - +ArgumentError+ -> if any value is nil or negative
|
45
|
+
class Storage
|
46
|
+
# retrieves a named bucket
|
47
|
+
# * *Args* :
|
48
|
+
# - +name+ -> the name of the bucket to be retrieved
|
49
|
+
# * *Returns* :
|
50
|
+
# - the bucket matching the name if found, nil otherwise
|
51
|
+
# * *Raises* :
|
52
|
+
# - +NoMethodError+ -> always, because this class is a jerk
|
53
|
+
def get(name)
|
54
|
+
raise NoMethodError
|
55
|
+
end
|
56
|
+
|
57
|
+
# saves a bucket into the storage
|
58
|
+
# * *Args* :
|
59
|
+
# - +bucket+ -> the Bucket to set. The <tt>name</tt> field in the Bucket option will be used as a key.
|
60
|
+
# * *Returns* :
|
61
|
+
# - nil
|
62
|
+
# * *Raises* :
|
63
|
+
# - +NoMethodError+ -> always, because this class is a jerk
|
64
|
+
def set(bucket)
|
65
|
+
raise NoMethodError
|
66
|
+
end
|
67
|
+
|
68
|
+
# updates the key fields that need updating into the storage
|
69
|
+
# this is often cheaper for certain types of storage than using set()
|
70
|
+
# * *Args* :
|
71
|
+
# - +bucket+ -> the Bucket to update. The <tt>name</tt> field in the Bucket option will be used as a key.
|
72
|
+
# * *Returns* :
|
73
|
+
# - nil
|
74
|
+
# * *Raises* :
|
75
|
+
# - +NoMethodError+ -> always, because this class is a jerk
|
76
|
+
def update(bucket)
|
77
|
+
raise NoMethodError
|
78
|
+
end
|
79
|
+
end
|
80
|
+
|
81
|
+
class Memory < Storage
|
82
|
+
def initialize
|
83
|
+
@buckets = {}
|
84
|
+
end
|
85
|
+
|
86
|
+
# retrieves a named bucket
|
87
|
+
# * *Args* :
|
88
|
+
# - +name+ -> the name of the bucket to be retrieved
|
89
|
+
# * *Returns* :
|
90
|
+
# - the bucket matching the name if found, nil otherwise
|
91
|
+
def get(name)
|
92
|
+
@buckets[name]
|
93
|
+
end
|
94
|
+
|
95
|
+
# saves a bucket into the storage
|
96
|
+
# * *Args* :
|
97
|
+
# - +bucket+ -> the Bucket to set. The <tt>name</tt> field in the Bucket option will be used as a key.
|
98
|
+
# * *Returns* :
|
99
|
+
# - the bucket that is provided in the Args
|
100
|
+
def set(bucket)
|
101
|
+
@buckets[bucket.name] = bucket
|
102
|
+
end
|
103
|
+
|
104
|
+
# updates the key fields that need updating into the storage
|
105
|
+
# this is often cheaper for certain types of storage than using set()
|
106
|
+
# * *Args* :
|
107
|
+
# - +bucket+ -> the Bucket to update. The <tt>name</tt> field in the Bucket option will be used as a key.
|
108
|
+
# * *Returns* :
|
109
|
+
# - nil
|
110
|
+
def update(bucket)
|
111
|
+
# already updated
|
112
|
+
end
|
113
|
+
end
|
114
|
+
|
115
|
+
class MySQL < Storage
|
116
|
+
def initialize(dbh, table, fields=["name","current","max","min","cost","refill_amount","refill_epoch","last_refill","total_used"])
|
117
|
+
@queries = {
|
118
|
+
'get' => dbh.prepare("SELECT `#{fields.join('`, `')}` FROM `#{table}` WHERE `#{fields[0]}` = ? LIMIT 1"),
|
119
|
+
'update' => dbh.prepare("UPDATE `#{table}` SET `#{fields[1]}` = ?, `#{fields[7]}` = ?, `#{fields[8]}` = ? WHERE `#{fields[0]}` = ?"),
|
120
|
+
'set' => dbh.prepare("REPLACE INTO `#{table}` (`#{fields.join('`, `')}`) VALUES (?,?,?,?,?,?,?,?,?)")
|
121
|
+
}
|
122
|
+
end
|
123
|
+
|
124
|
+
# retrieves a named bucket
|
125
|
+
# * *Args* :
|
126
|
+
# - +name+ -> the name of the bucket to be retrieved
|
127
|
+
# * *Returns* :
|
128
|
+
# - the bucket matching the name if found, nil otherwise
|
129
|
+
# * *Raises* :
|
130
|
+
# - +Mysql::Error+ -> any issue with the connection to the database or the SQL statements
|
131
|
+
def get(name)
|
132
|
+
rs = @queries['get'].execute(name)
|
133
|
+
bucket = nil
|
134
|
+
rs.each do |row|
|
135
|
+
bucket = Bucket.new(row[0],*row[1,8].map{|x| x.to_f})
|
136
|
+
end
|
137
|
+
bucket
|
138
|
+
end
|
139
|
+
|
140
|
+
# saves a bucket into the storage
|
141
|
+
# * *Args* :
|
142
|
+
# - +bucket+ -> the Bucket to set. The <tt>name</tt> field in the Bucket option will be used as a key.
|
143
|
+
# * *Returns* :
|
144
|
+
# - an empty result set
|
145
|
+
# * *Raises* :
|
146
|
+
# - +Mysql::Error+ -> any issue with the connection to the database or the SQL statements
|
147
|
+
def set(bucket)
|
148
|
+
@queries['set'].execute(bucket.name, bucket.current, bucket.max, bucket.min, bucket.cost, bucket.refill_amount, bucket.refill_epoch, bucket.last_refill, bucket.total_used)
|
149
|
+
end
|
150
|
+
|
151
|
+
# updates the key fields that need updating into the storage
|
152
|
+
# this is often cheaper for certain types of storage than using set()
|
153
|
+
# * *Args* :
|
154
|
+
# - +bucket+ -> the Bucket to update. The <tt>name</tt> field in the Bucket option will be used as a key.
|
155
|
+
# * *Returns* :
|
156
|
+
# - an empty result set
|
157
|
+
# * *Raises* :
|
158
|
+
# - +Mysql::Error+ -> any issue with the connection to the database or the SQL statements
|
159
|
+
def update(bucket)
|
160
|
+
@queries['update'].execute(bucket.current, bucket.last_refill, bucket.total_used, bucket.name)
|
161
|
+
end
|
162
|
+
end
|
163
|
+
|
164
|
+
SQLite3 = MySQL
|
165
|
+
|
166
|
+
class MemCache
|
167
|
+
def initialize(cache_handle)
|
168
|
+
@cache = cache_handle
|
169
|
+
end
|
170
|
+
|
171
|
+
# retrieves a named bucket
|
172
|
+
# * *Args* :
|
173
|
+
# - +name+ -> the name of the bucket to be retrieved
|
174
|
+
# * *Returns* :
|
175
|
+
# - the bucket matching the name if found, nil otherwise
|
176
|
+
def get(name)
|
177
|
+
value = @cache.get(name)
|
178
|
+
return nil unless value
|
179
|
+
row = value.split(/\|/)
|
180
|
+
bucket = nil
|
181
|
+
if row
|
182
|
+
bucket = Bucket.new(row[0],*row[1,8].map{|x| x.to_f})
|
183
|
+
end
|
184
|
+
bucket
|
185
|
+
end
|
186
|
+
|
187
|
+
# saves a bucket into the storage
|
188
|
+
# * *Args* :
|
189
|
+
# - +bucket+ -> the Bucket to set. The <tt>name</tt> field in the Bucket option will be used as a key.
|
190
|
+
# * *Returns* :
|
191
|
+
# - the bucket that is provided in the Args
|
192
|
+
def set(bucket)
|
193
|
+
@cache.set(bucket.name,bucket.values.join("|"))
|
194
|
+
end
|
195
|
+
|
196
|
+
alias :update :set
|
197
|
+
end
|
198
|
+
|
199
|
+
Memcache = MemCache
|
200
|
+
Redis = MemCache
|
201
|
+
|
202
|
+
# BucketBased is the star of the show. It takes a storage, a set of configurations, and the name of the default configuration.
|
203
|
+
# <tt>storage</tt>:: a method for saving and retrieving buckets (memory, mysql, sqlite3, memcache)
|
204
|
+
# <tt>bucket_configs</tt>:: a hash of name => Config pairs
|
205
|
+
# <tt>default_bucket_config</tt>:: the name of the default config to choose when automatically creating buckets for items that don't have buckets
|
206
|
+
class BucketBased
|
207
|
+
attr_reader :storage
|
208
|
+
def initialize(storage, bucket_configs, default_bucket_config='default')
|
209
|
+
@storage = storage
|
210
|
+
@bucket_configs = bucket_configs
|
211
|
+
if @bucket_configs.keys.length == 1
|
212
|
+
@default_bucket_config = @bucket_configs.keys[0]
|
213
|
+
else
|
214
|
+
@default_bucket_config = default_bucket_config
|
215
|
+
end
|
216
|
+
raise "Cannot find default config" unless @bucket_configs[@default_bucket_config]
|
217
|
+
end
|
218
|
+
|
219
|
+
# Used primarily to preallocate buckets that need an alternate configuration from the default so that they aren't automatically created with default configurations
|
220
|
+
# <tt>name</tt>:: the name of the item to track
|
221
|
+
# <tt>config_name</tt>:: the name of the config to use as a template for creating the bucket
|
222
|
+
# The new bucket will be saved into the storage for this instance of RateLimiter
|
223
|
+
def create_bucket(name, config_name=@default_bucket_config)
|
224
|
+
config = @bucket_configs[config_name]
|
225
|
+
raise "Config is nil" unless config
|
226
|
+
bucket = Bucket.new(name, config.start, config.max, config.min, config.cost, config.refill_amount, config.refill_epoch, Time.now.to_f, 0.0)
|
227
|
+
@storage.set(bucket)
|
228
|
+
end
|
229
|
+
|
230
|
+
# Returns <i>true</i> if the item <tt>name</tt> has enough credits, <i>false</i> otherwise
|
231
|
+
# It will automatically create buckets for items that don't already have buckets and it will do all the bookkeeping to deduct credits, regenerate credits, and track all the credits used.
|
232
|
+
# <tt>name</tt>:: the name of the item to track
|
233
|
+
# <tt>cost</tt>:: the cost of the transaction (defaults to the cost set in the Bucket if nil)
|
234
|
+
def use(name, cost=nil)
|
235
|
+
# create a bucket using the default config if it doesn't already exist
|
236
|
+
bkt = @storage.get(name)
|
237
|
+
unless bkt
|
238
|
+
create_bucket(name)
|
239
|
+
bkt = @storage.get(name)
|
240
|
+
end
|
241
|
+
unless bkt
|
242
|
+
raise Exception, "Could not find bucket"
|
243
|
+
end
|
244
|
+
# first credit the bucket for the time that has elapsed
|
245
|
+
epochs_elapsed = ((Time.now.to_f - bkt.last_refill)/bkt.refill_epoch).to_i
|
246
|
+
bkt.current += epochs_elapsed * bkt.refill_amount
|
247
|
+
bkt.current = bkt.max if bkt.current > bkt.max
|
248
|
+
bkt.last_refill += epochs_elapsed*bkt.refill_epoch
|
249
|
+
# now see if the bkt has enough to provide service
|
250
|
+
cost ||= bkt.cost # if the cost isn't provided, use the default cost
|
251
|
+
raise "Invalid cost: #{cost}" if cost < 0
|
252
|
+
enough = bkt.current >= cost # true if sufficient, false if insufficient
|
253
|
+
# track the total costs, but only if service will be rendered
|
254
|
+
bkt.total_used += cost if enough
|
255
|
+
# now deduct the cost, capping at the minimum
|
256
|
+
bkt.current -= cost
|
257
|
+
bkt.current = bkt.min if bkt.current < bkt.min
|
258
|
+
# now save the changes into the storage (if memory, then no changes are needed, we updated the object in memory)
|
259
|
+
@storage.update(bkt)
|
260
|
+
# return the verdict, did they have enough credits to pay the toll?
|
261
|
+
enough
|
262
|
+
end
|
263
|
+
end
|
264
|
+
end
|
@@ -0,0 +1,31 @@
|
|
1
|
+
# coding: utf-8
|
2
|
+
lib = File.expand_path('../lib', __FILE__)
|
3
|
+
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
|
4
|
+
require 'ratelimit/bucketbased/version'
|
5
|
+
|
6
|
+
Gem::Specification.new do |spec|
|
7
|
+
spec.name = "ratelimit-bucketbased"
|
8
|
+
spec.version = RateLimit::BucketBased::VERSION
|
9
|
+
spec.authors = ["chrislee35"]
|
10
|
+
spec.email = ["rubygems@chrislee.dhs.org"]
|
11
|
+
spec.description = %q{Simple rate limiting gem useful for regulating the speed at which service is provided, this provides an in-memory data structure for administering rate limits}
|
12
|
+
spec.summary = %q{Simple rate limiting gem useful for regulating the speed at which service is provided}
|
13
|
+
spec.homepage = "http://github.com/chrislee35/ratelimit-bucketbased"
|
14
|
+
spec.license = "MIT"
|
15
|
+
|
16
|
+
spec.files = `git ls-files`.split($/)
|
17
|
+
spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
|
18
|
+
spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
|
19
|
+
spec.require_paths = ["lib"]
|
20
|
+
|
21
|
+
spec.add_development_dependency "bundler", "~> 1.3"
|
22
|
+
spec.add_development_dependency "rake"
|
23
|
+
spec.add_development_dependency "sqlite3", ">= 1.3.6"
|
24
|
+
spec.add_development_dependency "memcache", ">= 1.2.13"
|
25
|
+
spec.add_development_dependency "redis", "~> 3.0.1"
|
26
|
+
|
27
|
+
spec.signing_key = "#{File.dirname(__FILE__)}/../gem-private_key.pem"
|
28
|
+
spec.cert_chain = ["#{File.dirname(__FILE__)}/../gem-public_cert.pem"]
|
29
|
+
|
30
|
+
spec.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
|
31
|
+
end
|
data/test/helper.rb
ADDED
data/test/test.db
ADDED
Binary file
|
@@ -0,0 +1,155 @@
|
|
1
|
+
unless Kernel.respond_to?(:require_relative)
|
2
|
+
module Kernel
|
3
|
+
def require_relative(path)
|
4
|
+
require File.join(File.dirname(caller[0]), path.to_str)
|
5
|
+
end
|
6
|
+
end
|
7
|
+
end
|
8
|
+
|
9
|
+
require_relative 'helper'
|
10
|
+
|
11
|
+
def get_configs
|
12
|
+
# set up the configs
|
13
|
+
start = max = 5
|
14
|
+
min = -5
|
15
|
+
cost = refill_amount = refill_epoch = 1
|
16
|
+
configs = {
|
17
|
+
'default' => RateLimit::Config.new('default', start, max, min, cost, refill_amount, refill_epoch),
|
18
|
+
'power' => RateLimit::Config.new('default', 10, 10, min, cost, refill_amount, refill_epoch)
|
19
|
+
}
|
20
|
+
end
|
21
|
+
|
22
|
+
def standard_tests(rl)
|
23
|
+
# add a bucket named "admin", using a non-default configuration
|
24
|
+
rl.create_bucket('admin', 'power')
|
25
|
+
0.upto(4) do
|
26
|
+
assert(rl.use("test"))
|
27
|
+
end
|
28
|
+
0.upto(9) do
|
29
|
+
assert(rl.use("admin"))
|
30
|
+
end
|
31
|
+
assert(! rl.use("test"))
|
32
|
+
assert(! rl.use("admin"))
|
33
|
+
# should be at -2, need two epochs to have enough to do another hit
|
34
|
+
sleep 2
|
35
|
+
assert(rl.use("test"))
|
36
|
+
assert(rl.use("admin"))
|
37
|
+
assert_equal(6,rl.storage.get("test").total_used)
|
38
|
+
assert_equal(11,rl.storage.get("admin").total_used)
|
39
|
+
end
|
40
|
+
|
41
|
+
class TestRateLimitBucketBased < Test::Unit::TestCase
|
42
|
+
|
43
|
+
|
44
|
+
def test_setup_a_rate_limiter_and_work_five_times_before_exhaution
|
45
|
+
#should "setup a rate limiter and work five times before exhaution" do
|
46
|
+
# create a Memory-based storage
|
47
|
+
storage = RateLimit::Memory.new
|
48
|
+
# create the rate limiter, setting the storage, the configurations, and the name of the default configuration
|
49
|
+
rl = RateLimit::BucketBased.new(storage, get_configs, 'default')
|
50
|
+
# run the standard tests
|
51
|
+
standard_tests(rl)
|
52
|
+
end
|
53
|
+
|
54
|
+
def test_raise_an_exception_with_a_cost_of_less_than_0
|
55
|
+
#should "raise an exception with a cost of less than 0" do
|
56
|
+
# set up the configs
|
57
|
+
start = max = 5
|
58
|
+
min = -5
|
59
|
+
cost = refill_amount = refill_epoch = 1
|
60
|
+
configs = {
|
61
|
+
'default' => RateLimit::Config.new('default', start, max, min, cost, refill_amount, refill_epoch),
|
62
|
+
'power' => RateLimit::Config.new('default', 10, 10, min, cost, refill_amount, refill_epoch)
|
63
|
+
}
|
64
|
+
# create a Memory-based storage
|
65
|
+
storage = RateLimit::Memory.new
|
66
|
+
# create the rate limiter, setting the storage, the configurations, and the name of the default configuration
|
67
|
+
rl = RateLimit::BucketBased.new(storage, configs, 'default')
|
68
|
+
assert_raise(RuntimeError) {
|
69
|
+
rl.use("test",-1)
|
70
|
+
}
|
71
|
+
end
|
72
|
+
end
|
73
|
+
|
74
|
+
class TestRateLimitBucketBasedSqlite3 < Test::Unit::TestCase
|
75
|
+
def cleanup
|
76
|
+
File.unlink("test/test.db") if File.exist?("test/test.db")
|
77
|
+
end
|
78
|
+
|
79
|
+
def test_track_the_changes_in_an_SQLite3_database
|
80
|
+
#should "track the changes in an SQLite3 database" do
|
81
|
+
# create a SQLite3-based storage: first create the database, then the table, then the storage
|
82
|
+
File.unlink("test/test.db") if File.exist?("test/test.db")
|
83
|
+
dbh = SQLite3::Database.new( "test/test.db" )
|
84
|
+
assert_not_nil(dbh)
|
85
|
+
res = dbh.execute("CREATE TABLE users (`name`,`current`,`max`,`min`,`cost`,`refill_amount`,`refill_epoch`,`last_refill`,`total_used`,primary key (`name`))")
|
86
|
+
assert(res)
|
87
|
+
storage = RateLimit::SQLite3.new(dbh,'users')
|
88
|
+
|
89
|
+
# create the rate limiter, setting the storage, the configurations, and the name of the default configuration
|
90
|
+
rl = RateLimit::BucketBased.new(storage, get_configs, 'default')
|
91
|
+
# run the standard tests
|
92
|
+
standard_tests(rl)
|
93
|
+
end
|
94
|
+
|
95
|
+
def test_track_the_changes_in_an_SQLite3_database_using_a_custom_field_set
|
96
|
+
#should "track the changes in an SQLite3 database using a custom field-set" do
|
97
|
+
# create a SQLite3-based storage: first create the database, then the table, then the storage
|
98
|
+
File.unlink("test/test.db") if File.exist?("test/test.db")
|
99
|
+
dbh = SQLite3::Database.new( "test/test.db" )
|
100
|
+
assert_not_nil(dbh)
|
101
|
+
fields = ["username","credits","max_credits","min_credits","cost_per_transaction","refill_credits","refill_seconds","last_refill_time","total_used_credits"]
|
102
|
+
res = dbh.execute("CREATE TABLE users (`#{fields.join('`, `')}`,primary key (`username`))")
|
103
|
+
assert(res)
|
104
|
+
storage = RateLimit::SQLite3.new(dbh,'users',fields)
|
105
|
+
|
106
|
+
# create the rate limiter, setting the storage, the configurations, and the name of the default configuration
|
107
|
+
rl = RateLimit::BucketBased.new(storage, get_configs, 'default')
|
108
|
+
# run the standard tests
|
109
|
+
standard_tests(rl)
|
110
|
+
end
|
111
|
+
end
|
112
|
+
|
113
|
+
class TestRateLimitBucketBasedMemcache < Test::Unit::TestCase
|
114
|
+
def setup
|
115
|
+
# create a MemCache-based storage
|
116
|
+
memcache = Memcache.new(:server => 'localhost:11211')
|
117
|
+
memcache.delete("test")
|
118
|
+
memcache.delete("admin")
|
119
|
+
end
|
120
|
+
|
121
|
+
def test_track_the_changes_in_memcache
|
122
|
+
#should "track the changes in memcache" do
|
123
|
+
# create a MemCache-based storage
|
124
|
+
memcache = Memcache.new(:server => 'localhost:11211')
|
125
|
+
assert_not_nil(memcache)
|
126
|
+
storage = RateLimit::Memcache.new(memcache)
|
127
|
+
|
128
|
+
# create the rate limiter, setting the storage, the configurations, and the name of the default configuration
|
129
|
+
rl = RateLimit::BucketBased.new(storage, get_configs, 'default')
|
130
|
+
# run the standard tests
|
131
|
+
standard_tests(rl)
|
132
|
+
end
|
133
|
+
end
|
134
|
+
|
135
|
+
class TestRateLimitBucketBasedRedis < Test::Unit::TestCase
|
136
|
+
def setup
|
137
|
+
# remove stale entries that could conflict with our test
|
138
|
+
redis = Redis.new(:server => 'localhost', :port => 6379)
|
139
|
+
redis.del("test")
|
140
|
+
redis.del("admin")
|
141
|
+
end
|
142
|
+
|
143
|
+
def test_track_the_changes_in_redis
|
144
|
+
#should "track the changes in redis" do
|
145
|
+
# create a Redis-based storage
|
146
|
+
redis = Redis.new(:server => 'localhost', :port => 6379)
|
147
|
+
assert_not_nil(redis)
|
148
|
+
storage = RateLimit::Redis.new(redis)
|
149
|
+
|
150
|
+
# create the rate limiter, setting the storage, the configurations, and the name of the default configuration
|
151
|
+
rl = RateLimit::BucketBased.new(storage, get_configs, 'default')
|
152
|
+
# run the standard tests
|
153
|
+
standard_tests(rl)
|
154
|
+
end
|
155
|
+
end
|
metadata
ADDED
@@ -0,0 +1,177 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: ratelimit-bucketbased
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.0.1
|
5
|
+
prerelease:
|
6
|
+
platform: ruby
|
7
|
+
authors:
|
8
|
+
- chrislee35
|
9
|
+
autorequire:
|
10
|
+
bindir: bin
|
11
|
+
cert_chain:
|
12
|
+
- !binary |-
|
13
|
+
LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURZakNDQWtxZ0F3SUJB
|
14
|
+
Z0lCQURBTkJna3Foa2lHOXcwQkFRVUZBREJYTVJFd0R3WURWUVFEREFoeWRX
|
15
|
+
SjUKWjJWdGN6RVlNQllHQ2dtU0pvbVQ4aXhrQVJrV0NHTm9jbWx6YkdWbE1S
|
16
|
+
TXdFUVlLQ1pJbWlaUHlMR1FCR1JZRApaR2h6TVJNd0VRWUtDWkltaVpQeUxH
|
17
|
+
UUJHUllEYjNKbk1CNFhEVEV6TURVeU1qRXlOVGswTjFvWERURTBNRFV5Ck1q
|
18
|
+
RXlOVGswTjFvd1Z6RVJNQThHQTFVRUF3d0ljblZpZVdkbGJYTXhHREFXQmdv
|
19
|
+
SmtpYUprL0lzWkFFWkZnaGoKYUhKcGMyeGxaVEVUTUJFR0NnbVNKb21UOGl4
|
20
|
+
a0FSa1dBMlJvY3pFVE1CRUdDZ21TSm9tVDhpeGtBUmtXQTI5eQpaekNDQVNJ
|
21
|
+
d0RRWUpLb1pJaHZjTkFRRUJCUUFEZ2dFUEFEQ0NBUW9DZ2dFQkFOY1ByeDhC
|
22
|
+
WmlXSVI5eFdXRzhJCnRxUjUzOHRTMXQrVUo0RlpGbCsxdnJ0VTlUaXVXWDNW
|
23
|
+
ajM3VHdVcGEyZkZremlLMG41S3VwVlRoeUVoY2VtNW0KT0dSanZnclJGYldR
|
24
|
+
SlNTc2NJS09wd3FVUkhWS1JwVjlnVnovSG56azhTK3hvdFVSMUJ1bzNVZ3Ir
|
25
|
+
STFqSGV3RApDZ3IreSt6Z1pidGp0SHNKdHN1dWprT2NQaEVqalVpbmo2OEw5
|
26
|
+
Rno5QmRlSlF0K0lhY2p3QXpVTGl4NmpXQ2h0ClVjK2crMHo4RXNyeWNhMkc2
|
27
|
+
STFHc3JnWDZXSHc4ZHlreVFEVDlkQ3RTMmZsQ093U0MxUjBLNVQveEhXNTRm
|
28
|
+
KzUKd2N3OG1tNTNLTE5lK3RtZ1ZDNlpIeU1FK3FKc0JuUDZ1eEYwYVRFbkdB
|
29
|
+
L2pEQlFEaFFOVEYwWlAvYWJ6eVRzTAp6alVDQXdFQUFhTTVNRGN3Q1FZRFZS
|
30
|
+
MFRCQUl3QURBTEJnTlZIUThFQkFNQ0JMQXdIUVlEVlIwT0JCWUVGTzh3Cith
|
31
|
+
ZVA3VDZrVkpibENnNmV1c09JSTlEZk1BMEdDU3FHU0liM0RRRUJCUVVBQTRJ
|
32
|
+
QkFRQkNReVJKTFhzQm8yRnkKOFc2ZS9XNFJlbVFScmxBdzlESzVPNlU3MUp0
|
33
|
+
ZWRWb2Iyb3ErT2Irem1TK1BpZkUyK0wrM1JpSjJINlZUbE96aQp4K0EwNjFN
|
34
|
+
VVhoR3JhcVZxNEoyRkM4a3Q0RVF5d0FEMFAwVGE1R1UyNENHU0YwOFkzR2tK
|
35
|
+
eTFTYTRYcVRDMllDCm81MXM3SlArdGtDQ3RwVllTZHpKaFRsbGllUkFXQnBH
|
36
|
+
VjFkdGFvZVVLRTZ0WVBNQmtvc3hTUmNWR2N6ay9TYzMKN2VRQ3BleFl5OUps
|
37
|
+
VUJJOXUzQnFJWTlFK2wrTVNuOGloWFNQbXlLMERncmhhQ3Urdm9hU0ZWT1g2
|
38
|
+
WStCNXFibwpqTFhNUXUyWmdJU1l3WE5qTmJHVkhlaHV0ODJVN1U5b2lIb1dj
|
39
|
+
ck9HYXphUlVtR085VFhQK2FKTEgwZ3cyZGNLCkFmTWdsWFBpCi0tLS0tRU5E
|
40
|
+
IENFUlRJRklDQVRFLS0tLS0K
|
41
|
+
date: 2013-06-02 00:00:00.000000000 Z
|
42
|
+
dependencies:
|
43
|
+
- !ruby/object:Gem::Dependency
|
44
|
+
name: bundler
|
45
|
+
requirement: !ruby/object:Gem::Requirement
|
46
|
+
none: false
|
47
|
+
requirements:
|
48
|
+
- - ~>
|
49
|
+
- !ruby/object:Gem::Version
|
50
|
+
version: '1.3'
|
51
|
+
type: :development
|
52
|
+
prerelease: false
|
53
|
+
version_requirements: !ruby/object:Gem::Requirement
|
54
|
+
none: false
|
55
|
+
requirements:
|
56
|
+
- - ~>
|
57
|
+
- !ruby/object:Gem::Version
|
58
|
+
version: '1.3'
|
59
|
+
- !ruby/object:Gem::Dependency
|
60
|
+
name: rake
|
61
|
+
requirement: !ruby/object:Gem::Requirement
|
62
|
+
none: false
|
63
|
+
requirements:
|
64
|
+
- - ! '>='
|
65
|
+
- !ruby/object:Gem::Version
|
66
|
+
version: '0'
|
67
|
+
type: :development
|
68
|
+
prerelease: false
|
69
|
+
version_requirements: !ruby/object:Gem::Requirement
|
70
|
+
none: false
|
71
|
+
requirements:
|
72
|
+
- - ! '>='
|
73
|
+
- !ruby/object:Gem::Version
|
74
|
+
version: '0'
|
75
|
+
- !ruby/object:Gem::Dependency
|
76
|
+
name: sqlite3
|
77
|
+
requirement: !ruby/object:Gem::Requirement
|
78
|
+
none: false
|
79
|
+
requirements:
|
80
|
+
- - ! '>='
|
81
|
+
- !ruby/object:Gem::Version
|
82
|
+
version: 1.3.6
|
83
|
+
type: :development
|
84
|
+
prerelease: false
|
85
|
+
version_requirements: !ruby/object:Gem::Requirement
|
86
|
+
none: false
|
87
|
+
requirements:
|
88
|
+
- - ! '>='
|
89
|
+
- !ruby/object:Gem::Version
|
90
|
+
version: 1.3.6
|
91
|
+
- !ruby/object:Gem::Dependency
|
92
|
+
name: memcache
|
93
|
+
requirement: !ruby/object:Gem::Requirement
|
94
|
+
none: false
|
95
|
+
requirements:
|
96
|
+
- - ! '>='
|
97
|
+
- !ruby/object:Gem::Version
|
98
|
+
version: 1.2.13
|
99
|
+
type: :development
|
100
|
+
prerelease: false
|
101
|
+
version_requirements: !ruby/object:Gem::Requirement
|
102
|
+
none: false
|
103
|
+
requirements:
|
104
|
+
- - ! '>='
|
105
|
+
- !ruby/object:Gem::Version
|
106
|
+
version: 1.2.13
|
107
|
+
- !ruby/object:Gem::Dependency
|
108
|
+
name: redis
|
109
|
+
requirement: !ruby/object:Gem::Requirement
|
110
|
+
none: false
|
111
|
+
requirements:
|
112
|
+
- - ~>
|
113
|
+
- !ruby/object:Gem::Version
|
114
|
+
version: 3.0.1
|
115
|
+
type: :development
|
116
|
+
prerelease: false
|
117
|
+
version_requirements: !ruby/object:Gem::Requirement
|
118
|
+
none: false
|
119
|
+
requirements:
|
120
|
+
- - ~>
|
121
|
+
- !ruby/object:Gem::Version
|
122
|
+
version: 3.0.1
|
123
|
+
description: Simple rate limiting gem useful for regulating the speed at which service
|
124
|
+
is provided, this provides an in-memory data structure for administering rate limits
|
125
|
+
email:
|
126
|
+
- rubygems@chrislee.dhs.org
|
127
|
+
executables: []
|
128
|
+
extensions: []
|
129
|
+
extra_rdoc_files: []
|
130
|
+
files:
|
131
|
+
- .gitignore
|
132
|
+
- Gemfile
|
133
|
+
- LICENSE.txt
|
134
|
+
- README.md
|
135
|
+
- Rakefile
|
136
|
+
- lib/ratelimit/bucketbased.rb
|
137
|
+
- lib/ratelimit/bucketbased/version.rb
|
138
|
+
- ratelimit-bucketbased.gemspec
|
139
|
+
- test/helper.rb
|
140
|
+
- test/test.db
|
141
|
+
- test/test_ratelimit-bucketbased.rb
|
142
|
+
homepage: http://github.com/chrislee35/ratelimit-bucketbased
|
143
|
+
licenses:
|
144
|
+
- MIT
|
145
|
+
post_install_message:
|
146
|
+
rdoc_options: []
|
147
|
+
require_paths:
|
148
|
+
- lib
|
149
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
150
|
+
none: false
|
151
|
+
requirements:
|
152
|
+
- - ! '>='
|
153
|
+
- !ruby/object:Gem::Version
|
154
|
+
version: '0'
|
155
|
+
segments:
|
156
|
+
- 0
|
157
|
+
hash: 711855890028664277
|
158
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
159
|
+
none: false
|
160
|
+
requirements:
|
161
|
+
- - ! '>='
|
162
|
+
- !ruby/object:Gem::Version
|
163
|
+
version: '0'
|
164
|
+
segments:
|
165
|
+
- 0
|
166
|
+
hash: 711855890028664277
|
167
|
+
requirements: []
|
168
|
+
rubyforge_project:
|
169
|
+
rubygems_version: 1.8.25
|
170
|
+
signing_key:
|
171
|
+
specification_version: 3
|
172
|
+
summary: Simple rate limiting gem useful for regulating the speed at which service
|
173
|
+
is provided
|
174
|
+
test_files:
|
175
|
+
- test/helper.rb
|
176
|
+
- test/test.db
|
177
|
+
- test/test_ratelimit-bucketbased.rb
|
metadata.gz.sig
ADDED
Binary file
|