lockistics 0.1.0

Sign up to get free protection for your applications and to get access to all the features.
data/.gitignore ADDED
@@ -0,0 +1,17 @@
1
+ *.gem
2
+ *.rbc
3
+ .bundle
4
+ .config
5
+ .yardoc
6
+ Gemfile.lock
7
+ InstalledFiles
8
+ _yardoc
9
+ coverage
10
+ doc/
11
+ lib/bundler/man
12
+ pkg
13
+ rdoc
14
+ spec/reports
15
+ test/tmp
16
+ test/version_tmp
17
+ tmp
data/Gemfile ADDED
@@ -0,0 +1,4 @@
1
+ source 'https://rubygems.org'
2
+
3
+ # Specify your gem's dependencies in lockistics.gemspec
4
+ gemspec
data/LICENSE ADDED
@@ -0,0 +1,24 @@
1
+ This is free and unencumbered software released into the public domain.
2
+
3
+ Anyone is free to copy, modify, publish, use, compile, sell, or
4
+ distribute this software, either in source code form or as a compiled
5
+ binary, for any purpose, commercial or non-commercial, and by any
6
+ means.
7
+
8
+ In jurisdictions that recognize copyright laws, the author or authors
9
+ of this software dedicate any and all copyright interest in the
10
+ software to the public domain. We make this dedication for the benefit
11
+ of the public at large and to the detriment of our heirs and
12
+ successors. We intend this dedication to be an overt act of
13
+ relinquishment in perpetuity of all present and future rights to this
14
+ software under copyright law.
15
+
16
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
17
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
18
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
19
+ IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
20
+ OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
21
+ ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
22
+ OTHER DEALINGS IN THE SOFTWARE.
23
+
24
+ For more information, please refer to <http://unlicense.org>
data/LICENSE.txt ADDED
@@ -0,0 +1,22 @@
1
+ Copyright (c) 2014 Kimmo Lehto
2
+
3
+ MIT License
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining
6
+ a copy of this software and associated documentation files (the
7
+ "Software"), to deal in the Software without restriction, including
8
+ without limitation the rights to use, copy, modify, merge, publish,
9
+ distribute, sublicense, and/or sell copies of the Software, and to
10
+ permit persons to whom the Software is furnished to do so, subject to
11
+ the following conditions:
12
+
13
+ The above copyright notice and this permission notice shall be
14
+ included in all copies or substantial portions of the Software.
15
+
16
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
17
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
18
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
19
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
20
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
21
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
22
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,170 @@
1
+ # Lockistics
2
+
3
+ Lockistics is basically a distributed mutex on Redis with statistics collecting included.
4
+
5
+ The likely use case for locking would be something like a Raketask that you don't want running multiple instances at once.
6
+
7
+ The likely use case for the statistics part would be that you want to how often something is being called or if a certain Raketask has been run today or not. You can also use it to find memory leaks or slow methods, kind of private NewRelic with zero features.
8
+
9
+ ## Installation
10
+
11
+ Add this line to your application's Gemfile:
12
+
13
+ gem 'lockistics'
14
+
15
+ And then execute:
16
+
17
+ $ bundle
18
+
19
+ Or install it yourself as:
20
+
21
+ $ gem install lockistics
22
+
23
+ #### Redis
24
+
25
+ Notice that you need a Redis v2.6.2+ as this gem uses LUA for race condition safe lock acquiring and min/max setting
26
+
27
+ ## Usage
28
+
29
+ You can use both parts separately if you just want to collect statistics or to just do simple locking.
30
+
31
+ Total, daily and hourly metrics you get for each key are:
32
+
33
+ - Number of locks
34
+ - Number of times having to wait for lock
35
+ - Number of failed locking attempts
36
+ - Minimum and maximum duration
37
+ - Minimum and maximum memory growth (using OS gem)
38
+ - Arbitary metrics you add during execution (more on this in examples)
39
+
40
+ ## Why?
41
+
42
+ Convenience mostly. There are redis-locking gems and some quite complex statistics modules, this does both with minimum dependencies, easy usage and Ruby 1.8.7 support.
43
+
44
+ ## Examples
45
+
46
+ #### Configure the gem
47
+
48
+ These are the default settings :
49
+
50
+ ```ruby
51
+ Lockistics.configure do |config|
52
+ config.redis = Redis.new
53
+ config.namespace = "lockistics"
54
+ config.expire = 300 # seconds
55
+ config.sleep = 0.5 # seconds to sleep between retries
56
+ config.retries = 10 # retry times
57
+ config.raise = true # raise Lockistics::TimeoutException when lock fails
58
+ end
59
+ ```
60
+
61
+ #### Getting and using a lock
62
+
63
+ ```ruby
64
+ # Get a lock, do what you must, release lock. No statistics collection.
65
+ Lockistics.lock("generate-stuff-raketask") do
66
+ doing_some_heavy_stuff
67
+ end
68
+ ```
69
+
70
+ ```ruby
71
+ # Some raketask that you don't want to run multiple times at once :
72
+ namespace :raketask
73
+ desc 'Generate stuff'
74
+ task :generate_stuff do
75
+ return nil unless Lockistics.lock("generate-stuff", :wait => false)
76
+ ...
77
+ end
78
+ end
79
+ ```
80
+
81
+ ```ruby
82
+ # Handle exception when you fail to acquire a lock in time:
83
+ begin
84
+ Lockistics.lock("stuff") do
85
+ ...
86
+ end
87
+ rescue Lockistics::Timeout
88
+ ...
89
+ end
90
+ ```
91
+
92
+ ```ruby
93
+ # Don't raise exceptions
94
+ Lockistics.lock("stuff", :raise => false) do
95
+ ...
96
+ end
97
+ ```
98
+
99
+ #### Statistics collection without locking
100
+
101
+ It works exactly like the locking, but the method is `meter`.
102
+
103
+ ```ruby
104
+ # Perform something, statistics will be collected behind the scenes.
105
+ Lockistics.meter("generate-stuff-raketask") do
106
+ doing_some_stuff
107
+ end
108
+ ```
109
+
110
+ ```ruby
111
+ # Adding custom metrics
112
+ Lockistics.meter("generate-stuff") do |meter|
113
+ results = do_stuff
114
+ if results.empty?
115
+ meter.incr "empty_results"
116
+ else
117
+ meter.incrby "stuffs_done", results.size
118
+ end
119
+ end
120
+ ```
121
+
122
+ #### Statistics collection with locking
123
+
124
+ It works exactly like the above, but the method is `meterlock`.
125
+
126
+ ```ruby
127
+ # Adding custom metrics
128
+ Lockistics.meterlock("generate-stuff", :raise => false) do |meter|
129
+ results = do_stuff
130
+ if results.empty?
131
+ meter.incr "empty_results"
132
+ else
133
+ # Sets min and/or max for a key (min.stuffs_done + max.stuffs_done)
134
+ # Only sets if value is minimum or maximum for the periods.
135
+ meter.set_minmax "stuffs_done", results.size
136
+ end
137
+ end
138
+ ```
139
+
140
+ #### Getting the statistics out
141
+
142
+ You can query statistics for locking/metering keys.
143
+
144
+ ```ruby
145
+ stats = Lockistics.statistics('generate'stuff')
146
+ # Get the last run
147
+ stats.last_run
148
+ => Mon May 19 16:38:52 +0300 2014
149
+ # Get totals:
150
+ stats.total
151
+ => {"invocations" => 50, "lock-timeouts" => 1,
152
+ "max.stuffs-generated" => 10, "max_rss" => 400 ..}
153
+ stats.daily
154
+ => [{:time => #<Time..> "invocations" => 50, "lock-timeouts" => 1,
155
+ "max.stuffs-generated" => 10, "max_rss" => 400 ..}, {..}]
156
+ stats.hourly
157
+ => [{:time => #<Time..> "invocations" => 50, "lock-timeouts" => 1,
158
+ "max.stuffs-generated" => 10, "max_rss" => 400 ..}, {..}]
159
+ stats.all
160
+ => { :daily => [...], :hourly => [...], :total => {...},
161
+ :last_run:time => #<Time..>}
162
+ ```
163
+
164
+ ## Contributing
165
+
166
+ 1. Fork it
167
+ 2. Create your feature branch (`git checkout -b my-new-feature`)
168
+ 3. Commit your changes (`git commit -am 'Add some feature'`)
169
+ 4. Push to the branch (`git push origin my-new-feature`)
170
+ 5. Create new Pull Request
data/Rakefile ADDED
@@ -0,0 +1,3 @@
1
+ require "bundler/gem_tasks"
2
+ require 'rspec/core/rake_task'
3
+ RSpec::Core::RakeTask.new(:spec)
@@ -0,0 +1,34 @@
1
+ require 'redis'
2
+
3
+ module Lockistics
4
+ class Configuration
5
+
6
+ attr_accessor :redis
7
+ attr_accessor :namespace
8
+ attr_accessor :logger
9
+ attr_accessor :expire
10
+ attr_accessor :sleep
11
+ attr_accessor :retries
12
+ attr_accessor :raise
13
+
14
+ def initialize
15
+ @redis = Redis.new
16
+ @namespace = 'lockistics'
17
+ @expire = 10
18
+ @sleep = 0.5
19
+ @retries = 10
20
+ end
21
+
22
+ def lock_defaults
23
+ {
24
+ :redis => redis,
25
+ :namespace => namespace,
26
+ :expire => expire,
27
+ :sleep => sleep,
28
+ :retries => retries,
29
+ :wait => true,
30
+ :raise => true
31
+ }
32
+ end
33
+ end
34
+ end
@@ -0,0 +1,67 @@
1
+ module Lockistics
2
+
3
+ class LockTimeout < StandardError; end
4
+
5
+ class Lock
6
+
7
+ attr_accessor :key, :options, :lock_retries
8
+
9
+ LUA_ACQUIRE = "return redis.call('setnx', KEYS[1], 1) == 1 and redis.call('expire', KEYS[1], KEYS[2]) and 1 or 0"
10
+
11
+ def initialize(key, options={})
12
+ @key = key
13
+ @options = Lockistics.configuration.lock_defaults.merge(options)
14
+ @options[:expire] = 999_999_999 unless @options[:expire].to_i > 0 # :expire => false
15
+ @exceeded_before_release = false
16
+ @lock_retries = 0
17
+ end
18
+
19
+ def acquire_lock
20
+ Lockistics.known_keys(key)
21
+ if got_lock?
22
+ true
23
+ elsif options[:wait]
24
+ wait_for_lock
25
+ else
26
+ false
27
+ end
28
+ end
29
+
30
+ def wait_for_lock
31
+ until got_lock?
32
+ @lock_retries += 1
33
+ if lock_retries <= options[:retries]
34
+ sleep options[:sleep]
35
+ elsif options[:raise]
36
+ raise LockTimeout, "while waiting for #{key}"
37
+ else
38
+ return false
39
+ end
40
+ end
41
+ true
42
+ end
43
+
44
+ def exceeded_before_release?
45
+ @exceeded_before_release
46
+ end
47
+
48
+ def release_lock
49
+ @exceeded_before_release = redis.del(namespaced_key) == 0
50
+ end
51
+
52
+ def namespaced_key
53
+ @namespaced_key ||= "#{Lockistics.configuration.namespace}.lock.#{key}"
54
+ end
55
+
56
+ private
57
+
58
+ def redis
59
+ Lockistics.redis
60
+ end
61
+
62
+ def got_lock?
63
+ redis.eval(LUA_ACQUIRE, [namespaced_key, options[:expire]]) == 1
64
+ end
65
+
66
+ end
67
+ end
@@ -0,0 +1,165 @@
1
+ require 'os'
2
+
3
+ module Lockistics
4
+ class Meter
5
+
6
+ attr_accessor :key, :options
7
+
8
+ LUA_SETMAX = <<-EOB.gsub("\n", " ").gsub(/\s+/, " ")
9
+ if redis.call("hexists", KEYS[1], KEYS[2]) == 0 or
10
+ tonumber(redis.call("hget", KEYS[1], KEYS[2])) < tonumber(ARGV[1])
11
+ then
12
+ return redis.call("hset", KEYS[1], KEYS[2], ARGV[1])
13
+ else
14
+ return 0
15
+ end
16
+ EOB
17
+
18
+ LUA_SETMIN = <<-EOB.gsub("\n", " ").gsub(/\s+/, " ")
19
+ if redis.call("hexists", KEYS[1], KEYS[2]) == 0 or
20
+ tonumber(redis.call("hget", KEYS[1], KEYS[2])) > tonumber(ARGV[1])
21
+ then
22
+ return redis.call("hset", KEYS[1], KEYS[2], ARGV[1])
23
+ else
24
+ return 0
25
+ end
26
+ EOB
27
+
28
+ def initialize(key, options={})
29
+ @key = key
30
+ @options = options
31
+ @lock_timeouts = 0
32
+ end
33
+
34
+ def with_lock(&block)
35
+ raise ArgumentError, "with_lock called without block" unless block_given?
36
+ raise ArgumentError, "lock not defined" if lock.nil?
37
+ lock.acquire_lock
38
+ yield self
39
+ ensure
40
+ lock.release_lock
41
+ end
42
+
43
+ def perform(&block)
44
+ raise ArgumentError, "perform called without block" unless block_given?
45
+ before_perform
46
+ lock.nil? ? yield(self) : with_lock(&block)
47
+ rescue Lockistics::LockTimeout
48
+ @lock_timeouts = 1
49
+ raise
50
+ ensure
51
+ after_perform
52
+ end
53
+
54
+ # You can add custom metrics during runtime
55
+ #
56
+ # @example
57
+ # Lockistics.meter do |meter|
58
+ # foo = FooGenerator.new
59
+ # foo.perform
60
+ # meter.incrby 'foos-generated', foo.count
61
+ # end
62
+ def incrby(key, value)
63
+ return nil if value == 0
64
+ [:hourly, :daily, :total].each do |period|
65
+ redis.hincrby namespaced_hash(period), key, value
66
+ end
67
+ end
68
+
69
+ # You can add custom metrics during runtime with
70
+ # this.
71
+ #
72
+ # This is a shortcut to incrby(key, 1)
73
+ #
74
+ # @example
75
+ # Lockistics.meter do |meter|
76
+ # foo = FooGenerator.new
77
+ # foo.perform
78
+ # meter.incr 'failed-foo-generations' unless foo.success?
79
+ # end
80
+ def incr(key)
81
+ incrby(key, 1)
82
+ end
83
+
84
+ def set_minmax(key, value)
85
+ [:hourly, :daily, :total].each do |period|
86
+ redis_hsetmax(namespaced_hash(period), "max.#{key}", value)
87
+ redis_hsetmin(namespaced_hash(period), "min.#{key}", value)
88
+ end
89
+ end
90
+
91
+ private
92
+
93
+ def redis
94
+ Lockistics.configuration.redis
95
+ end
96
+
97
+ def redis_hsetmax(hash, key, value)
98
+ redis.eval(LUA_SETMAX, [hash, key], [value]) == 1
99
+ end
100
+
101
+ def redis_hsetmin(hash, key, value)
102
+ redis.eval(LUA_SETMIN, [hash, key], [value]) == 1
103
+ end
104
+
105
+ def lock
106
+ options[:lock]
107
+ end
108
+
109
+ def before_perform
110
+ Lockistics.known_keys(key)
111
+ @start_time = Time.now.to_f
112
+ @start_rss = OS.rss_bytes
113
+ redis.pipelined do
114
+ redis.sadd "#{Lockistics.configuration.namespace}.#{key}.hourlies", hourly_timestamp
115
+ redis.sadd "#{Lockistics.configuration.namespace}.#{key}.dailies", daily_timestamp
116
+ redis.set "#{Lockistics.configuration.namespace}.#{key}.last_run", Time.now.to_i
117
+ end
118
+ end
119
+
120
+ def after_perform
121
+ unless @lock_timeouts > 0
122
+ @duration = ((Time.now.to_f - @start_time) * 1000).round
123
+ @rss_increase = ((OS.rss_bytes - @start_rss) / 1024).round
124
+ end
125
+ add_meter_statistics unless options[:no_metrics]
126
+ add_lock_statistics unless lock.nil?
127
+ end
128
+
129
+ def add_meter_statistics
130
+ incrby 'invocations', 1
131
+ set_minmax 'rss', @rss_increase unless @rss_increase.nil?
132
+ set_minmax 'time', @duration unless @duration.nil?
133
+ end
134
+
135
+ def add_lock_statistics
136
+ redis.pipelined do
137
+ incrby 'lock-invocations', 1
138
+ incrby 'lock-retries', lock.lock_retries
139
+ incrby 'lock-timeouts', @lock_timeouts
140
+ if lock.exceeded_before_release?
141
+ incrby 'lock-exceeded-before-release', 1
142
+ end
143
+ end
144
+ end
145
+
146
+ def hourly_timestamp
147
+ @hourly_timestamp ||= Time.now.strftime("%Y%m%d%H")
148
+ end
149
+
150
+ def daily_timestamp
151
+ @daily_timestamp ||= Time.now.strftime("%Y%m%d")
152
+ end
153
+
154
+ def namespaced_hash(period)
155
+ case period
156
+ when :hourly
157
+ "#{Lockistics.configuration.namespace}.#{key}.hourly.#{hourly_timestamp}"
158
+ when :daily
159
+ "#{Lockistics.configuration.namespace}.#{key}.daily.#{daily_timestamp}"
160
+ when :total
161
+ "#{Lockistics.configuration.namespace}.#{key}.total"
162
+ end
163
+ end
164
+ end
165
+ end
@@ -0,0 +1,89 @@
1
+ require 'time'
2
+ module Lockistics
3
+ class Statistics
4
+
5
+ attr_accessor :key
6
+
7
+ def initialize(key)
8
+ @key = key
9
+ end
10
+
11
+ def last_run
12
+ unix_time = redis.get(namespaced("#{key}.last_run"))
13
+ if unix_time
14
+ Time.at(unix_time.to_i)
15
+ end
16
+ end
17
+
18
+ def all(since=nil)
19
+ {
20
+ :daily => daily(since),
21
+ :hourly => hourly(since),
22
+ :total => total,
23
+ :last_run => last_run
24
+ }
25
+ end
26
+
27
+ def daily(since=nil)
28
+ daily_hashes(since).collect{|stamped| redis.hgetall(stamped.first).merge(:time => stamped.last )}
29
+ end
30
+
31
+ def hourly(since=nil)
32
+ hourly_hashes(since).collect{|stamped| redis.hgetall(stamped.first).merge(:time => stamped.last)}
33
+ end
34
+
35
+ def total
36
+ redis.hgetall total_hash
37
+ end
38
+
39
+ private
40
+
41
+ def date_to_hourly_stamp(date=nil)
42
+ if date.nil?
43
+ 0
44
+ elsif date.respond_to?(strftime)
45
+ date.strftime("%Y%m%d%H")
46
+ elsif date.match(/^\d+$/)
47
+ Time.at(date).strftime("%Y%m%d%H")
48
+ end
49
+ end
50
+
51
+ def date_to_daily_stamp(date=nil)
52
+ if date.nil?
53
+ 0
54
+ elsif date.respond_to?(strftime)
55
+ date.strftime("%Y%m%d")
56
+ elsif date.match(/^\d+$/)
57
+ Time.at(date).strftime("%Y%m%d")
58
+ end
59
+ end
60
+
61
+ def redis
62
+ Lockistics.redis
63
+ end
64
+
65
+ def daily_hashes(since=nil)
66
+ stamped_ts = date_to_daily_stamp(since)
67
+ redis.smembers(namespaced("#{key}.dailies")).collect do |ts|
68
+ next if ts.to_i < stamped_ts
69
+ [namespaced("#{key}.daily.#{ts}"), Time.parse(ts)]
70
+ end
71
+ end
72
+
73
+ def hourly_hashes(since=nil)
74
+ stamped_ts = date_to_hourly_stamp(since)
75
+ redis.smembers(namespaced("#{key}.hourlies")).collect do |ts|
76
+ next if ts.to_i < stamped_ts
77
+ [namespaced("#{key}.hourly.#{ts}"), Time.parse(ts)]
78
+ end
79
+ end
80
+
81
+ def total_hash
82
+ namespaced "#{key}.total"
83
+ end
84
+
85
+ def namespaced(extra_key=nil)
86
+ "#{Lockistics.configuration.namespace}#{extra_key.nil? ? "" : ".#{extra_key}"}"
87
+ end
88
+ end
89
+ end
@@ -0,0 +1,3 @@
1
+ module Lockistics
2
+ VERSION = "0.1.0"
3
+ end
data/lib/lockistics.rb ADDED
@@ -0,0 +1,137 @@
1
+ require "lockistics/version"
2
+ require "lockistics/configuration"
3
+ require "lockistics/lock"
4
+ require "lockistics/meter"
5
+ require "lockistics/statistics"
6
+
7
+ # Lockistics is basically a distributed mutex on Redis.
8
+ #
9
+ # In addition to locking it also collects statistics data
10
+ # of the locking events.
11
+ #
12
+ # You can use each part separately if you just want to
13
+ # collect statistics or to do simple locking.
14
+ #
15
+ # Total, daily and hourly metrics you get for each key are:
16
+ # - Number of locks
17
+ # - Number of times having to wait for lock
18
+ # - Number of failed locking attempts
19
+ # - Minimum and maximum duration
20
+ # - Minimum and maximum memory growth (using OS gem)
21
+ # - Arbitary metrics you add during execution
22
+ module Lockistics
23
+
24
+ # Configure the gem
25
+ #
26
+ # @example
27
+ # Lockistics.configure do |config|
28
+ # config.redis = Redis.new
29
+ # config.namespace = "production.locks"
30
+ # config.expire = 300 # seconds
31
+ # config.sleep = 0.5 # seconds to sleep between retries
32
+ # config.retries = 10 # retry times
33
+ # config.raise = true # raise Lockistics::TimeoutException when lock fails
34
+ # end
35
+ def self.configure(&block)
36
+ yield configuration
37
+ end
38
+
39
+ # Returns an instance of Lockistics::Configuration
40
+ def self.configuration
41
+ @configuration ||= Configuration.new
42
+ end
43
+
44
+ # Accessor to the redis configured via Lockistics::Configuration
45
+ def self.redis
46
+ configuration.redis
47
+ end
48
+
49
+ # Get a hold of a lock or wait for it to be released
50
+ #
51
+ # Given a block, will release the lock after block exection
52
+ #
53
+ # @example
54
+ # Lockistics.lock("generate-stuff-raketask") do
55
+ # doing_some_heavy_stuff
56
+ # end
57
+ # or
58
+ # return nil unless Lockistics.lock("generate-stuff", :wait => false)
59
+ # or
60
+ # begin
61
+ # Lockistics.lock("stuff") do
62
+ # ...
63
+ # end
64
+ # rescue Lockistics::Timeout
65
+ # ...
66
+ # end
67
+ def self.lock(key, options={}, &block)
68
+ if block_given?
69
+ Meter.new(key, options.merge(:no_metrics => true)).perform(&block)
70
+ else
71
+ Lock.new(key, options).acquire_lock
72
+ end
73
+ end
74
+
75
+ # Don't perform locking, just collect metrics.
76
+ #
77
+ # @example
78
+ # Lockistics.meter("generate-stuff") do |meter|
79
+ # do_stuff
80
+ # meter.incrby :stuffs_generated, 50
81
+ # end
82
+ def self.meter(key, options={}, &block)
83
+ Meter.new(key, options.merge(:lock => nil)).perform(&block)
84
+ end
85
+
86
+ # Perform locking and statistics collection
87
+ #
88
+ # @example
89
+ # Lockistics.meterlock("generate-stuff") do |meter|
90
+ # results = do_stuff
91
+ # if results.empty?
92
+ # meter.incr "empty_results"
93
+ # else
94
+ # meter.incrby "stuffs_done", results.size
95
+ # end
96
+ # end
97
+ def self.meterlock(key, options={}, &block)
98
+ Meter.new(key, options.merge(:lock => Lock.new(key, options))).perform(&block)
99
+ end
100
+
101
+ # Manually release a lock
102
+ #
103
+ # @example
104
+ # Lockistics.release("generate-stuff")
105
+ # or
106
+ # lock = Lockistics.lock("generate-stuff", :wait => false)
107
+ # if lock.acquire_lock
108
+ # do_some_stuff
109
+ # lock.release_lock
110
+ # end
111
+ def self.release(key, options={})
112
+ Lock.new(key).release_lock
113
+ end
114
+
115
+ # Returns an instance of Lockistics::Statistics for the key.
116
+ #
117
+ # @example
118
+ # stats = Lockistics.statistics('generate-stuff')
119
+ # stats.last_run
120
+ # => Mon May 19 16:38:52 +0300 2014
121
+ # stats.total
122
+ # => {"invocations" => 50, ...}
123
+ def self.statistics(key)
124
+ Statistics.new(key)
125
+ end
126
+
127
+ # Get a list of known keys or add a key to
128
+ # known keys list.
129
+ def self.known_keys(key=nil)
130
+ if key.nil?
131
+ redis.smembers "#{Lockistics.configuration.namespace}.known_keys"
132
+ else
133
+ redis.sadd "#{Lockistics.configuration.namespace}.known_keys", key
134
+ end
135
+ end
136
+
137
+ end
@@ -0,0 +1,28 @@
1
+ # coding: utf-8
2
+ lib = File.expand_path('../lib', __FILE__)
3
+ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
4
+ require 'lockistics/version'
5
+
6
+ Gem::Specification.new do |spec|
7
+ spec.name = "lockistics"
8
+ spec.version = Lockistics::VERSION
9
+ spec.authors = ["Kimmo Lehto"]
10
+ spec.email = ["kimmo.lehto@gmail.com"]
11
+ spec.description = %q{Statsistics collecting locking}
12
+ spec.summary = %q{With lockistics you can use Redis to create distributed locks and collect statistics how often and how long your locks are held}
13
+ spec.homepage = "https://github.com/kke/lockistics"
14
+ spec.license = "MIT"
15
+
16
+ spec.files = `git ls-files`.split($/)
17
+ spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
18
+ spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
19
+ spec.require_paths = ["lib"]
20
+
21
+ spec.add_runtime_dependency "redis"
22
+ spec.add_runtime_dependency "os"
23
+
24
+ spec.add_development_dependency "bundler", "~> 1.3"
25
+ spec.add_development_dependency "rake"
26
+ spec.add_development_dependency "rspec"
27
+ spec.add_development_dependency "fakeredis"
28
+ end
@@ -0,0 +1,53 @@
1
+ require File.expand_path('../spec_helper.rb', __FILE__)
2
+
3
+ describe "Lockistics.lock" do
4
+ it "should only allow one instance to lock something at once" do
5
+ lock1 = Lockistics.lock("ltest1", :wait => false, :raise => false, :expire => 100)
6
+ sleep 1
7
+ lock2 = Lockistics.lock("ltest1", :wait => false, :raise => false, :expire => 100)
8
+ lock1.should be_true
9
+ lock2.should be_false
10
+ Lockistics.release("ltest1")
11
+ lock2 = Lockistics.lock("ltest1", :wait => false, :raise => false, :expire => 100)
12
+ lock2.should be_true
13
+ end
14
+
15
+ it "should allow locking again after block mode" do
16
+ lock1_ok = false
17
+ lock2_ok = false
18
+ lock1 = Lockistics.lock("ltest1", :wait => false, :raise => false, :expire => 100) do
19
+ lock1_ok = true
20
+ sleep 1
21
+ end
22
+ lock2 = Lockistics.lock("ltest1", :wait => false, :raise => false, :expire => 100) do
23
+ lock2_ok = true
24
+ end
25
+ lock1_ok.should be_true
26
+ lock2_ok.should be_true
27
+ Lockistics.release("ltest1")
28
+ end
29
+
30
+ it "should allow locking different keys at once" do
31
+ lock1_ok = false
32
+ lock2_ok = false
33
+ lock1 = Lockistics.lock("ltest1", :wait => false, :raise => false, :expire => 100) do
34
+ lock1_ok = true
35
+ lock2 = Lockistics.lock("ltest2", :wait => false, :raise => false, :expire => 100) do
36
+ lock2_ok = true
37
+ end
38
+ end
39
+ lock1_ok.should be_true
40
+ lock2_ok.should be_true
41
+ Lockistics.release("ltest1")
42
+ Lockistics.release("ltest2")
43
+ end
44
+
45
+ it "should raise when lock not acquired" do
46
+ lock1_ok = false
47
+ lock2_ok = false
48
+ lock1 = Lockistics.lock("ltest1", :wait => false, :raise => false, :expire => 100)
49
+ expect {Lockistics.lock("ltest1")}.to raise_error(Lockistics::LockTimeout)
50
+ Lockistics.release("ltest1")
51
+ Lockistics.release("ltest2")
52
+ end
53
+ end
@@ -0,0 +1,110 @@
1
+ require File.expand_path('../spec_helper.rb', __FILE__)
2
+
3
+ describe "Lockistics.meter" do
4
+ it "should collect daily metrics" do
5
+ Lockistics.meter("mtest1") do
6
+ sleep 1
7
+ end
8
+ last_daily_ts = Lockistics.redis.smembers("lockistics.mtest1.dailies").last
9
+ last_daily_ts.should_not be_nil
10
+ last_daily = Lockistics.redis.hgetall("lockistics.mtest1.daily.#{last_daily_ts}")
11
+ last_daily.should be_a Hash
12
+ last_daily["max.time"].to_i.should be > 1000
13
+ last_max_time = last_daily["max.time"].to_i
14
+ last_daily["invocations"].to_i.should eq 1
15
+ Lockistics.meter("mtest1") do
16
+ sleep 3
17
+ end
18
+ last_daily_ts = Lockistics.redis.smembers("lockistics.mtest1.dailies").last
19
+ last_daily_ts.should_not be_nil
20
+ last_daily = Lockistics.redis.hgetall("lockistics.mtest1.daily.#{last_daily_ts}")
21
+ last_daily.should be_a Hash
22
+ last_daily["max.time"].to_i.should be > 2000
23
+ last_daily["min.time"].to_i.should eq last_max_time
24
+ last_daily["invocations"].to_i.should eq 2
25
+ Lockistics.redis.del("lockistics.mtest1.daily.#{last_daily_ts}")
26
+ end
27
+
28
+ it "should collect hourly metrics" do
29
+ Lockistics.meter("mtest2") do
30
+ sleep 1
31
+ end
32
+ last_hourly_ts = Lockistics.redis.smembers("lockistics.mtest2.hourlies").last
33
+ last_hourly_ts.should_not be_nil
34
+ last_hourly = Lockistics.redis.hgetall("lockistics.mtest2.hourly.#{last_hourly_ts}")
35
+ last_hourly.should be_a Hash
36
+ last_hourly["max.time"].to_i.should be > 1000
37
+ last_max_time = last_hourly["max.time"].to_i
38
+ last_hourly["invocations"].to_i.should eq 1
39
+ Lockistics.meter("mtest2") do
40
+ sleep 3
41
+ end
42
+ last_hourly_ts = Lockistics.redis.smembers("lockistics.mtest2.hourlies").last
43
+ last_hourly_ts.should_not be_nil
44
+ last_hourly = Lockistics.redis.hgetall("lockistics.mtest2.hourly.#{last_hourly_ts}")
45
+ last_hourly.should be_a Hash
46
+ last_hourly["max.time"].to_i.should be > 2000
47
+ last_hourly["min.time"].to_i.should eq last_max_time
48
+ last_hourly["invocations"].to_i.should eq 2
49
+ Lockistics.redis.del("lockistics.mtest2.hourly.#{last_hourly_ts}")
50
+ end
51
+
52
+ it "should collect total metrics" do
53
+ Lockistics.meter("mtest3") do
54
+ sleep 1
55
+ end
56
+ last_total = Lockistics.redis.hgetall("lockistics.mtest3.total")
57
+ last_total.should be_a Hash
58
+ last_invocations = last_total["invocations"].to_i
59
+ Lockistics.meter("mtest3") do
60
+ sleep 3
61
+ end
62
+ last_total = Lockistics.redis.hgetall("lockistics.mtest3.total")
63
+ last_total["invocations"].to_i.should eq last_invocations + 1
64
+ Lockistics.redis.del("lockistics.mtest3.total")
65
+ end
66
+
67
+ it "should collect custom metrics" do
68
+ Lockistics.meter("mtest4") do |meter|
69
+ meter.incr "stuffs_done"
70
+ meter.incrby "stuffs_done2", 5
71
+ meter.set_minmax "stuffs_done3", 10
72
+ end
73
+ last_hourly_ts = Lockistics.redis.smembers("lockistics.mtest4.hourlies").last
74
+ last_hourly = Lockistics.redis.hgetall("lockistics.mtest4.hourly.#{last_hourly_ts}")
75
+ last_hourly["min.stuffs_done3"].to_i.should eq 10
76
+ last_hourly["max.stuffs_done3"].to_i.should eq 10
77
+ last_hourly["stuffs_done2"].to_i.should eq 5
78
+ last_hourly["stuffs_done"].to_i.should eq 1
79
+ Lockistics.meter("mtest4") do |meter|
80
+ meter.incr "stuffs_done"
81
+ meter.incrby "stuffs_done2", 5
82
+ meter.set_minmax "stuffs_done3", 20
83
+ end
84
+ last_hourly_ts = Lockistics.redis.smembers("lockistics.mtest4.hourlies").last
85
+ last_hourly = Lockistics.redis.hgetall("lockistics.mtest4.hourly.#{last_hourly_ts}")
86
+ last_hourly["min.stuffs_done3"].to_i.should eq 10
87
+ last_hourly["max.stuffs_done3"].to_i.should eq 20
88
+ last_hourly["stuffs_done2"].to_i.should eq 10
89
+ last_hourly["stuffs_done"].to_i.should eq 2
90
+ end
91
+
92
+ it "should collect metrics while locking too" do
93
+ Lockistics.meterlock("mtest4") do
94
+ sleep 1
95
+ end
96
+ last_hourly_ts = Lockistics.redis.smembers("lockistics.mtest4.dailies").last
97
+ last_hourly = Lockistics.redis.hgetall("lockistics.mtest4.dailies.#{last_hourly_ts}")
98
+ end
99
+
100
+ it "should know known keys" do
101
+ Lockistics.redis.del("lockistics.known_keys")
102
+ Lockistics.lock("mtest1", :wait => false, :raise => false) {}
103
+ Lockistics.meter("mtest2") {}
104
+ Lockistics.meterlock("mtest3", :wait => false, :raise => false) {}
105
+ Lockistics.known_keys.include?("mtest1").should be_true
106
+ Lockistics.known_keys.include?("mtest2").should be_true
107
+ Lockistics.known_keys.include?("mtest3").should be_true
108
+ Lockistics.known_keys.include?("mtest4").should be_false
109
+ end
110
+ end
@@ -0,0 +1,56 @@
1
+ require 'rspec'
2
+ require 'redis'
3
+ require 'fakeredis'
4
+ require File.expand_path('../../lib/lockistics.rb', __FILE__)
5
+
6
+ RSpec.configure do |config|
7
+ config.treat_symbols_as_metadata_keys_with_true_values = true
8
+ config.run_all_when_everything_filtered = true
9
+ config.filter_run :focus
10
+ config.order = 'random'
11
+ end
12
+
13
+ class Redis
14
+ def eval(script, keys=[], args=[])
15
+ case script
16
+ when Lockistics::Lock::LUA_ACQUIRE
17
+ fake_acquire(keys, args)
18
+ when Lockistics::Meter::LUA_SETMAX
19
+ fake_hsetmax(keys, args)
20
+ when Lockistics::Meter::LUA_SETMIN
21
+ fake_hsetmin(keys, args)
22
+ else
23
+ super
24
+ end
25
+ end
26
+
27
+ def fake_acquire(keys, args=[])
28
+ return false if existing = get(keys.first)
29
+ if setnx(keys.first, 1)
30
+ expire keys.first, keys.last
31
+ 1
32
+ else
33
+ 0
34
+ end
35
+ end
36
+
37
+ def fake_hsetmax(keys=[], args=[])
38
+ existing = hget(keys[0], keys[1])
39
+ if existing.nil? || args.first > existing.to_i
40
+ hset(keys[0], keys[1], args.first)
41
+ 1
42
+ else
43
+ 0
44
+ end
45
+ end
46
+
47
+ def fake_hsetmin(keys=[], args=[])
48
+ existing = hget(keys[0], keys[1])
49
+ if existing.nil? || args.first < existing.to_i
50
+ hset(keys[0], keys[1], args.first)
51
+ 1
52
+ else
53
+ 0
54
+ end
55
+ end
56
+ end
@@ -0,0 +1,37 @@
1
+ require File.expand_path('../spec_helper.rb', __FILE__)
2
+
3
+ describe "Lockistics.statistics" do
4
+ it "should get total metrics" do
5
+ Lockistics.meter("stest1") do
6
+ sleep 1
7
+ end
8
+ stats = Lockistics.statistics("stest1")
9
+ stats.total["max.time"].to_i.should be > 1000
10
+ stats.total["invocations"].to_i.should eq 1
11
+ Lockistics.meter("stest1") do
12
+ sleep 3
13
+ end
14
+ stats = Lockistics.statistics("stest1")
15
+ stats.total["max.time"].to_i.should be > 3000
16
+ stats.total["invocations"].to_i.should eq 2
17
+ end
18
+
19
+ it "should get daily metrics" do
20
+ Lockistics.meter("stest2") do
21
+ sleep 1
22
+ end
23
+ stats = Lockistics.statistics("stest2")
24
+ stats.daily.should be_a Array
25
+ stats.daily.first[:time].should be_a Time
26
+ end
27
+
28
+ it "should get hourly metrics" do
29
+ Lockistics.meter("stest3") do
30
+ sleep 1
31
+ end
32
+ stats = Lockistics.statistics("stest3")
33
+ stats.hourly.should be_a Array
34
+ stats.hourly.first[:time].should be_a Time
35
+ end
36
+ end
37
+
metadata ADDED
@@ -0,0 +1,169 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: lockistics
3
+ version: !ruby/object:Gem::Version
4
+ hash: 27
5
+ prerelease:
6
+ segments:
7
+ - 0
8
+ - 1
9
+ - 0
10
+ version: 0.1.0
11
+ platform: ruby
12
+ authors:
13
+ - Kimmo Lehto
14
+ autorequire:
15
+ bindir: bin
16
+ cert_chain: []
17
+
18
+ date: 2014-05-19 00:00:00 Z
19
+ dependencies:
20
+ - !ruby/object:Gem::Dependency
21
+ version_requirements: &id001 !ruby/object:Gem::Requirement
22
+ none: false
23
+ requirements:
24
+ - - ">="
25
+ - !ruby/object:Gem::Version
26
+ hash: 3
27
+ segments:
28
+ - 0
29
+ version: "0"
30
+ name: redis
31
+ prerelease: false
32
+ type: :runtime
33
+ requirement: *id001
34
+ - !ruby/object:Gem::Dependency
35
+ version_requirements: &id002 !ruby/object:Gem::Requirement
36
+ none: false
37
+ requirements:
38
+ - - ">="
39
+ - !ruby/object:Gem::Version
40
+ hash: 3
41
+ segments:
42
+ - 0
43
+ version: "0"
44
+ name: os
45
+ prerelease: false
46
+ type: :runtime
47
+ requirement: *id002
48
+ - !ruby/object:Gem::Dependency
49
+ version_requirements: &id003 !ruby/object:Gem::Requirement
50
+ none: false
51
+ requirements:
52
+ - - ~>
53
+ - !ruby/object:Gem::Version
54
+ hash: 9
55
+ segments:
56
+ - 1
57
+ - 3
58
+ version: "1.3"
59
+ name: bundler
60
+ prerelease: false
61
+ type: :development
62
+ requirement: *id003
63
+ - !ruby/object:Gem::Dependency
64
+ version_requirements: &id004 !ruby/object:Gem::Requirement
65
+ none: false
66
+ requirements:
67
+ - - ">="
68
+ - !ruby/object:Gem::Version
69
+ hash: 3
70
+ segments:
71
+ - 0
72
+ version: "0"
73
+ name: rake
74
+ prerelease: false
75
+ type: :development
76
+ requirement: *id004
77
+ - !ruby/object:Gem::Dependency
78
+ version_requirements: &id005 !ruby/object:Gem::Requirement
79
+ none: false
80
+ requirements:
81
+ - - ">="
82
+ - !ruby/object:Gem::Version
83
+ hash: 3
84
+ segments:
85
+ - 0
86
+ version: "0"
87
+ name: rspec
88
+ prerelease: false
89
+ type: :development
90
+ requirement: *id005
91
+ - !ruby/object:Gem::Dependency
92
+ version_requirements: &id006 !ruby/object:Gem::Requirement
93
+ none: false
94
+ requirements:
95
+ - - ">="
96
+ - !ruby/object:Gem::Version
97
+ hash: 3
98
+ segments:
99
+ - 0
100
+ version: "0"
101
+ name: fakeredis
102
+ prerelease: false
103
+ type: :development
104
+ requirement: *id006
105
+ description: Statsistics collecting locking
106
+ email:
107
+ - kimmo.lehto@gmail.com
108
+ executables: []
109
+
110
+ extensions: []
111
+
112
+ extra_rdoc_files: []
113
+
114
+ files:
115
+ - .gitignore
116
+ - Gemfile
117
+ - LICENSE
118
+ - LICENSE.txt
119
+ - README.md
120
+ - Rakefile
121
+ - lib/lockistics.rb
122
+ - lib/lockistics/configuration.rb
123
+ - lib/lockistics/lock.rb
124
+ - lib/lockistics/meter.rb
125
+ - lib/lockistics/statistics.rb
126
+ - lib/lockistics/version.rb
127
+ - lockistics.gemspec
128
+ - spec/locking_spec.rb
129
+ - spec/meter_spec.rb
130
+ - spec/spec_helper.rb
131
+ - spec/statistics_spec.rb
132
+ homepage: https://github.com/kke/lockistics
133
+ licenses:
134
+ - MIT
135
+ post_install_message:
136
+ rdoc_options: []
137
+
138
+ require_paths:
139
+ - lib
140
+ required_ruby_version: !ruby/object:Gem::Requirement
141
+ none: false
142
+ requirements:
143
+ - - ">="
144
+ - !ruby/object:Gem::Version
145
+ hash: 3
146
+ segments:
147
+ - 0
148
+ version: "0"
149
+ required_rubygems_version: !ruby/object:Gem::Requirement
150
+ none: false
151
+ requirements:
152
+ - - ">="
153
+ - !ruby/object:Gem::Version
154
+ hash: 3
155
+ segments:
156
+ - 0
157
+ version: "0"
158
+ requirements: []
159
+
160
+ rubyforge_project:
161
+ rubygems_version: 1.8.25
162
+ signing_key:
163
+ specification_version: 3
164
+ summary: With lockistics you can use Redis to create distributed locks and collect statistics how often and how long your locks are held
165
+ test_files:
166
+ - spec/locking_spec.rb
167
+ - spec/meter_spec.rb
168
+ - spec/spec_helper.rb
169
+ - spec/statistics_spec.rb