lockistics 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/.gitignore ADDED
@@ -0,0 +1,17 @@
1
+ *.gem
2
+ *.rbc
3
+ .bundle
4
+ .config
5
+ .yardoc
6
+ Gemfile.lock
7
+ InstalledFiles
8
+ _yardoc
9
+ coverage
10
+ doc/
11
+ lib/bundler/man
12
+ pkg
13
+ rdoc
14
+ spec/reports
15
+ test/tmp
16
+ test/version_tmp
17
+ tmp
data/Gemfile ADDED
@@ -0,0 +1,4 @@
1
+ source 'https://rubygems.org'
2
+
3
+ # Specify your gem's dependencies in lockistics.gemspec
4
+ gemspec
data/LICENSE ADDED
@@ -0,0 +1,24 @@
1
+ This is free and unencumbered software released into the public domain.
2
+
3
+ Anyone is free to copy, modify, publish, use, compile, sell, or
4
+ distribute this software, either in source code form or as a compiled
5
+ binary, for any purpose, commercial or non-commercial, and by any
6
+ means.
7
+
8
+ In jurisdictions that recognize copyright laws, the author or authors
9
+ of this software dedicate any and all copyright interest in the
10
+ software to the public domain. We make this dedication for the benefit
11
+ of the public at large and to the detriment of our heirs and
12
+ successors. We intend this dedication to be an overt act of
13
+ relinquishment in perpetuity of all present and future rights to this
14
+ software under copyright law.
15
+
16
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
17
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
18
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
19
+ IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
20
+ OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
21
+ ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
22
+ OTHER DEALINGS IN THE SOFTWARE.
23
+
24
+ For more information, please refer to <http://unlicense.org>
data/LICENSE.txt ADDED
@@ -0,0 +1,22 @@
1
+ Copyright (c) 2014 Kimmo Lehto
2
+
3
+ MIT License
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining
6
+ a copy of this software and associated documentation files (the
7
+ "Software"), to deal in the Software without restriction, including
8
+ without limitation the rights to use, copy, modify, merge, publish,
9
+ distribute, sublicense, and/or sell copies of the Software, and to
10
+ permit persons to whom the Software is furnished to do so, subject to
11
+ the following conditions:
12
+
13
+ The above copyright notice and this permission notice shall be
14
+ included in all copies or substantial portions of the Software.
15
+
16
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
17
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
18
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
19
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
20
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
21
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
22
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,170 @@
1
+ # Lockistics
2
+
3
+ Lockistics is basically a distributed mutex on Redis with statistics collecting included.
4
+
5
+ The likely use case for locking would be something like a Raketask that you don't want running multiple instances at once.
6
+
7
+ The likely use case for the statistics part would be that you want to how often something is being called or if a certain Raketask has been run today or not. You can also use it to find memory leaks or slow methods, kind of private NewRelic with zero features.
8
+
9
+ ## Installation
10
+
11
+ Add this line to your application's Gemfile:
12
+
13
+ gem 'lockistics'
14
+
15
+ And then execute:
16
+
17
+ $ bundle
18
+
19
+ Or install it yourself as:
20
+
21
+ $ gem install lockistics
22
+
23
+ #### Redis
24
+
25
+ Notice that you need a Redis v2.6.2+ as this gem uses LUA for race condition safe lock acquiring and min/max setting
26
+
27
+ ## Usage
28
+
29
+ You can use both parts separately if you just want to collect statistics or to just do simple locking.
30
+
31
+ Total, daily and hourly metrics you get for each key are:
32
+
33
+ - Number of locks
34
+ - Number of times having to wait for lock
35
+ - Number of failed locking attempts
36
+ - Minimum and maximum duration
37
+ - Minimum and maximum memory growth (using OS gem)
38
+ - Arbitary metrics you add during execution (more on this in examples)
39
+
40
+ ## Why?
41
+
42
+ Convenience mostly. There are redis-locking gems and some quite complex statistics modules, this does both with minimum dependencies, easy usage and Ruby 1.8.7 support.
43
+
44
+ ## Examples
45
+
46
+ #### Configure the gem
47
+
48
+ These are the default settings :
49
+
50
+ ```ruby
51
+ Lockistics.configure do |config|
52
+ config.redis = Redis.new
53
+ config.namespace = "lockistics"
54
+ config.expire = 300 # seconds
55
+ config.sleep = 0.5 # seconds to sleep between retries
56
+ config.retries = 10 # retry times
57
+ config.raise = true # raise Lockistics::TimeoutException when lock fails
58
+ end
59
+ ```
60
+
61
+ #### Getting and using a lock
62
+
63
+ ```ruby
64
+ # Get a lock, do what you must, release lock. No statistics collection.
65
+ Lockistics.lock("generate-stuff-raketask") do
66
+ doing_some_heavy_stuff
67
+ end
68
+ ```
69
+
70
+ ```ruby
71
+ # Some raketask that you don't want to run multiple times at once :
72
+ namespace :raketask
73
+ desc 'Generate stuff'
74
+ task :generate_stuff do
75
+ return nil unless Lockistics.lock("generate-stuff", :wait => false)
76
+ ...
77
+ end
78
+ end
79
+ ```
80
+
81
+ ```ruby
82
+ # Handle exception when you fail to acquire a lock in time:
83
+ begin
84
+ Lockistics.lock("stuff") do
85
+ ...
86
+ end
87
+ rescue Lockistics::Timeout
88
+ ...
89
+ end
90
+ ```
91
+
92
+ ```ruby
93
+ # Don't raise exceptions
94
+ Lockistics.lock("stuff", :raise => false) do
95
+ ...
96
+ end
97
+ ```
98
+
99
+ #### Statistics collection without locking
100
+
101
+ It works exactly like the locking, but the method is `meter`.
102
+
103
+ ```ruby
104
+ # Perform something, statistics will be collected behind the scenes.
105
+ Lockistics.meter("generate-stuff-raketask") do
106
+ doing_some_stuff
107
+ end
108
+ ```
109
+
110
+ ```ruby
111
+ # Adding custom metrics
112
+ Lockistics.meter("generate-stuff") do |meter|
113
+ results = do_stuff
114
+ if results.empty?
115
+ meter.incr "empty_results"
116
+ else
117
+ meter.incrby "stuffs_done", results.size
118
+ end
119
+ end
120
+ ```
121
+
122
+ #### Statistics collection with locking
123
+
124
+ It works exactly like the above, but the method is `meterlock`.
125
+
126
+ ```ruby
127
+ # Adding custom metrics
128
+ Lockistics.meterlock("generate-stuff", :raise => false) do |meter|
129
+ results = do_stuff
130
+ if results.empty?
131
+ meter.incr "empty_results"
132
+ else
133
+ # Sets min and/or max for a key (min.stuffs_done + max.stuffs_done)
134
+ # Only sets if value is minimum or maximum for the periods.
135
+ meter.set_minmax "stuffs_done", results.size
136
+ end
137
+ end
138
+ ```
139
+
140
+ #### Getting the statistics out
141
+
142
+ You can query statistics for locking/metering keys.
143
+
144
+ ```ruby
145
+ stats = Lockistics.statistics('generate'stuff')
146
+ # Get the last run
147
+ stats.last_run
148
+ => Mon May 19 16:38:52 +0300 2014
149
+ # Get totals:
150
+ stats.total
151
+ => {"invocations" => 50, "lock-timeouts" => 1,
152
+ "max.stuffs-generated" => 10, "max_rss" => 400 ..}
153
+ stats.daily
154
+ => [{:time => #<Time..> "invocations" => 50, "lock-timeouts" => 1,
155
+ "max.stuffs-generated" => 10, "max_rss" => 400 ..}, {..}]
156
+ stats.hourly
157
+ => [{:time => #<Time..> "invocations" => 50, "lock-timeouts" => 1,
158
+ "max.stuffs-generated" => 10, "max_rss" => 400 ..}, {..}]
159
+ stats.all
160
+ => { :daily => [...], :hourly => [...], :total => {...},
161
+ :last_run:time => #<Time..>}
162
+ ```
163
+
164
+ ## Contributing
165
+
166
+ 1. Fork it
167
+ 2. Create your feature branch (`git checkout -b my-new-feature`)
168
+ 3. Commit your changes (`git commit -am 'Add some feature'`)
169
+ 4. Push to the branch (`git push origin my-new-feature`)
170
+ 5. Create new Pull Request
data/Rakefile ADDED
@@ -0,0 +1,3 @@
1
+ require "bundler/gem_tasks"
2
+ require 'rspec/core/rake_task'
3
+ RSpec::Core::RakeTask.new(:spec)
@@ -0,0 +1,34 @@
1
+ require 'redis'
2
+
3
+ module Lockistics
4
+ class Configuration
5
+
6
+ attr_accessor :redis
7
+ attr_accessor :namespace
8
+ attr_accessor :logger
9
+ attr_accessor :expire
10
+ attr_accessor :sleep
11
+ attr_accessor :retries
12
+ attr_accessor :raise
13
+
14
+ def initialize
15
+ @redis = Redis.new
16
+ @namespace = 'lockistics'
17
+ @expire = 10
18
+ @sleep = 0.5
19
+ @retries = 10
20
+ end
21
+
22
+ def lock_defaults
23
+ {
24
+ :redis => redis,
25
+ :namespace => namespace,
26
+ :expire => expire,
27
+ :sleep => sleep,
28
+ :retries => retries,
29
+ :wait => true,
30
+ :raise => true
31
+ }
32
+ end
33
+ end
34
+ end
@@ -0,0 +1,67 @@
1
+ module Lockistics
2
+
3
+ class LockTimeout < StandardError; end
4
+
5
+ class Lock
6
+
7
+ attr_accessor :key, :options, :lock_retries
8
+
9
+ LUA_ACQUIRE = "return redis.call('setnx', KEYS[1], 1) == 1 and redis.call('expire', KEYS[1], KEYS[2]) and 1 or 0"
10
+
11
+ def initialize(key, options={})
12
+ @key = key
13
+ @options = Lockistics.configuration.lock_defaults.merge(options)
14
+ @options[:expire] = 999_999_999 unless @options[:expire].to_i > 0 # :expire => false
15
+ @exceeded_before_release = false
16
+ @lock_retries = 0
17
+ end
18
+
19
+ def acquire_lock
20
+ Lockistics.known_keys(key)
21
+ if got_lock?
22
+ true
23
+ elsif options[:wait]
24
+ wait_for_lock
25
+ else
26
+ false
27
+ end
28
+ end
29
+
30
+ def wait_for_lock
31
+ until got_lock?
32
+ @lock_retries += 1
33
+ if lock_retries <= options[:retries]
34
+ sleep options[:sleep]
35
+ elsif options[:raise]
36
+ raise LockTimeout, "while waiting for #{key}"
37
+ else
38
+ return false
39
+ end
40
+ end
41
+ true
42
+ end
43
+
44
+ def exceeded_before_release?
45
+ @exceeded_before_release
46
+ end
47
+
48
+ def release_lock
49
+ @exceeded_before_release = redis.del(namespaced_key) == 0
50
+ end
51
+
52
+ def namespaced_key
53
+ @namespaced_key ||= "#{Lockistics.configuration.namespace}.lock.#{key}"
54
+ end
55
+
56
+ private
57
+
58
+ def redis
59
+ Lockistics.redis
60
+ end
61
+
62
+ def got_lock?
63
+ redis.eval(LUA_ACQUIRE, [namespaced_key, options[:expire]]) == 1
64
+ end
65
+
66
+ end
67
+ end
@@ -0,0 +1,165 @@
1
+ require 'os'
2
+
3
+ module Lockistics
4
+ class Meter
5
+
6
+ attr_accessor :key, :options
7
+
8
+ LUA_SETMAX = <<-EOB.gsub("\n", " ").gsub(/\s+/, " ")
9
+ if redis.call("hexists", KEYS[1], KEYS[2]) == 0 or
10
+ tonumber(redis.call("hget", KEYS[1], KEYS[2])) < tonumber(ARGV[1])
11
+ then
12
+ return redis.call("hset", KEYS[1], KEYS[2], ARGV[1])
13
+ else
14
+ return 0
15
+ end
16
+ EOB
17
+
18
+ LUA_SETMIN = <<-EOB.gsub("\n", " ").gsub(/\s+/, " ")
19
+ if redis.call("hexists", KEYS[1], KEYS[2]) == 0 or
20
+ tonumber(redis.call("hget", KEYS[1], KEYS[2])) > tonumber(ARGV[1])
21
+ then
22
+ return redis.call("hset", KEYS[1], KEYS[2], ARGV[1])
23
+ else
24
+ return 0
25
+ end
26
+ EOB
27
+
28
+ def initialize(key, options={})
29
+ @key = key
30
+ @options = options
31
+ @lock_timeouts = 0
32
+ end
33
+
34
+ def with_lock(&block)
35
+ raise ArgumentError, "with_lock called without block" unless block_given?
36
+ raise ArgumentError, "lock not defined" if lock.nil?
37
+ lock.acquire_lock
38
+ yield self
39
+ ensure
40
+ lock.release_lock
41
+ end
42
+
43
+ def perform(&block)
44
+ raise ArgumentError, "perform called without block" unless block_given?
45
+ before_perform
46
+ lock.nil? ? yield(self) : with_lock(&block)
47
+ rescue Lockistics::LockTimeout
48
+ @lock_timeouts = 1
49
+ raise
50
+ ensure
51
+ after_perform
52
+ end
53
+
54
+ # You can add custom metrics during runtime
55
+ #
56
+ # @example
57
+ # Lockistics.meter do |meter|
58
+ # foo = FooGenerator.new
59
+ # foo.perform
60
+ # meter.incrby 'foos-generated', foo.count
61
+ # end
62
+ def incrby(key, value)
63
+ return nil if value == 0
64
+ [:hourly, :daily, :total].each do |period|
65
+ redis.hincrby namespaced_hash(period), key, value
66
+ end
67
+ end
68
+
69
+ # You can add custom metrics during runtime with
70
+ # this.
71
+ #
72
+ # This is a shortcut to incrby(key, 1)
73
+ #
74
+ # @example
75
+ # Lockistics.meter do |meter|
76
+ # foo = FooGenerator.new
77
+ # foo.perform
78
+ # meter.incr 'failed-foo-generations' unless foo.success?
79
+ # end
80
+ def incr(key)
81
+ incrby(key, 1)
82
+ end
83
+
84
+ def set_minmax(key, value)
85
+ [:hourly, :daily, :total].each do |period|
86
+ redis_hsetmax(namespaced_hash(period), "max.#{key}", value)
87
+ redis_hsetmin(namespaced_hash(period), "min.#{key}", value)
88
+ end
89
+ end
90
+
91
+ private
92
+
93
+ def redis
94
+ Lockistics.configuration.redis
95
+ end
96
+
97
+ def redis_hsetmax(hash, key, value)
98
+ redis.eval(LUA_SETMAX, [hash, key], [value]) == 1
99
+ end
100
+
101
+ def redis_hsetmin(hash, key, value)
102
+ redis.eval(LUA_SETMIN, [hash, key], [value]) == 1
103
+ end
104
+
105
+ def lock
106
+ options[:lock]
107
+ end
108
+
109
+ def before_perform
110
+ Lockistics.known_keys(key)
111
+ @start_time = Time.now.to_f
112
+ @start_rss = OS.rss_bytes
113
+ redis.pipelined do
114
+ redis.sadd "#{Lockistics.configuration.namespace}.#{key}.hourlies", hourly_timestamp
115
+ redis.sadd "#{Lockistics.configuration.namespace}.#{key}.dailies", daily_timestamp
116
+ redis.set "#{Lockistics.configuration.namespace}.#{key}.last_run", Time.now.to_i
117
+ end
118
+ end
119
+
120
+ def after_perform
121
+ unless @lock_timeouts > 0
122
+ @duration = ((Time.now.to_f - @start_time) * 1000).round
123
+ @rss_increase = ((OS.rss_bytes - @start_rss) / 1024).round
124
+ end
125
+ add_meter_statistics unless options[:no_metrics]
126
+ add_lock_statistics unless lock.nil?
127
+ end
128
+
129
+ def add_meter_statistics
130
+ incrby 'invocations', 1
131
+ set_minmax 'rss', @rss_increase unless @rss_increase.nil?
132
+ set_minmax 'time', @duration unless @duration.nil?
133
+ end
134
+
135
+ def add_lock_statistics
136
+ redis.pipelined do
137
+ incrby 'lock-invocations', 1
138
+ incrby 'lock-retries', lock.lock_retries
139
+ incrby 'lock-timeouts', @lock_timeouts
140
+ if lock.exceeded_before_release?
141
+ incrby 'lock-exceeded-before-release', 1
142
+ end
143
+ end
144
+ end
145
+
146
+ def hourly_timestamp
147
+ @hourly_timestamp ||= Time.now.strftime("%Y%m%d%H")
148
+ end
149
+
150
+ def daily_timestamp
151
+ @daily_timestamp ||= Time.now.strftime("%Y%m%d")
152
+ end
153
+
154
+ def namespaced_hash(period)
155
+ case period
156
+ when :hourly
157
+ "#{Lockistics.configuration.namespace}.#{key}.hourly.#{hourly_timestamp}"
158
+ when :daily
159
+ "#{Lockistics.configuration.namespace}.#{key}.daily.#{daily_timestamp}"
160
+ when :total
161
+ "#{Lockistics.configuration.namespace}.#{key}.total"
162
+ end
163
+ end
164
+ end
165
+ end
@@ -0,0 +1,89 @@
1
+ require 'time'
2
+ module Lockistics
3
+ class Statistics
4
+
5
+ attr_accessor :key
6
+
7
+ def initialize(key)
8
+ @key = key
9
+ end
10
+
11
+ def last_run
12
+ unix_time = redis.get(namespaced("#{key}.last_run"))
13
+ if unix_time
14
+ Time.at(unix_time.to_i)
15
+ end
16
+ end
17
+
18
+ def all(since=nil)
19
+ {
20
+ :daily => daily(since),
21
+ :hourly => hourly(since),
22
+ :total => total,
23
+ :last_run => last_run
24
+ }
25
+ end
26
+
27
+ def daily(since=nil)
28
+ daily_hashes(since).collect{|stamped| redis.hgetall(stamped.first).merge(:time => stamped.last )}
29
+ end
30
+
31
+ def hourly(since=nil)
32
+ hourly_hashes(since).collect{|stamped| redis.hgetall(stamped.first).merge(:time => stamped.last)}
33
+ end
34
+
35
+ def total
36
+ redis.hgetall total_hash
37
+ end
38
+
39
+ private
40
+
41
+ def date_to_hourly_stamp(date=nil)
42
+ if date.nil?
43
+ 0
44
+ elsif date.respond_to?(strftime)
45
+ date.strftime("%Y%m%d%H")
46
+ elsif date.match(/^\d+$/)
47
+ Time.at(date).strftime("%Y%m%d%H")
48
+ end
49
+ end
50
+
51
+ def date_to_daily_stamp(date=nil)
52
+ if date.nil?
53
+ 0
54
+ elsif date.respond_to?(strftime)
55
+ date.strftime("%Y%m%d")
56
+ elsif date.match(/^\d+$/)
57
+ Time.at(date).strftime("%Y%m%d")
58
+ end
59
+ end
60
+
61
+ def redis
62
+ Lockistics.redis
63
+ end
64
+
65
+ def daily_hashes(since=nil)
66
+ stamped_ts = date_to_daily_stamp(since)
67
+ redis.smembers(namespaced("#{key}.dailies")).collect do |ts|
68
+ next if ts.to_i < stamped_ts
69
+ [namespaced("#{key}.daily.#{ts}"), Time.parse(ts)]
70
+ end
71
+ end
72
+
73
+ def hourly_hashes(since=nil)
74
+ stamped_ts = date_to_hourly_stamp(since)
75
+ redis.smembers(namespaced("#{key}.hourlies")).collect do |ts|
76
+ next if ts.to_i < stamped_ts
77
+ [namespaced("#{key}.hourly.#{ts}"), Time.parse(ts)]
78
+ end
79
+ end
80
+
81
+ def total_hash
82
+ namespaced "#{key}.total"
83
+ end
84
+
85
+ def namespaced(extra_key=nil)
86
+ "#{Lockistics.configuration.namespace}#{extra_key.nil? ? "" : ".#{extra_key}"}"
87
+ end
88
+ end
89
+ end
@@ -0,0 +1,3 @@
1
+ module Lockistics
2
+ VERSION = "0.1.0"
3
+ end
data/lib/lockistics.rb ADDED
@@ -0,0 +1,137 @@
1
+ require "lockistics/version"
2
+ require "lockistics/configuration"
3
+ require "lockistics/lock"
4
+ require "lockistics/meter"
5
+ require "lockistics/statistics"
6
+
7
+ # Lockistics is basically a distributed mutex on Redis.
8
+ #
9
+ # In addition to locking it also collects statistics data
10
+ # of the locking events.
11
+ #
12
+ # You can use each part separately if you just want to
13
+ # collect statistics or to do simple locking.
14
+ #
15
+ # Total, daily and hourly metrics you get for each key are:
16
+ # - Number of locks
17
+ # - Number of times having to wait for lock
18
+ # - Number of failed locking attempts
19
+ # - Minimum and maximum duration
20
+ # - Minimum and maximum memory growth (using OS gem)
21
+ # - Arbitary metrics you add during execution
22
+ module Lockistics
23
+
24
+ # Configure the gem
25
+ #
26
+ # @example
27
+ # Lockistics.configure do |config|
28
+ # config.redis = Redis.new
29
+ # config.namespace = "production.locks"
30
+ # config.expire = 300 # seconds
31
+ # config.sleep = 0.5 # seconds to sleep between retries
32
+ # config.retries = 10 # retry times
33
+ # config.raise = true # raise Lockistics::TimeoutException when lock fails
34
+ # end
35
+ def self.configure(&block)
36
+ yield configuration
37
+ end
38
+
39
+ # Returns an instance of Lockistics::Configuration
40
+ def self.configuration
41
+ @configuration ||= Configuration.new
42
+ end
43
+
44
+ # Accessor to the redis configured via Lockistics::Configuration
45
+ def self.redis
46
+ configuration.redis
47
+ end
48
+
49
+ # Get a hold of a lock or wait for it to be released
50
+ #
51
+ # Given a block, will release the lock after block exection
52
+ #
53
+ # @example
54
+ # Lockistics.lock("generate-stuff-raketask") do
55
+ # doing_some_heavy_stuff
56
+ # end
57
+ # or
58
+ # return nil unless Lockistics.lock("generate-stuff", :wait => false)
59
+ # or
60
+ # begin
61
+ # Lockistics.lock("stuff") do
62
+ # ...
63
+ # end
64
+ # rescue Lockistics::Timeout
65
+ # ...
66
+ # end
67
+ def self.lock(key, options={}, &block)
68
+ if block_given?
69
+ Meter.new(key, options.merge(:no_metrics => true)).perform(&block)
70
+ else
71
+ Lock.new(key, options).acquire_lock
72
+ end
73
+ end
74
+
75
+ # Don't perform locking, just collect metrics.
76
+ #
77
+ # @example
78
+ # Lockistics.meter("generate-stuff") do |meter|
79
+ # do_stuff
80
+ # meter.incrby :stuffs_generated, 50
81
+ # end
82
+ def self.meter(key, options={}, &block)
83
+ Meter.new(key, options.merge(:lock => nil)).perform(&block)
84
+ end
85
+
86
+ # Perform locking and statistics collection
87
+ #
88
+ # @example
89
+ # Lockistics.meterlock("generate-stuff") do |meter|
90
+ # results = do_stuff
91
+ # if results.empty?
92
+ # meter.incr "empty_results"
93
+ # else
94
+ # meter.incrby "stuffs_done", results.size
95
+ # end
96
+ # end
97
+ def self.meterlock(key, options={}, &block)
98
+ Meter.new(key, options.merge(:lock => Lock.new(key, options))).perform(&block)
99
+ end
100
+
101
+ # Manually release a lock
102
+ #
103
+ # @example
104
+ # Lockistics.release("generate-stuff")
105
+ # or
106
+ # lock = Lockistics.lock("generate-stuff", :wait => false)
107
+ # if lock.acquire_lock
108
+ # do_some_stuff
109
+ # lock.release_lock
110
+ # end
111
+ def self.release(key, options={})
112
+ Lock.new(key).release_lock
113
+ end
114
+
115
+ # Returns an instance of Lockistics::Statistics for the key.
116
+ #
117
+ # @example
118
+ # stats = Lockistics.statistics('generate-stuff')
119
+ # stats.last_run
120
+ # => Mon May 19 16:38:52 +0300 2014
121
+ # stats.total
122
+ # => {"invocations" => 50, ...}
123
+ def self.statistics(key)
124
+ Statistics.new(key)
125
+ end
126
+
127
+ # Get a list of known keys or add a key to
128
+ # known keys list.
129
+ def self.known_keys(key=nil)
130
+ if key.nil?
131
+ redis.smembers "#{Lockistics.configuration.namespace}.known_keys"
132
+ else
133
+ redis.sadd "#{Lockistics.configuration.namespace}.known_keys", key
134
+ end
135
+ end
136
+
137
+ end
@@ -0,0 +1,28 @@
1
+ # coding: utf-8
2
+ lib = File.expand_path('../lib', __FILE__)
3
+ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
4
+ require 'lockistics/version'
5
+
6
+ Gem::Specification.new do |spec|
7
+ spec.name = "lockistics"
8
+ spec.version = Lockistics::VERSION
9
+ spec.authors = ["Kimmo Lehto"]
10
+ spec.email = ["kimmo.lehto@gmail.com"]
11
+ spec.description = %q{Statsistics collecting locking}
12
+ spec.summary = %q{With lockistics you can use Redis to create distributed locks and collect statistics how often and how long your locks are held}
13
+ spec.homepage = "https://github.com/kke/lockistics"
14
+ spec.license = "MIT"
15
+
16
+ spec.files = `git ls-files`.split($/)
17
+ spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
18
+ spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
19
+ spec.require_paths = ["lib"]
20
+
21
+ spec.add_runtime_dependency "redis"
22
+ spec.add_runtime_dependency "os"
23
+
24
+ spec.add_development_dependency "bundler", "~> 1.3"
25
+ spec.add_development_dependency "rake"
26
+ spec.add_development_dependency "rspec"
27
+ spec.add_development_dependency "fakeredis"
28
+ end
@@ -0,0 +1,53 @@
1
+ require File.expand_path('../spec_helper.rb', __FILE__)
2
+
3
+ describe "Lockistics.lock" do
4
+ it "should only allow one instance to lock something at once" do
5
+ lock1 = Lockistics.lock("ltest1", :wait => false, :raise => false, :expire => 100)
6
+ sleep 1
7
+ lock2 = Lockistics.lock("ltest1", :wait => false, :raise => false, :expire => 100)
8
+ lock1.should be_true
9
+ lock2.should be_false
10
+ Lockistics.release("ltest1")
11
+ lock2 = Lockistics.lock("ltest1", :wait => false, :raise => false, :expire => 100)
12
+ lock2.should be_true
13
+ end
14
+
15
+ it "should allow locking again after block mode" do
16
+ lock1_ok = false
17
+ lock2_ok = false
18
+ lock1 = Lockistics.lock("ltest1", :wait => false, :raise => false, :expire => 100) do
19
+ lock1_ok = true
20
+ sleep 1
21
+ end
22
+ lock2 = Lockistics.lock("ltest1", :wait => false, :raise => false, :expire => 100) do
23
+ lock2_ok = true
24
+ end
25
+ lock1_ok.should be_true
26
+ lock2_ok.should be_true
27
+ Lockistics.release("ltest1")
28
+ end
29
+
30
+ it "should allow locking different keys at once" do
31
+ lock1_ok = false
32
+ lock2_ok = false
33
+ lock1 = Lockistics.lock("ltest1", :wait => false, :raise => false, :expire => 100) do
34
+ lock1_ok = true
35
+ lock2 = Lockistics.lock("ltest2", :wait => false, :raise => false, :expire => 100) do
36
+ lock2_ok = true
37
+ end
38
+ end
39
+ lock1_ok.should be_true
40
+ lock2_ok.should be_true
41
+ Lockistics.release("ltest1")
42
+ Lockistics.release("ltest2")
43
+ end
44
+
45
+ it "should raise when lock not acquired" do
46
+ lock1_ok = false
47
+ lock2_ok = false
48
+ lock1 = Lockistics.lock("ltest1", :wait => false, :raise => false, :expire => 100)
49
+ expect {Lockistics.lock("ltest1")}.to raise_error(Lockistics::LockTimeout)
50
+ Lockistics.release("ltest1")
51
+ Lockistics.release("ltest2")
52
+ end
53
+ end
@@ -0,0 +1,110 @@
1
+ require File.expand_path('../spec_helper.rb', __FILE__)
2
+
3
+ describe "Lockistics.meter" do
4
+ it "should collect daily metrics" do
5
+ Lockistics.meter("mtest1") do
6
+ sleep 1
7
+ end
8
+ last_daily_ts = Lockistics.redis.smembers("lockistics.mtest1.dailies").last
9
+ last_daily_ts.should_not be_nil
10
+ last_daily = Lockistics.redis.hgetall("lockistics.mtest1.daily.#{last_daily_ts}")
11
+ last_daily.should be_a Hash
12
+ last_daily["max.time"].to_i.should be > 1000
13
+ last_max_time = last_daily["max.time"].to_i
14
+ last_daily["invocations"].to_i.should eq 1
15
+ Lockistics.meter("mtest1") do
16
+ sleep 3
17
+ end
18
+ last_daily_ts = Lockistics.redis.smembers("lockistics.mtest1.dailies").last
19
+ last_daily_ts.should_not be_nil
20
+ last_daily = Lockistics.redis.hgetall("lockistics.mtest1.daily.#{last_daily_ts}")
21
+ last_daily.should be_a Hash
22
+ last_daily["max.time"].to_i.should be > 2000
23
+ last_daily["min.time"].to_i.should eq last_max_time
24
+ last_daily["invocations"].to_i.should eq 2
25
+ Lockistics.redis.del("lockistics.mtest1.daily.#{last_daily_ts}")
26
+ end
27
+
28
+ it "should collect hourly metrics" do
29
+ Lockistics.meter("mtest2") do
30
+ sleep 1
31
+ end
32
+ last_hourly_ts = Lockistics.redis.smembers("lockistics.mtest2.hourlies").last
33
+ last_hourly_ts.should_not be_nil
34
+ last_hourly = Lockistics.redis.hgetall("lockistics.mtest2.hourly.#{last_hourly_ts}")
35
+ last_hourly.should be_a Hash
36
+ last_hourly["max.time"].to_i.should be > 1000
37
+ last_max_time = last_hourly["max.time"].to_i
38
+ last_hourly["invocations"].to_i.should eq 1
39
+ Lockistics.meter("mtest2") do
40
+ sleep 3
41
+ end
42
+ last_hourly_ts = Lockistics.redis.smembers("lockistics.mtest2.hourlies").last
43
+ last_hourly_ts.should_not be_nil
44
+ last_hourly = Lockistics.redis.hgetall("lockistics.mtest2.hourly.#{last_hourly_ts}")
45
+ last_hourly.should be_a Hash
46
+ last_hourly["max.time"].to_i.should be > 2000
47
+ last_hourly["min.time"].to_i.should eq last_max_time
48
+ last_hourly["invocations"].to_i.should eq 2
49
+ Lockistics.redis.del("lockistics.mtest2.hourly.#{last_hourly_ts}")
50
+ end
51
+
52
+ it "should collect total metrics" do
53
+ Lockistics.meter("mtest3") do
54
+ sleep 1
55
+ end
56
+ last_total = Lockistics.redis.hgetall("lockistics.mtest3.total")
57
+ last_total.should be_a Hash
58
+ last_invocations = last_total["invocations"].to_i
59
+ Lockistics.meter("mtest3") do
60
+ sleep 3
61
+ end
62
+ last_total = Lockistics.redis.hgetall("lockistics.mtest3.total")
63
+ last_total["invocations"].to_i.should eq last_invocations + 1
64
+ Lockistics.redis.del("lockistics.mtest3.total")
65
+ end
66
+
67
+ it "should collect custom metrics" do
68
+ Lockistics.meter("mtest4") do |meter|
69
+ meter.incr "stuffs_done"
70
+ meter.incrby "stuffs_done2", 5
71
+ meter.set_minmax "stuffs_done3", 10
72
+ end
73
+ last_hourly_ts = Lockistics.redis.smembers("lockistics.mtest4.hourlies").last
74
+ last_hourly = Lockistics.redis.hgetall("lockistics.mtest4.hourly.#{last_hourly_ts}")
75
+ last_hourly["min.stuffs_done3"].to_i.should eq 10
76
+ last_hourly["max.stuffs_done3"].to_i.should eq 10
77
+ last_hourly["stuffs_done2"].to_i.should eq 5
78
+ last_hourly["stuffs_done"].to_i.should eq 1
79
+ Lockistics.meter("mtest4") do |meter|
80
+ meter.incr "stuffs_done"
81
+ meter.incrby "stuffs_done2", 5
82
+ meter.set_minmax "stuffs_done3", 20
83
+ end
84
+ last_hourly_ts = Lockistics.redis.smembers("lockistics.mtest4.hourlies").last
85
+ last_hourly = Lockistics.redis.hgetall("lockistics.mtest4.hourly.#{last_hourly_ts}")
86
+ last_hourly["min.stuffs_done3"].to_i.should eq 10
87
+ last_hourly["max.stuffs_done3"].to_i.should eq 20
88
+ last_hourly["stuffs_done2"].to_i.should eq 10
89
+ last_hourly["stuffs_done"].to_i.should eq 2
90
+ end
91
+
92
+ it "should collect metrics while locking too" do
93
+ Lockistics.meterlock("mtest4") do
94
+ sleep 1
95
+ end
96
+ last_hourly_ts = Lockistics.redis.smembers("lockistics.mtest4.dailies").last
97
+ last_hourly = Lockistics.redis.hgetall("lockistics.mtest4.dailies.#{last_hourly_ts}")
98
+ end
99
+
100
+ it "should know known keys" do
101
+ Lockistics.redis.del("lockistics.known_keys")
102
+ Lockistics.lock("mtest1", :wait => false, :raise => false) {}
103
+ Lockistics.meter("mtest2") {}
104
+ Lockistics.meterlock("mtest3", :wait => false, :raise => false) {}
105
+ Lockistics.known_keys.include?("mtest1").should be_true
106
+ Lockistics.known_keys.include?("mtest2").should be_true
107
+ Lockistics.known_keys.include?("mtest3").should be_true
108
+ Lockistics.known_keys.include?("mtest4").should be_false
109
+ end
110
+ end
@@ -0,0 +1,56 @@
1
+ require 'rspec'
2
+ require 'redis'
3
+ require 'fakeredis'
4
+ require File.expand_path('../../lib/lockistics.rb', __FILE__)
5
+
6
+ RSpec.configure do |config|
7
+ config.treat_symbols_as_metadata_keys_with_true_values = true
8
+ config.run_all_when_everything_filtered = true
9
+ config.filter_run :focus
10
+ config.order = 'random'
11
+ end
12
+
13
+ class Redis
14
+ def eval(script, keys=[], args=[])
15
+ case script
16
+ when Lockistics::Lock::LUA_ACQUIRE
17
+ fake_acquire(keys, args)
18
+ when Lockistics::Meter::LUA_SETMAX
19
+ fake_hsetmax(keys, args)
20
+ when Lockistics::Meter::LUA_SETMIN
21
+ fake_hsetmin(keys, args)
22
+ else
23
+ super
24
+ end
25
+ end
26
+
27
+ def fake_acquire(keys, args=[])
28
+ return false if existing = get(keys.first)
29
+ if setnx(keys.first, 1)
30
+ expire keys.first, keys.last
31
+ 1
32
+ else
33
+ 0
34
+ end
35
+ end
36
+
37
+ def fake_hsetmax(keys=[], args=[])
38
+ existing = hget(keys[0], keys[1])
39
+ if existing.nil? || args.first > existing.to_i
40
+ hset(keys[0], keys[1], args.first)
41
+ 1
42
+ else
43
+ 0
44
+ end
45
+ end
46
+
47
+ def fake_hsetmin(keys=[], args=[])
48
+ existing = hget(keys[0], keys[1])
49
+ if existing.nil? || args.first < existing.to_i
50
+ hset(keys[0], keys[1], args.first)
51
+ 1
52
+ else
53
+ 0
54
+ end
55
+ end
56
+ end
@@ -0,0 +1,37 @@
1
+ require File.expand_path('../spec_helper.rb', __FILE__)
2
+
3
+ describe "Lockistics.statistics" do
4
+ it "should get total metrics" do
5
+ Lockistics.meter("stest1") do
6
+ sleep 1
7
+ end
8
+ stats = Lockistics.statistics("stest1")
9
+ stats.total["max.time"].to_i.should be > 1000
10
+ stats.total["invocations"].to_i.should eq 1
11
+ Lockistics.meter("stest1") do
12
+ sleep 3
13
+ end
14
+ stats = Lockistics.statistics("stest1")
15
+ stats.total["max.time"].to_i.should be > 3000
16
+ stats.total["invocations"].to_i.should eq 2
17
+ end
18
+
19
+ it "should get daily metrics" do
20
+ Lockistics.meter("stest2") do
21
+ sleep 1
22
+ end
23
+ stats = Lockistics.statistics("stest2")
24
+ stats.daily.should be_a Array
25
+ stats.daily.first[:time].should be_a Time
26
+ end
27
+
28
+ it "should get hourly metrics" do
29
+ Lockistics.meter("stest3") do
30
+ sleep 1
31
+ end
32
+ stats = Lockistics.statistics("stest3")
33
+ stats.hourly.should be_a Array
34
+ stats.hourly.first[:time].should be_a Time
35
+ end
36
+ end
37
+
metadata ADDED
@@ -0,0 +1,169 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: lockistics
3
+ version: !ruby/object:Gem::Version
4
+ hash: 27
5
+ prerelease:
6
+ segments:
7
+ - 0
8
+ - 1
9
+ - 0
10
+ version: 0.1.0
11
+ platform: ruby
12
+ authors:
13
+ - Kimmo Lehto
14
+ autorequire:
15
+ bindir: bin
16
+ cert_chain: []
17
+
18
+ date: 2014-05-19 00:00:00 Z
19
+ dependencies:
20
+ - !ruby/object:Gem::Dependency
21
+ version_requirements: &id001 !ruby/object:Gem::Requirement
22
+ none: false
23
+ requirements:
24
+ - - ">="
25
+ - !ruby/object:Gem::Version
26
+ hash: 3
27
+ segments:
28
+ - 0
29
+ version: "0"
30
+ name: redis
31
+ prerelease: false
32
+ type: :runtime
33
+ requirement: *id001
34
+ - !ruby/object:Gem::Dependency
35
+ version_requirements: &id002 !ruby/object:Gem::Requirement
36
+ none: false
37
+ requirements:
38
+ - - ">="
39
+ - !ruby/object:Gem::Version
40
+ hash: 3
41
+ segments:
42
+ - 0
43
+ version: "0"
44
+ name: os
45
+ prerelease: false
46
+ type: :runtime
47
+ requirement: *id002
48
+ - !ruby/object:Gem::Dependency
49
+ version_requirements: &id003 !ruby/object:Gem::Requirement
50
+ none: false
51
+ requirements:
52
+ - - ~>
53
+ - !ruby/object:Gem::Version
54
+ hash: 9
55
+ segments:
56
+ - 1
57
+ - 3
58
+ version: "1.3"
59
+ name: bundler
60
+ prerelease: false
61
+ type: :development
62
+ requirement: *id003
63
+ - !ruby/object:Gem::Dependency
64
+ version_requirements: &id004 !ruby/object:Gem::Requirement
65
+ none: false
66
+ requirements:
67
+ - - ">="
68
+ - !ruby/object:Gem::Version
69
+ hash: 3
70
+ segments:
71
+ - 0
72
+ version: "0"
73
+ name: rake
74
+ prerelease: false
75
+ type: :development
76
+ requirement: *id004
77
+ - !ruby/object:Gem::Dependency
78
+ version_requirements: &id005 !ruby/object:Gem::Requirement
79
+ none: false
80
+ requirements:
81
+ - - ">="
82
+ - !ruby/object:Gem::Version
83
+ hash: 3
84
+ segments:
85
+ - 0
86
+ version: "0"
87
+ name: rspec
88
+ prerelease: false
89
+ type: :development
90
+ requirement: *id005
91
+ - !ruby/object:Gem::Dependency
92
+ version_requirements: &id006 !ruby/object:Gem::Requirement
93
+ none: false
94
+ requirements:
95
+ - - ">="
96
+ - !ruby/object:Gem::Version
97
+ hash: 3
98
+ segments:
99
+ - 0
100
+ version: "0"
101
+ name: fakeredis
102
+ prerelease: false
103
+ type: :development
104
+ requirement: *id006
105
+ description: Statsistics collecting locking
106
+ email:
107
+ - kimmo.lehto@gmail.com
108
+ executables: []
109
+
110
+ extensions: []
111
+
112
+ extra_rdoc_files: []
113
+
114
+ files:
115
+ - .gitignore
116
+ - Gemfile
117
+ - LICENSE
118
+ - LICENSE.txt
119
+ - README.md
120
+ - Rakefile
121
+ - lib/lockistics.rb
122
+ - lib/lockistics/configuration.rb
123
+ - lib/lockistics/lock.rb
124
+ - lib/lockistics/meter.rb
125
+ - lib/lockistics/statistics.rb
126
+ - lib/lockistics/version.rb
127
+ - lockistics.gemspec
128
+ - spec/locking_spec.rb
129
+ - spec/meter_spec.rb
130
+ - spec/spec_helper.rb
131
+ - spec/statistics_spec.rb
132
+ homepage: https://github.com/kke/lockistics
133
+ licenses:
134
+ - MIT
135
+ post_install_message:
136
+ rdoc_options: []
137
+
138
+ require_paths:
139
+ - lib
140
+ required_ruby_version: !ruby/object:Gem::Requirement
141
+ none: false
142
+ requirements:
143
+ - - ">="
144
+ - !ruby/object:Gem::Version
145
+ hash: 3
146
+ segments:
147
+ - 0
148
+ version: "0"
149
+ required_rubygems_version: !ruby/object:Gem::Requirement
150
+ none: false
151
+ requirements:
152
+ - - ">="
153
+ - !ruby/object:Gem::Version
154
+ hash: 3
155
+ segments:
156
+ - 0
157
+ version: "0"
158
+ requirements: []
159
+
160
+ rubyforge_project:
161
+ rubygems_version: 1.8.25
162
+ signing_key:
163
+ specification_version: 3
164
+ summary: With lockistics you can use Redis to create distributed locks and collect statistics how often and how long your locks are held
165
+ test_files:
166
+ - spec/locking_spec.rb
167
+ - spec/meter_spec.rb
168
+ - spec/spec_helper.rb
169
+ - spec/statistics_spec.rb