lock_and_cache 2.0.2 → 2.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG +8 -0
- data/README.md +90 -15
- data/lib/lock_and_cache.rb +50 -119
- data/lib/lock_and_cache/action.rb +105 -0
- data/lib/lock_and_cache/key.rb +81 -0
- data/lib/lock_and_cache/version.rb +1 -1
- data/spec/lock_and_cache_spec.rb +124 -11
- metadata +4 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 8b4c2d0489b674d6efd75d4a368fda53ffc7f869
|
4
|
+
data.tar.gz: ededebda48c306638c3d19c2047f1fff49dc00fc
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 4837ba9ea728c9b330e489aebd0cd0d0f13ddbb7d99259ccfd958edbb9bafb4f58ac1e7f2dd37f3686d03be9d823b6a970df21b313306fff88f75a6b6c0b2a82
|
7
|
+
data.tar.gz: 2f07f2ec076d96bc6f6eb7febb2538daf1e3af305ba2947c227a1da6ff586d8cd7cfc56fb86418f6b3b353e518712d86a6dfe00496c759d46f3c864b40a2437e
|
data/CHANGELOG
CHANGED
data/README.md
CHANGED
@@ -30,7 +30,7 @@ As you can see, most caching libraries only take care of (1) and (4).
|
|
30
30
|
|
31
31
|
## Practice
|
32
32
|
|
33
|
-
### Locking
|
33
|
+
### Locking
|
34
34
|
|
35
35
|
Based on [antirez's Redlock algorithm](http://redis.io/topics/distlock).
|
36
36
|
|
@@ -42,36 +42,104 @@ LockAndCache.storage = Redis.new
|
|
42
42
|
|
43
43
|
It will use this redis for both locking and storing cached values.
|
44
44
|
|
45
|
-
### Caching
|
45
|
+
### Caching
|
46
46
|
|
47
47
|
(be sure to set up storage as above)
|
48
48
|
|
49
|
-
|
49
|
+
#### Standalone mode
|
50
50
|
|
51
51
|
```ruby
|
52
|
-
|
53
|
-
|
54
|
-
|
52
|
+
LockAndCache.lock_and_cache('stock_price') do
|
53
|
+
# get yer stock quote
|
54
|
+
end
|
55
|
+
```
|
56
|
+
|
57
|
+
But that's probably not very useful without parameters
|
58
|
+
|
59
|
+
```ruby
|
60
|
+
LockAndCache.lock_and_cache('stock_price', company: 'MSFT', date: '2015-05-05') do
|
61
|
+
# get yer stock quote
|
62
|
+
end
|
63
|
+
```
|
64
|
+
|
65
|
+
And you probably want an expiry
|
66
|
+
|
67
|
+
```ruby
|
68
|
+
LockAndCache.lock_and_cache('stock_price', {company: 'MSFT', date: '2015-05-05'}, expires: 10) do
|
69
|
+
# get yer stock quote
|
70
|
+
end
|
71
|
+
```
|
72
|
+
|
73
|
+
Note how we separated options (`{expires: 10}`) from a hash that is part of the cache key (`{company: 'MSFT', date: '2015-05-05'}`).
|
74
|
+
|
75
|
+
You can clear a cache:
|
76
|
+
|
77
|
+
```ruby
|
78
|
+
LockAndCache.lock_and_cache('stock_price', company: 'MSFT', date: '2015-05-05')
|
79
|
+
```
|
80
|
+
|
81
|
+
One other crazy thing: let's say you want to check more often if the external stock price service returned nil
|
82
|
+
|
83
|
+
```ruby
|
84
|
+
LockAndCache.lock_and_cache('stock_price', {company: 'MSFT', date: '2015-05-05'}, expires: 10, nil_expires: 1) do
|
85
|
+
# get yer stock quote
|
86
|
+
end
|
87
|
+
```
|
88
|
+
|
89
|
+
#### Context mode
|
90
|
+
|
91
|
+
"Context mode" simply adds the class name, method name, and context key (the results of `#id` or `#lock_and_cache_key`) of the caller to the cache key.
|
92
|
+
|
93
|
+
(This gem evolved from https://github.com/seamusabshere/cache_method, where you always cached a method call...)
|
94
|
+
|
95
|
+
```ruby
|
96
|
+
class Stock
|
97
|
+
include LockAndCache
|
98
|
+
|
99
|
+
def initialize(ticker_symbol)
|
100
|
+
[...]
|
101
|
+
end
|
102
|
+
|
103
|
+
def price(date)
|
104
|
+
lock_and_cache(date, expires: 10) do # <------ see how the cache key depends on the method args?
|
55
105
|
# do the work
|
56
106
|
end
|
57
107
|
end
|
108
|
+
|
109
|
+
def lock_and_cache_key # <---------- if you don't define this, it will try to call #id
|
110
|
+
ticker_symbol
|
111
|
+
end
|
58
112
|
end
|
59
113
|
```
|
60
114
|
|
61
|
-
The key will be `{
|
115
|
+
The key will be `{ StockQuote, :get, $id, $date, }`. In other words, it auto-detects the class, method, context key ... and you add other args if you want.
|
62
116
|
|
63
|
-
|
117
|
+
Here's how to clear a cache in context mode:
|
64
118
|
|
65
119
|
```ruby
|
66
|
-
|
67
|
-
# [...]
|
68
|
-
# if you don't define this, it will try to call #id
|
69
|
-
def lock_and_cache_key
|
70
|
-
[author, title]
|
71
|
-
end
|
72
|
-
end
|
120
|
+
blog.lock_and_cache_clear(:get, date)
|
73
121
|
```
|
74
122
|
|
123
|
+
## Special features
|
124
|
+
|
125
|
+
### Locking of course!
|
126
|
+
|
127
|
+
Most caching libraries don't do locking, meaning that >1 process can be calculating a cached value at the same time. Since you presumably cache things because they cost CPU, database reads, or money, doesn't it make sense to lock while caching?
|
128
|
+
|
129
|
+
### Heartbeat
|
130
|
+
|
131
|
+
If the process holding the lock dies, we automatically remove the lock so somebody else can do it (using heartbeats and redlock extends).
|
132
|
+
|
133
|
+
### Context mode
|
134
|
+
|
135
|
+
This pulls information about the context of a lock_and_cache block from the surrounding class, method, and object... so that you don't have to!
|
136
|
+
|
137
|
+
Standalone mode is cool too, tho.
|
138
|
+
|
139
|
+
### nil_expires
|
140
|
+
|
141
|
+
You can expire nil values with a different timeout (`nil_expires`) than other values (`expires`).
|
142
|
+
|
75
143
|
## Tunables
|
76
144
|
|
77
145
|
* `LockAndCache.storage=[redis]`
|
@@ -83,6 +151,13 @@ end
|
|
83
151
|
* [redis](https://github.com/redis/redis-rb)
|
84
152
|
* [redlock](https://github.com/leandromoreira/redlock-rb)
|
85
153
|
|
154
|
+
## Wishlist
|
155
|
+
|
156
|
+
* Convert most tests to use standalone mode, which is easier to understand
|
157
|
+
* Check options
|
158
|
+
* Lengthen heartbeat so it's not so sensitive
|
159
|
+
* Clarify which options are seconds or milliseconds
|
160
|
+
|
86
161
|
## Contributing
|
87
162
|
|
88
163
|
1. Fork it ( https://github.com/[my-github-username]/lock_and_cache/fork )
|
data/lib/lock_and_cache.rb
CHANGED
@@ -1,4 +1,3 @@
|
|
1
|
-
require 'lock_and_cache/version'
|
2
1
|
require 'timeout'
|
3
2
|
require 'digest/sha1'
|
4
3
|
require 'base64'
|
@@ -7,6 +6,10 @@ require 'redlock'
|
|
7
6
|
require 'active_support'
|
8
7
|
require 'active_support/core_ext'
|
9
8
|
|
9
|
+
require_relative 'lock_and_cache/version'
|
10
|
+
require_relative 'lock_and_cache/action'
|
11
|
+
require_relative 'lock_and_cache/key'
|
12
|
+
|
10
13
|
# Lock and cache methods using redis!
|
11
14
|
#
|
12
15
|
# I bet you're caching, but are you locking?
|
@@ -33,13 +36,37 @@ module LockAndCache
|
|
33
36
|
@storage
|
34
37
|
end
|
35
38
|
|
36
|
-
# Flush LockAndCache's storage
|
39
|
+
# Flush LockAndCache's storage.
|
37
40
|
#
|
38
41
|
# @note If you are sharing a redis database, it will clear it...
|
42
|
+
#
|
43
|
+
# @note If you want to clear a single key, try `LockAndCache.clear(key)` (standalone mode) or `#lock_and_cache_clear(method_id, *key_parts)` in context mode.
|
39
44
|
def LockAndCache.flush
|
40
45
|
storage.flushdb
|
41
46
|
end
|
42
47
|
|
48
|
+
# Lock and cache based on a key.
|
49
|
+
#
|
50
|
+
# @param key_parts [*] Parts that should be used to construct a key.
|
51
|
+
#
|
52
|
+
# @note Standalone mode. See also "context mode," where you mix LockAndCache into a class and call it from within its methods.
|
53
|
+
#
|
54
|
+
# @note A single hash arg is treated as a cached key. `LockAndCache.lock_and_cache(foo: :bar, expires: 100)` will be treated as a cache key of `foo: :bar, expires: 100` (which is probably wrong!!!). `LockAndCache.lock_and_cache({foo: :bar}, expires: 100)` will be treated as a cache key of `foo: :bar` and options `expires: 100`. This is the opposite of context mode and is true because we don't have any context to set the cache key from otherwise.
|
55
|
+
def LockAndCache.lock_and_cache(*key_parts_and_options, &blk)
|
56
|
+
options = (key_parts_and_options.last.is_a?(Hash) && key_parts_and_options.length > 1) ? key_parts_and_options.pop : {}
|
57
|
+
key = LockAndCache::Key.new key_parts_and_options
|
58
|
+
action = LockAndCache::Action.new key, options, blk
|
59
|
+
action.perform
|
60
|
+
end
|
61
|
+
|
62
|
+
# Clear a single key
|
63
|
+
#
|
64
|
+
# @note Standalone mode. See also "context mode," where you mix LockAndCache into a class and call it from within its methods.
|
65
|
+
def LockAndCache.clear(*key_parts)
|
66
|
+
key = LockAndCache::Key.new key_parts
|
67
|
+
key.clear
|
68
|
+
end
|
69
|
+
|
43
70
|
# @param seconds [Numeric] Maximum wait time to get a lock
|
44
71
|
#
|
45
72
|
# @note Can be overridden by putting `max_lock_wait:` in your call to `#lock_and_cache`
|
@@ -57,133 +84,37 @@ module LockAndCache
|
|
57
84
|
@lock_manager
|
58
85
|
end
|
59
86
|
|
60
|
-
#
|
61
|
-
|
62
|
-
|
63
|
-
attr_reader :method_id
|
64
|
-
|
65
|
-
def initialize(obj, method_id, parts)
|
66
|
-
@obj = obj
|
67
|
-
@method_id = method_id.to_sym
|
68
|
-
@_parts = parts
|
69
|
-
end
|
70
|
-
|
71
|
-
# A (non-cryptographic) digest of the key parts for use as the cache key
|
72
|
-
def digest
|
73
|
-
@digest ||= ::Digest::SHA1.hexdigest ::Marshal.dump(key)
|
74
|
-
end
|
75
|
-
|
76
|
-
# A (non-cryptographic) digest of the key parts for use as the lock key
|
77
|
-
def lock_digest
|
78
|
-
@lock_digest ||= 'lock/' + digest
|
79
|
-
end
|
80
|
-
|
81
|
-
# A human-readable representation of the key parts
|
82
|
-
def key
|
83
|
-
@key ||= [obj_class_name, method_id, parts]
|
84
|
-
end
|
85
|
-
|
86
|
-
alias debug key
|
87
|
-
|
88
|
-
# An array of the parts we use for the key
|
89
|
-
def parts
|
90
|
-
@parts ||= @_parts.map do |v|
|
91
|
-
case v
|
92
|
-
when ::String, ::Symbol, ::Hash, ::Array
|
93
|
-
v
|
94
|
-
else
|
95
|
-
v.respond_to?(:lock_and_cache_key) ? v.lock_and_cache_key : v.id
|
96
|
-
end
|
97
|
-
end
|
98
|
-
end
|
99
|
-
|
100
|
-
# An object (or its class's) name
|
101
|
-
def obj_class_name
|
102
|
-
@obj_class_name ||= (obj.class == ::Class) ? obj.name : obj.class.name
|
103
|
-
end
|
104
|
-
|
105
|
-
end
|
106
|
-
|
87
|
+
# Check if a method is locked on an object.
|
88
|
+
#
|
89
|
+
# @note Subject mode - this is expected to be called on an object that has LockAndCache mixed in. See also standalone mode.
|
107
90
|
def lock_and_cache_locked?(method_id, *key_parts)
|
108
|
-
|
109
|
-
key
|
110
|
-
LockAndCache.storage.exists key.lock_digest
|
91
|
+
key = LockAndCache::Key.new key_parts, context: self, method_id: method_id
|
92
|
+
key.locked?
|
111
93
|
end
|
112
94
|
|
113
95
|
# Clear a lock and cache given exactly the method and exactly the same arguments
|
96
|
+
#
|
97
|
+
# @note Subject mode - this is expected to be called on an object that has LockAndCache mixed in. See also standalone mode.
|
114
98
|
def lock_and_cache_clear(method_id, *key_parts)
|
115
|
-
|
116
|
-
key
|
117
|
-
Thread.exclusive { $stderr.puts "[lock_and_cache] clear #{key.debug} #{Base64.encode64(key.digest).strip} #{Digest::SHA1.hexdigest key.digest}" } if debug
|
118
|
-
LockAndCache.storage.del key.digest
|
119
|
-
LockAndCache.storage.del key.lock_digest
|
99
|
+
key = LockAndCache::Key.new key_parts, context: self, method_id: method_id
|
100
|
+
key.clear
|
120
101
|
end
|
121
102
|
|
122
103
|
# Lock and cache a method given key parts.
|
123
104
|
#
|
124
|
-
#
|
105
|
+
# This is the defining characteristic of context mode: the cache key will automatically include the class name of the object calling it (the context!) and the name of the method it is called from.
|
106
|
+
#
|
107
|
+
# @param key_parts_and_options [*] Parts that you want to include in the lock and cache key. If the last element is a Hash, it will be treated as options.
|
125
108
|
#
|
126
109
|
# @return The cached value (possibly newly calculated).
|
127
|
-
|
128
|
-
|
129
|
-
|
130
|
-
|
131
|
-
|
132
|
-
options =
|
133
|
-
|
134
|
-
|
135
|
-
|
136
|
-
digest = key.digest
|
137
|
-
storage = LockAndCache.storage or raise("must set LockAndCache.storage=[Redis]")
|
138
|
-
Thread.exclusive { $stderr.puts "[lock_and_cache] A1 #{key.debug} #{Base64.encode64(digest).strip} #{Digest::SHA1.hexdigest digest}" } if debug
|
139
|
-
if storage.exists(digest) and (existing = storage.get(digest)).is_a?(String)
|
140
|
-
return ::Marshal.load(existing)
|
141
|
-
end
|
142
|
-
Thread.exclusive { $stderr.puts "[lock_and_cache] B1 #{key.debug} #{Base64.encode64(digest).strip} #{Digest::SHA1.hexdigest digest}" } if debug
|
143
|
-
retval = nil
|
144
|
-
lock_manager = LockAndCache.lock_manager
|
145
|
-
lock_digest = key.lock_digest
|
146
|
-
lock_info = nil
|
147
|
-
begin
|
148
|
-
Timeout.timeout(max_lock_wait, TimeoutWaitingForLock) do
|
149
|
-
until lock_info = lock_manager.lock(lock_digest, LockAndCache::LOCK_HEARTBEAT_EXPIRES*1000)
|
150
|
-
Thread.exclusive { $stderr.puts "[lock_and_cache] C1 #{key.debug} #{Base64.encode64(digest).strip} #{Digest::SHA1.hexdigest digest}" } if debug
|
151
|
-
sleep rand
|
152
|
-
end
|
153
|
-
end
|
154
|
-
Thread.exclusive { $stderr.puts "[lock_and_cache] D1 #{key.debug} #{Base64.encode64(digest).strip} #{Digest::SHA1.hexdigest digest}" } if debug
|
155
|
-
if storage.exists(digest) and (existing = storage.get(digest)).is_a?(String)
|
156
|
-
Thread.exclusive { $stderr.puts "[lock_and_cache] E1 #{key.debug} #{Base64.encode64(digest).strip} #{Digest::SHA1.hexdigest digest}" } if debug
|
157
|
-
retval = ::Marshal.load existing
|
158
|
-
end
|
159
|
-
unless retval
|
160
|
-
Thread.exclusive { $stderr.puts "[lock_and_cache] F1 #{key.debug} #{Base64.encode64(digest).strip} #{Digest::SHA1.hexdigest digest}" } if debug
|
161
|
-
done = false
|
162
|
-
begin
|
163
|
-
lock_extender = Thread.new do
|
164
|
-
loop do
|
165
|
-
Thread.exclusive { $stderr.puts "[lock_and_cache] heartbeat1 #{key.debug} #{Base64.encode64(digest).strip} #{Digest::SHA1.hexdigest digest}" } if debug
|
166
|
-
break if done
|
167
|
-
sleep LockAndCache::LOCK_HEARTBEAT_PERIOD
|
168
|
-
break if done
|
169
|
-
Thread.exclusive { $stderr.puts "[lock_and_cache] heartbeat2 #{key.debug} #{Base64.encode64(digest).strip} #{Digest::SHA1.hexdigest digest}" } if debug
|
170
|
-
lock_manager.lock lock_digest, LockAndCache::LOCK_HEARTBEAT_EXPIRES*1000, extend: lock_info
|
171
|
-
end
|
172
|
-
end
|
173
|
-
retval = yield
|
174
|
-
if expires
|
175
|
-
storage.setex digest, expires, ::Marshal.dump(retval)
|
176
|
-
else
|
177
|
-
storage.set digest, ::Marshal.dump(retval)
|
178
|
-
end
|
179
|
-
ensure
|
180
|
-
done = true
|
181
|
-
lock_extender.join if lock_extender.status.nil?
|
182
|
-
end
|
183
|
-
end
|
184
|
-
ensure
|
185
|
-
lock_manager.unlock lock_info if lock_info
|
186
|
-
end
|
187
|
-
retval
|
110
|
+
#
|
111
|
+
# @note Subject mode - this is expected to be called on an object that has LockAndCache mixed in. See also standalone mode.
|
112
|
+
#
|
113
|
+
# @note A single hash arg is treated as an options hash. `lock_and_cache(expires: 100)` will be treated as options `expires: 100`. This is the opposite of standalone mode and true because we want to support people constructing cache keys from the context (context) PLUS an arbitrary hash of stuff.
|
114
|
+
def lock_and_cache(*key_parts_and_options, &blk)
|
115
|
+
options = key_parts_and_options.last.is_a?(Hash) ? key_parts_and_options.pop : {}
|
116
|
+
key = LockAndCache::Key.new key_parts_and_options, context: self, caller: caller
|
117
|
+
action = LockAndCache::Action.new key, options, blk
|
118
|
+
action.perform
|
188
119
|
end
|
189
120
|
end
|
@@ -0,0 +1,105 @@
|
|
1
|
+
module LockAndCache
|
2
|
+
# @private
|
3
|
+
class Action
|
4
|
+
attr_reader :key
|
5
|
+
attr_reader :options
|
6
|
+
attr_reader :blk
|
7
|
+
|
8
|
+
def initialize(key, options, blk)
|
9
|
+
raise "need a block" unless blk
|
10
|
+
@key = key
|
11
|
+
@options = options.stringify_keys
|
12
|
+
@blk = blk
|
13
|
+
end
|
14
|
+
|
15
|
+
def expires
|
16
|
+
options['expires']
|
17
|
+
end
|
18
|
+
|
19
|
+
def nil_expires
|
20
|
+
options['nil_expires']
|
21
|
+
end
|
22
|
+
|
23
|
+
def digest
|
24
|
+
@digest ||= key.digest
|
25
|
+
end
|
26
|
+
|
27
|
+
def lock_digest
|
28
|
+
@lock_digest ||= key.lock_digest
|
29
|
+
end
|
30
|
+
|
31
|
+
def storage
|
32
|
+
@storage ||= LockAndCache.storage or raise("must set LockAndCache.storage=[Redis]")
|
33
|
+
end
|
34
|
+
|
35
|
+
def perform
|
36
|
+
debug = (ENV['LOCK_AND_CACHE_DEBUG'] == 'true')
|
37
|
+
max_lock_wait = options.fetch 'max_lock_wait', LockAndCache.max_lock_wait
|
38
|
+
Thread.exclusive { $stderr.puts "[lock_and_cache] A1 #{key.debug} #{Base64.encode64(digest).strip} #{Digest::SHA1.hexdigest digest}" } if debug
|
39
|
+
if storage.exists(digest) and (existing = storage.get(digest)).is_a?(String)
|
40
|
+
return ::Marshal.load(existing)
|
41
|
+
end
|
42
|
+
Thread.exclusive { $stderr.puts "[lock_and_cache] B1 #{key.debug} #{Base64.encode64(digest).strip} #{Digest::SHA1.hexdigest digest}" } if debug
|
43
|
+
retval = nil
|
44
|
+
lock_manager = LockAndCache.lock_manager
|
45
|
+
lock_info = nil
|
46
|
+
begin
|
47
|
+
Timeout.timeout(max_lock_wait, TimeoutWaitingForLock) do
|
48
|
+
until lock_info = lock_manager.lock(lock_digest, LockAndCache::LOCK_HEARTBEAT_EXPIRES*1000)
|
49
|
+
Thread.exclusive { $stderr.puts "[lock_and_cache] C1 #{key.debug} #{Base64.encode64(digest).strip} #{Digest::SHA1.hexdigest digest}" } if debug
|
50
|
+
sleep rand
|
51
|
+
end
|
52
|
+
end
|
53
|
+
Thread.exclusive { $stderr.puts "[lock_and_cache] D1 #{key.debug} #{Base64.encode64(digest).strip} #{Digest::SHA1.hexdigest digest}" } if debug
|
54
|
+
if storage.exists(digest) and (existing = storage.get(digest)).is_a?(String)
|
55
|
+
Thread.exclusive { $stderr.puts "[lock_and_cache] E1 #{key.debug} #{Base64.encode64(digest).strip} #{Digest::SHA1.hexdigest digest}" } if debug
|
56
|
+
retval = ::Marshal.load existing
|
57
|
+
end
|
58
|
+
unless retval
|
59
|
+
Thread.exclusive { $stderr.puts "[lock_and_cache] F1 #{key.debug} #{Base64.encode64(digest).strip} #{Digest::SHA1.hexdigest digest}" } if debug
|
60
|
+
done = false
|
61
|
+
begin
|
62
|
+
lock_extender = Thread.new do
|
63
|
+
loop do
|
64
|
+
Thread.exclusive { $stderr.puts "[lock_and_cache] heartbeat1 #{key.debug} #{Base64.encode64(digest).strip} #{Digest::SHA1.hexdigest digest}" } if debug
|
65
|
+
break if done
|
66
|
+
sleep LockAndCache::LOCK_HEARTBEAT_PERIOD
|
67
|
+
break if done
|
68
|
+
Thread.exclusive { $stderr.puts "[lock_and_cache] heartbeat2 #{key.debug} #{Base64.encode64(digest).strip} #{Digest::SHA1.hexdigest digest}" } if debug
|
69
|
+
lock_manager.lock lock_digest, LockAndCache::LOCK_HEARTBEAT_EXPIRES*1000, extend: lock_info
|
70
|
+
end
|
71
|
+
end
|
72
|
+
retval = blk.call
|
73
|
+
retval.nil? ? set_nil : set_non_nil(retval)
|
74
|
+
ensure
|
75
|
+
done = true
|
76
|
+
lock_extender.join if lock_extender.status.nil?
|
77
|
+
end
|
78
|
+
end
|
79
|
+
ensure
|
80
|
+
lock_manager.unlock lock_info if lock_info
|
81
|
+
end
|
82
|
+
retval
|
83
|
+
end
|
84
|
+
|
85
|
+
NIL = Marshal.dump nil
|
86
|
+
def set_nil
|
87
|
+
if nil_expires
|
88
|
+
storage.setex digest, nil_expires, NIL
|
89
|
+
elsif expires
|
90
|
+
storage.setex digest, expires, NIL
|
91
|
+
else
|
92
|
+
storage.set digest, NIL
|
93
|
+
end
|
94
|
+
end
|
95
|
+
|
96
|
+
def set_non_nil(retval)
|
97
|
+
raise "expected not null #{retval.inspect}" if retval.nil?
|
98
|
+
if expires
|
99
|
+
storage.setex digest, expires, ::Marshal.dump(retval)
|
100
|
+
else
|
101
|
+
storage.set digest, ::Marshal.dump(retval)
|
102
|
+
end
|
103
|
+
end
|
104
|
+
end
|
105
|
+
end
|
@@ -0,0 +1,81 @@
|
|
1
|
+
module LockAndCache
|
2
|
+
# @private
|
3
|
+
class Key
|
4
|
+
class << self
|
5
|
+
# @private
|
6
|
+
#
|
7
|
+
# Extract the method id from a method's caller array.
|
8
|
+
def extract_method_id_from_caller(kaller)
|
9
|
+
kaller[0] =~ METHOD_NAME_IN_CALLER
|
10
|
+
raise "couldn't get method_id from #{kaller[0]}" unless $1
|
11
|
+
$1.to_sym
|
12
|
+
end
|
13
|
+
|
14
|
+
# @private
|
15
|
+
#
|
16
|
+
# Get a context object's class name, which is its own name if it's an object.
|
17
|
+
def extract_class_name(context)
|
18
|
+
(self.class == ::Class) ? self.name : self.class.name
|
19
|
+
end
|
20
|
+
end
|
21
|
+
|
22
|
+
METHOD_NAME_IN_CALLER = /in `([^']+)'/
|
23
|
+
|
24
|
+
attr_reader :method_id
|
25
|
+
attr_reader :class_name
|
26
|
+
|
27
|
+
def initialize(parts, options = {})
|
28
|
+
@_parts = parts
|
29
|
+
@method_id = if options.has_key?(:method_id)
|
30
|
+
options[:method_id]
|
31
|
+
elsif options.has_key?(:caller)
|
32
|
+
Key.extract_method_id_from_caller options[:caller]
|
33
|
+
end
|
34
|
+
@class_name = Key.extract_class_name options[:context] if options.has_key?(:context)
|
35
|
+
end
|
36
|
+
|
37
|
+
# A (non-cryptographic) digest of the key parts for use as the cache key
|
38
|
+
def digest
|
39
|
+
@digest ||= ::Digest::SHA1.hexdigest ::Marshal.dump(key)
|
40
|
+
end
|
41
|
+
|
42
|
+
# A (non-cryptographic) digest of the key parts for use as the lock key
|
43
|
+
def lock_digest
|
44
|
+
@lock_digest ||= 'lock/' + digest
|
45
|
+
end
|
46
|
+
|
47
|
+
# A human-readable representation of the key parts
|
48
|
+
def key
|
49
|
+
@key ||= if method_id
|
50
|
+
[class_name, method_id, parts]
|
51
|
+
else
|
52
|
+
parts
|
53
|
+
end
|
54
|
+
end
|
55
|
+
|
56
|
+
def locked?
|
57
|
+
LockAndCache.storage.exists lock_digest
|
58
|
+
end
|
59
|
+
|
60
|
+
def clear
|
61
|
+
Thread.exclusive { $stderr.puts "[lock_and_cache] clear #{debug} #{Base64.encode64(digest).strip} #{Digest::SHA1.hexdigest digest}" } if ENV['LOCK_AND_CACHE_DEBUG'] == 'true'
|
62
|
+
storage = LockAndCache.storage
|
63
|
+
storage.del digest
|
64
|
+
storage.del lock_digest
|
65
|
+
end
|
66
|
+
|
67
|
+
alias debug key
|
68
|
+
|
69
|
+
# An array of the parts we use for the key
|
70
|
+
def parts
|
71
|
+
@parts ||= @_parts.map do |v|
|
72
|
+
case v
|
73
|
+
when ::String, ::Symbol, ::Hash, ::Array
|
74
|
+
v
|
75
|
+
else
|
76
|
+
v.respond_to?(:lock_and_cache_key) ? v.lock_and_cache_key : v.id
|
77
|
+
end
|
78
|
+
end
|
79
|
+
end
|
80
|
+
end
|
81
|
+
end
|
data/spec/lock_and_cache_spec.rb
CHANGED
@@ -7,6 +7,8 @@ class Foo
|
|
7
7
|
@id = id
|
8
8
|
@count = 0
|
9
9
|
@count_exp = 0
|
10
|
+
@click_single_hash_arg_as_options = 0
|
11
|
+
@click_last_hash_as_options = 0
|
10
12
|
end
|
11
13
|
|
12
14
|
def click
|
@@ -22,11 +24,25 @@ class Foo
|
|
22
24
|
end
|
23
25
|
|
24
26
|
def click_exp
|
25
|
-
lock_and_cache(expires: 1
|
27
|
+
lock_and_cache(expires: 1) do
|
26
28
|
@count_exp += 1
|
27
29
|
end
|
28
30
|
end
|
29
31
|
|
32
|
+
# foo will be treated as option, so this is cacheable
|
33
|
+
def click_single_hash_arg_as_options
|
34
|
+
lock_and_cache(foo: rand, expires: 1) do
|
35
|
+
@click_single_hash_arg_as_options += 1
|
36
|
+
end
|
37
|
+
end
|
38
|
+
|
39
|
+
# foo will be treated as part of cache key, so this is uncacheable
|
40
|
+
def click_last_hash_as_options
|
41
|
+
lock_and_cache({foo: rand}, expires: 1) do
|
42
|
+
@click_last_hash_as_options += 1
|
43
|
+
end
|
44
|
+
end
|
45
|
+
|
30
46
|
def lock_and_cache_key
|
31
47
|
@id
|
32
48
|
end
|
@@ -114,7 +130,7 @@ describe LockAndCache do
|
|
114
130
|
it "can be expired" do
|
115
131
|
expect(foo.click_exp).to eq(1)
|
116
132
|
expect(foo.click_exp).to eq(1)
|
117
|
-
sleep
|
133
|
+
sleep 1.5
|
118
134
|
expect(foo.click_exp).to eq(2)
|
119
135
|
end
|
120
136
|
|
@@ -122,6 +138,20 @@ describe LockAndCache do
|
|
122
138
|
expect(foo.click_null).to eq(nil)
|
123
139
|
expect(foo.click_null).to eq(nil)
|
124
140
|
end
|
141
|
+
|
142
|
+
it "treats single hash arg as options" do
|
143
|
+
expect(foo.click_single_hash_arg_as_options).to eq(1)
|
144
|
+
expect(foo.click_single_hash_arg_as_options).to eq(1)
|
145
|
+
sleep 1.1
|
146
|
+
expect(foo.click_single_hash_arg_as_options).to eq(2)
|
147
|
+
end
|
148
|
+
|
149
|
+
it "treats last hash as options" do
|
150
|
+
expect(foo.click_last_hash_as_options).to eq(1)
|
151
|
+
expect(foo.click_last_hash_as_options).to eq(2) # it's uncacheable to prove we're not using as part of options
|
152
|
+
expect(foo.click_last_hash_as_options).to eq(3)
|
153
|
+
end
|
154
|
+
|
125
155
|
end
|
126
156
|
|
127
157
|
describe "locking" do
|
@@ -237,17 +267,100 @@ describe LockAndCache do
|
|
237
267
|
|
238
268
|
end
|
239
269
|
|
240
|
-
describe '
|
241
|
-
it
|
242
|
-
|
243
|
-
|
244
|
-
expect(
|
270
|
+
describe 'standalone' do
|
271
|
+
it 'works like you expect' do
|
272
|
+
count = 0
|
273
|
+
expect(LockAndCache.lock_and_cache('hello') { count += 1 }).to eq(1)
|
274
|
+
expect(count).to eq(1)
|
275
|
+
expect(LockAndCache.lock_and_cache('hello') { count += 1 }).to eq(1)
|
276
|
+
expect(count).to eq(1)
|
277
|
+
end
|
278
|
+
|
279
|
+
it 'allows expiry' do
|
280
|
+
count = 0
|
281
|
+
expect(LockAndCache.lock_and_cache('hello', expires: 1) { count += 1 }).to eq(1)
|
282
|
+
expect(count).to eq(1)
|
283
|
+
expect(LockAndCache.lock_and_cache('hello') { count += 1 }).to eq(1)
|
284
|
+
expect(count).to eq(1)
|
285
|
+
sleep 1.1
|
286
|
+
expect(LockAndCache.lock_and_cache('hello') { count += 1 }).to eq(2)
|
287
|
+
expect(count).to eq(2)
|
288
|
+
end
|
289
|
+
|
290
|
+
it 'allows clearing' do
|
291
|
+
count = 0
|
292
|
+
expect(LockAndCache.lock_and_cache('hello') { count += 1 }).to eq(1)
|
293
|
+
expect(count).to eq(1)
|
294
|
+
LockAndCache.clear('hello')
|
295
|
+
expect(LockAndCache.lock_and_cache('hello') { count += 1 }).to eq(2)
|
296
|
+
expect(count).to eq(2)
|
297
|
+
end
|
298
|
+
|
299
|
+
it 'allows multi-part keys' do
|
300
|
+
count = 0
|
301
|
+
expect(LockAndCache.lock_and_cache(['hello', 1, { target: 'world' }]) { count += 1 }).to eq(1)
|
302
|
+
expect(count).to eq(1)
|
303
|
+
expect(LockAndCache.lock_and_cache(['hello', 1, { target: 'world' }]) { count += 1 }).to eq(1)
|
304
|
+
expect(count).to eq(1)
|
305
|
+
end
|
306
|
+
|
307
|
+
it 'treats a single hash arg as a cache key (not as options)' do
|
308
|
+
count = 0
|
309
|
+
LockAndCache.lock_and_cache(hello: 'world', expires: 100) { count += 1 }
|
310
|
+
expect(count).to eq(1)
|
311
|
+
LockAndCache.lock_and_cache(hello: 'world', expires: 100) { count += 1 }
|
312
|
+
expect(count).to eq(1)
|
313
|
+
LockAndCache.lock_and_cache(hello: 'world', expires: 200) { count += 1 } # expires is being treated as part of cache key
|
314
|
+
expect(count).to eq(2)
|
315
|
+
end
|
316
|
+
|
317
|
+
it "correctly identifies options hash" do
|
318
|
+
count = 0
|
319
|
+
LockAndCache.lock_and_cache({ hello: 'world' }, expires: 1, ignored: rand) { count += 1 }
|
320
|
+
expect(count).to eq(1)
|
321
|
+
LockAndCache.lock_and_cache({ hello: 'world' }, expires: 1, ignored: rand) { count += 1 } # expires is not being treated as part of cache key
|
322
|
+
expect(count).to eq(1)
|
323
|
+
sleep 1.1
|
324
|
+
LockAndCache.lock_and_cache({ hello: 'world' }) { count += 1 }
|
325
|
+
expect(count).to eq(2)
|
326
|
+
end
|
327
|
+
end
|
328
|
+
|
329
|
+
describe "shorter expiry for null results" do
|
330
|
+
it "optionally caches null for less time" do
|
331
|
+
count = 0
|
332
|
+
LockAndCache.lock_and_cache('hello', nil_expires: 1, expires: 2) { count += 1; nil }
|
333
|
+
expect(count).to eq(1)
|
334
|
+
LockAndCache.lock_and_cache('hello', nil_expires: 1, expires: 2) { count += 1; nil }
|
335
|
+
expect(count).to eq(1)
|
336
|
+
sleep 1.1 # this is enough to expire
|
337
|
+
LockAndCache.lock_and_cache('hello', nil_expires: 1, expires: 2) { count += 1; nil }
|
338
|
+
expect(count).to eq(2)
|
245
339
|
end
|
246
340
|
|
247
|
-
it "
|
248
|
-
|
249
|
-
|
250
|
-
expect(
|
341
|
+
it "normally caches null for the same amount of time" do
|
342
|
+
count = 0
|
343
|
+
expect(LockAndCache.lock_and_cache('hello', expires: 1) { count += 1; nil }).to be_nil
|
344
|
+
expect(count).to eq(1)
|
345
|
+
expect(LockAndCache.lock_and_cache('hello', expires: 1) { count += 1; nil }).to be_nil
|
346
|
+
expect(count).to eq(1)
|
347
|
+
sleep 1.1
|
348
|
+
expect(LockAndCache.lock_and_cache('hello', expires: 1) { count += 1; nil }).to be_nil
|
349
|
+
expect(count).to eq(2)
|
350
|
+
end
|
351
|
+
|
352
|
+
it "caches non-null for normal time" do
|
353
|
+
count = 0
|
354
|
+
LockAndCache.lock_and_cache('hello', nil_expires: 1, expires: 2) { count += 1; true }
|
355
|
+
expect(count).to eq(1)
|
356
|
+
LockAndCache.lock_and_cache('hello', nil_expires: 1, expires: 2) { count += 1; true }
|
357
|
+
expect(count).to eq(1)
|
358
|
+
sleep 1.1
|
359
|
+
LockAndCache.lock_and_cache('hello', nil_expires: 1, expires: 2) { count += 1; true }
|
360
|
+
expect(count).to eq(1)
|
361
|
+
sleep 1
|
362
|
+
LockAndCache.lock_and_cache('hello', nil_expires: 1, expires: 2) { count += 1; true }
|
363
|
+
expect(count).to eq(2)
|
251
364
|
end
|
252
365
|
end
|
253
366
|
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: lock_and_cache
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 2.0
|
4
|
+
version: 2.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Seamus Abshere
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2015-10-
|
11
|
+
date: 2015-10-26 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activesupport
|
@@ -168,6 +168,8 @@ files:
|
|
168
168
|
- README.md
|
169
169
|
- Rakefile
|
170
170
|
- lib/lock_and_cache.rb
|
171
|
+
- lib/lock_and_cache/action.rb
|
172
|
+
- lib/lock_and_cache/key.rb
|
171
173
|
- lib/lock_and_cache/version.rb
|
172
174
|
- lock_and_cache.gemspec
|
173
175
|
- spec/lock_and_cache_spec.rb
|