sidekiq-lock 0.3.1 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.gitignore +1 -0
- data/.travis.yml +4 -4
- data/CHANGELOG.md +4 -0
- data/README.md +35 -7
- data/lib/sidekiq/lock.rb +11 -1
- data/lib/sidekiq/lock/container.rb +15 -0
- data/lib/sidekiq/lock/middleware.rb +2 -2
- data/lib/sidekiq/lock/testing/inline.rb +2 -2
- data/lib/sidekiq/lock/version.rb +1 -1
- data/lib/sidekiq/lock/worker.rb +1 -1
- data/test/lib/container_test.rb +23 -0
- data/test/lib/middleware_test.rb +9 -10
- data/test/lib/testing/inline_test.rb +9 -6
- data/test/lib/worker_test.rb +49 -11
- data/test/test_helper.rb +8 -4
- metadata +5 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 7fb91d967eb073b2fb524357b936d473f3ab03450f3a1e7f45f9ae53895a210e
|
4
|
+
data.tar.gz: 10e5ac0b30ea212f675d9ce11ba63da4114b7ade9f1da89bbdcd209cfaa2f40e
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 3181d04fc0b7fbb7f9807851abf37d69b09333b38fef0c8388671a750a4087a5a185fd57dce6469095fb26ee42728b2e856f623b2643a3c15a9ed244197e6d0f
|
7
|
+
data.tar.gz: b1d1bd1c051b60188853436fd62ecec774cac2e303134e12dac203d166159b51e141f4a1a626ed952b6aa4591352f11e283bd17d371174bfdd25a6c3cac9745b
|
data/.gitignore
CHANGED
data/.travis.yml
CHANGED
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,7 @@
|
|
1
|
+
## 0.4.0 (July 18, 2018)
|
2
|
+
|
3
|
+
- make lock container configurable (non breaking change) - in case you would like to something else than `Thread.current` - now you easily can
|
4
|
+
|
1
5
|
## 0.3.1 (March 3, 2018)
|
2
6
|
|
3
7
|
- do not assume `ActiveSupport` is loaded / or old `Sidekiq` patches are present (add own symbolize keys logic)
|
data/README.md
CHANGED
@@ -3,14 +3,14 @@
|
|
3
3
|
[](https://codeclimate.com/github/emq/sidekiq-lock)
|
4
4
|
[](https://travis-ci.org/emq/sidekiq-lock)
|
5
5
|
[](https://coveralls.io/r/emq/sidekiq-lock)
|
6
|
-
[](https://gemnasium.com/emq/sidekiq-lock)
|
7
6
|
[](http://badge.fury.io/rb/sidekiq-lock)
|
8
7
|
|
9
8
|
**Note:** This is a _complete_ piece of software, it should work across all future sidekiq & ruby versions.
|
10
9
|
|
11
10
|
Redis-based simple locking mechanism for [sidekiq][2]. Uses [SET command][1] introduced in Redis 2.6.16.
|
12
11
|
|
13
|
-
It can be handy if you push a lot of jobs into the queue(s), but you don't want to execute specific jobs at the same
|
12
|
+
It can be handy if you push a lot of jobs into the queue(s), but you don't want to execute specific jobs at the same
|
13
|
+
time - it provides a `lock` method that you can use in whatever way you want.
|
14
14
|
|
15
15
|
## Installation
|
16
16
|
|
@@ -34,7 +34,8 @@ $ bundle
|
|
34
34
|
|
35
35
|
Sidekiq-lock is a middleware/module combination, let me go through my thought process here :).
|
36
36
|
|
37
|
-
In your worker class include `Sidekiq::Lock::Worker` module and provide `lock` attribute inside `sidekiq_options`,
|
37
|
+
In your worker class include `Sidekiq::Lock::Worker` module and provide `lock` attribute inside `sidekiq_options`,
|
38
|
+
for example:
|
38
39
|
|
39
40
|
``` ruby
|
40
41
|
class Worker
|
@@ -52,13 +53,17 @@ end
|
|
52
53
|
|
53
54
|
What will happen is:
|
54
55
|
|
55
|
-
- middleware will setup a `Sidekiq::Lock::RedisLock` object under `Thread.current[Sidekiq::Lock::THREAD_KEY]`
|
56
|
+
- middleware will setup a `Sidekiq::Lock::RedisLock` object under `Thread.current[Sidekiq::Lock::THREAD_KEY]`
|
57
|
+
(it should work in most use cases without any problems - but it's configurable, more below) - assuming you provided
|
58
|
+
`lock` options, otherwise it will do nothing, just execute your worker's code
|
56
59
|
|
57
|
-
- `Sidekiq::Lock::Worker` module provides a `lock` method that just simply points to that thread variable, just as
|
60
|
+
- `Sidekiq::Lock::Worker` module provides a `lock` method that just simply points to that thread variable, just as
|
61
|
+
a convenience
|
58
62
|
|
59
63
|
So now in your worker class you can call (whenever you need):
|
60
64
|
|
61
|
-
- `lock.acquire!` - will try to acquire the lock, if returns false on failure (that means some other process / thread
|
65
|
+
- `lock.acquire!` - will try to acquire the lock, if returns false on failure (that means some other process / thread
|
66
|
+
took the lock first)
|
62
67
|
- `lock.acquired?` - set to `true` when lock is successfully acquired
|
63
68
|
- `lock.release!` - deletes the lock (only if it's: acquired by current thread and not already expired)
|
64
69
|
|
@@ -115,9 +120,32 @@ Sidekiq.configure_server do |config|
|
|
115
120
|
end
|
116
121
|
```
|
117
122
|
|
123
|
+
### Customizing lock _container_
|
124
|
+
|
125
|
+
If you would like to change default behavior of storing lock instance in `Thread.current` for whatever reason you
|
126
|
+
can do that as well via server configuration:
|
127
|
+
|
128
|
+
``` ruby
|
129
|
+
# Any thread-safe class that implements .fetch and .store methods will do
|
130
|
+
class CustomStorage
|
131
|
+
def fetch
|
132
|
+
# returns stored lock instance
|
133
|
+
end
|
134
|
+
|
135
|
+
def store(lock_instance)
|
136
|
+
# store lock
|
137
|
+
end
|
138
|
+
end
|
139
|
+
|
140
|
+
Sidekiq.configure_server do |config|
|
141
|
+
config.lock_container = CustomStorage.new
|
142
|
+
end
|
143
|
+
```
|
144
|
+
|
118
145
|
### Inline testing
|
119
146
|
|
120
|
-
As you know middleware is not invoked when testing jobs inline, you can require in your test/spec helper file
|
147
|
+
As you know middleware is not invoked when testing jobs inline, you can require in your test/spec helper file
|
148
|
+
`sidekiq/lock/testing/inline` to include two methods that will help you setting / clearing up lock manually:
|
121
149
|
|
122
150
|
- `set_sidekiq_lock(worker_class, payload)` - note: payload should be an array of worker arguments
|
123
151
|
- `clear_sidekiq_lock`
|
data/lib/sidekiq/lock.rb
CHANGED
@@ -1,11 +1,20 @@
|
|
1
|
+
require 'sidekiq/lock/container'
|
1
2
|
require 'sidekiq/lock/middleware'
|
2
3
|
require 'sidekiq/lock/redis_lock'
|
3
4
|
require 'sidekiq/lock/version'
|
4
5
|
require 'sidekiq/lock/worker'
|
5
6
|
|
6
7
|
module Sidekiq
|
8
|
+
def self.lock_container
|
9
|
+
@lock_container ||= Lock::Container.new
|
10
|
+
end
|
11
|
+
|
7
12
|
def self.lock_method
|
8
|
-
@lock_method ||=
|
13
|
+
@lock_method ||= Lock::METHOD_NAME
|
14
|
+
end
|
15
|
+
|
16
|
+
def self.lock_container=(container)
|
17
|
+
@lock_container = container
|
9
18
|
end
|
10
19
|
|
11
20
|
def self.lock_method=(method)
|
@@ -14,6 +23,7 @@ module Sidekiq
|
|
14
23
|
|
15
24
|
module Lock
|
16
25
|
THREAD_KEY = :sidekiq_lock
|
26
|
+
METHOD_NAME = :lock
|
17
27
|
end
|
18
28
|
end
|
19
29
|
|
@@ -1,7 +1,7 @@
|
|
1
1
|
module Sidekiq
|
2
2
|
module Lock
|
3
3
|
class Middleware
|
4
|
-
def call(worker, msg,
|
4
|
+
def call(worker, msg, _queue)
|
5
5
|
options = lock_options(worker)
|
6
6
|
setup_lock(options, msg['args']) unless options.nil?
|
7
7
|
|
@@ -11,7 +11,7 @@ module Sidekiq
|
|
11
11
|
private
|
12
12
|
|
13
13
|
def setup_lock(options, payload)
|
14
|
-
|
14
|
+
Sidekiq.lock_container.store(RedisLock.new(options, payload))
|
15
15
|
end
|
16
16
|
|
17
17
|
def lock_options(worker)
|
@@ -1,8 +1,8 @@
|
|
1
1
|
def set_sidekiq_lock(worker_class, payload)
|
2
2
|
options = worker_class.get_sidekiq_options['lock']
|
3
|
-
|
3
|
+
Sidekiq.lock_container.store(Sidekiq::Lock::RedisLock.new(options, payload))
|
4
4
|
end
|
5
5
|
|
6
6
|
def clear_sidekiq_lock
|
7
|
-
|
7
|
+
Sidekiq.lock_container.store(nil)
|
8
8
|
end
|
data/lib/sidekiq/lock/version.rb
CHANGED
data/lib/sidekiq/lock/worker.rb
CHANGED
@@ -0,0 +1,23 @@
|
|
1
|
+
require 'test_helper'
|
2
|
+
require 'open3'
|
3
|
+
|
4
|
+
module Sidekiq
|
5
|
+
module Lock
|
6
|
+
describe Container do
|
7
|
+
it 'stores and fetches given value under Thread.current' do
|
8
|
+
begin
|
9
|
+
container = Sidekiq::Lock::Container.new
|
10
|
+
thread_key = Sidekiq::Lock::Container::THREAD_KEY
|
11
|
+
|
12
|
+
Thread.current[thread_key] = 'value'
|
13
|
+
assert_equal 'value', container.fetch
|
14
|
+
|
15
|
+
container.store 'new-value'
|
16
|
+
assert_equal Thread.current[thread_key], 'new-value'
|
17
|
+
ensure
|
18
|
+
Thread.current[thread_key] = nil
|
19
|
+
end
|
20
|
+
end
|
21
|
+
end
|
22
|
+
end
|
23
|
+
end
|
data/test/lib/middleware_test.rb
CHANGED
@@ -6,36 +6,35 @@ module Sidekiq
|
|
6
6
|
before do
|
7
7
|
Sidekiq.redis = REDIS
|
8
8
|
Sidekiq.redis { |c| c.flushdb }
|
9
|
-
|
9
|
+
reset_lock_variable!
|
10
10
|
end
|
11
11
|
|
12
|
-
let(:handler){ Sidekiq::Lock::Middleware.new }
|
12
|
+
let(:handler) { Sidekiq::Lock::Middleware.new }
|
13
13
|
|
14
14
|
it 'sets lock variable with provided static lock options' do
|
15
|
-
handler.call(LockWorker.new, {'class' => LockWorker, 'args' => []}, 'default') do
|
15
|
+
handler.call(LockWorker.new, { 'class' => LockWorker, 'args' => [] }, 'default') do
|
16
16
|
true
|
17
17
|
end
|
18
18
|
|
19
|
-
assert_kind_of RedisLock,
|
19
|
+
assert_kind_of RedisLock, lock_container_variable
|
20
20
|
end
|
21
21
|
|
22
22
|
it 'sets lock variable with provided dynamic options' do
|
23
|
-
handler.call(DynamicLockWorker.new, {'class' => DynamicLockWorker, 'args' => [1234, 1000]}, 'default') do
|
23
|
+
handler.call(DynamicLockWorker.new, { 'class' => DynamicLockWorker, 'args' => [1234, 1000] }, 'default') do
|
24
24
|
true
|
25
25
|
end
|
26
26
|
|
27
|
-
assert_equal "lock:1234",
|
28
|
-
assert_equal 2000,
|
27
|
+
assert_equal "lock:1234", lock_container_variable.name
|
28
|
+
assert_equal 2000, lock_container_variable.timeout
|
29
29
|
end
|
30
30
|
|
31
31
|
it 'sets nothing for workers without lock options' do
|
32
|
-
handler.call(RegularWorker.new, {'class' => RegularWorker, 'args' => []}, 'default') do
|
32
|
+
handler.call(RegularWorker.new, { 'class' => RegularWorker, 'args' => [] }, 'default') do
|
33
33
|
true
|
34
34
|
end
|
35
35
|
|
36
|
-
assert_nil
|
36
|
+
assert_nil lock_container_variable
|
37
37
|
end
|
38
|
-
|
39
38
|
end
|
40
39
|
end
|
41
40
|
end
|
@@ -2,20 +2,23 @@ require "test_helper"
|
|
2
2
|
require "sidekiq/lock/testing/inline"
|
3
3
|
|
4
4
|
describe "inline test helper" do
|
5
|
-
after {
|
5
|
+
after { reset_lock_variable! }
|
6
6
|
|
7
7
|
it "has helper fuction for setting lock" do
|
8
|
-
Sidekiq::Lock::RedisLock
|
8
|
+
Sidekiq::Lock::RedisLock
|
9
|
+
.expects(:new)
|
10
|
+
.with({ timeout: 1, name: 'lock-worker' }, 'worker argument')
|
11
|
+
.returns('lock set')
|
12
|
+
|
9
13
|
set_sidekiq_lock(LockWorker, 'worker argument')
|
10
|
-
assert_equal 'lock set',
|
14
|
+
assert_equal 'lock set', lock_container_variable
|
11
15
|
end
|
12
16
|
|
13
17
|
it "has helper fuction for clearing lock" do
|
14
18
|
set_lock_variable! "test"
|
15
|
-
assert_equal "test",
|
19
|
+
assert_equal "test", lock_container_variable
|
16
20
|
|
17
21
|
clear_sidekiq_lock
|
18
|
-
assert_nil
|
22
|
+
assert_nil lock_container_variable
|
19
23
|
end
|
20
|
-
|
21
24
|
end
|
data/test/lib/worker_test.rb
CHANGED
@@ -3,26 +3,64 @@ require 'test_helper'
|
|
3
3
|
module Sidekiq
|
4
4
|
module Lock
|
5
5
|
describe Worker do
|
6
|
-
after {
|
6
|
+
# after { }
|
7
7
|
|
8
|
-
|
9
|
-
|
10
|
-
|
8
|
+
class CustomContainer
|
9
|
+
def initialize
|
10
|
+
@lock = nil
|
11
|
+
end
|
12
|
+
|
13
|
+
def fetch
|
14
|
+
@lock
|
15
|
+
end
|
16
|
+
|
17
|
+
def store(lock)
|
18
|
+
@lock = lock
|
19
|
+
end
|
11
20
|
end
|
12
21
|
|
22
|
+
# it 'sets lock method that points to thread variable' do
|
23
|
+
# set_lock_variable! "test"
|
24
|
+
# assert_equal "test", LockWorker.new.lock
|
25
|
+
# end
|
26
|
+
|
13
27
|
it 'allows method name configuration' do
|
14
|
-
|
28
|
+
begin
|
29
|
+
Sidekiq.lock_method = :custom_lock_name
|
15
30
|
|
16
|
-
|
17
|
-
|
18
|
-
|
19
|
-
|
31
|
+
class WorkerWithCustomLockName
|
32
|
+
include Sidekiq::Worker
|
33
|
+
include Sidekiq::Lock::Worker
|
34
|
+
end
|
35
|
+
|
36
|
+
set_lock_variable! "custom_name"
|
20
37
|
|
21
|
-
|
38
|
+
assert_equal "custom_name", WorkerWithCustomLockName.new.custom_lock_name
|
22
39
|
|
23
|
-
|
40
|
+
reset_lock_variable!
|
41
|
+
ensure
|
42
|
+
|
43
|
+
Sidekiq.lock_method = Sidekiq::Lock::METHOD_NAME
|
44
|
+
end
|
24
45
|
end
|
25
46
|
|
47
|
+
it 'allows container configuration' do
|
48
|
+
begin
|
49
|
+
container = CustomContainer.new
|
50
|
+
Sidekiq.lock_container = container
|
51
|
+
|
52
|
+
class WorkerWithCustomContainer
|
53
|
+
include Sidekiq::Worker
|
54
|
+
include Sidekiq::Lock::Worker
|
55
|
+
end
|
56
|
+
|
57
|
+
container.store "lock-variable"
|
58
|
+
|
59
|
+
assert_equal "lock-variable", WorkerWithCustomContainer.new.lock
|
60
|
+
ensure
|
61
|
+
Sidekiq.lock_container = Sidekiq::Lock::Container.new
|
62
|
+
end
|
63
|
+
end
|
26
64
|
end
|
27
65
|
end
|
28
66
|
end
|
data/test/test_helper.rb
CHANGED
@@ -20,10 +20,14 @@ def redis(command, *args)
|
|
20
20
|
end
|
21
21
|
end
|
22
22
|
|
23
|
-
def set_lock_variable!(value
|
24
|
-
|
23
|
+
def set_lock_variable!(value)
|
24
|
+
Sidekiq.lock_container.store(value)
|
25
25
|
end
|
26
26
|
|
27
|
-
def
|
28
|
-
|
27
|
+
def reset_lock_variable!
|
28
|
+
set_lock_variable!(nil)
|
29
|
+
end
|
30
|
+
|
31
|
+
def lock_container_variable
|
32
|
+
Sidekiq.lock_container.fetch
|
29
33
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: sidekiq-lock
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.4.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Rafal Wojsznis
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2018-
|
11
|
+
date: 2018-07-18 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: sidekiq
|
@@ -157,12 +157,14 @@ files:
|
|
157
157
|
- gemfiles/sidekiq_5.gemfile
|
158
158
|
- lib/sidekiq-lock.rb
|
159
159
|
- lib/sidekiq/lock.rb
|
160
|
+
- lib/sidekiq/lock/container.rb
|
160
161
|
- lib/sidekiq/lock/middleware.rb
|
161
162
|
- lib/sidekiq/lock/redis_lock.rb
|
162
163
|
- lib/sidekiq/lock/testing/inline.rb
|
163
164
|
- lib/sidekiq/lock/version.rb
|
164
165
|
- lib/sidekiq/lock/worker.rb
|
165
166
|
- sidekiq-lock.gemspec
|
167
|
+
- test/lib/container_test.rb
|
166
168
|
- test/lib/lock_test.rb
|
167
169
|
- test/lib/middleware_test.rb
|
168
170
|
- test/lib/redis_lock_test.rb
|
@@ -195,6 +197,7 @@ signing_key:
|
|
195
197
|
specification_version: 4
|
196
198
|
summary: Simple redis-based lock mechanism for your sidekiq workers
|
197
199
|
test_files:
|
200
|
+
- test/lib/container_test.rb
|
198
201
|
- test/lib/lock_test.rb
|
199
202
|
- test/lib/middleware_test.rb
|
200
203
|
- test/lib/redis_lock_test.rb
|