sidekiq-lock 0.3.1 → 0.4.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.gitignore +1 -0
- data/.travis.yml +4 -4
- data/CHANGELOG.md +4 -0
- data/README.md +35 -7
- data/lib/sidekiq/lock.rb +11 -1
- data/lib/sidekiq/lock/container.rb +15 -0
- data/lib/sidekiq/lock/middleware.rb +2 -2
- data/lib/sidekiq/lock/testing/inline.rb +2 -2
- data/lib/sidekiq/lock/version.rb +1 -1
- data/lib/sidekiq/lock/worker.rb +1 -1
- data/test/lib/container_test.rb +23 -0
- data/test/lib/middleware_test.rb +9 -10
- data/test/lib/testing/inline_test.rb +9 -6
- data/test/lib/worker_test.rb +49 -11
- data/test/test_helper.rb +8 -4
- metadata +5 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 7fb91d967eb073b2fb524357b936d473f3ab03450f3a1e7f45f9ae53895a210e
|
4
|
+
data.tar.gz: 10e5ac0b30ea212f675d9ce11ba63da4114b7ade9f1da89bbdcd209cfaa2f40e
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 3181d04fc0b7fbb7f9807851abf37d69b09333b38fef0c8388671a750a4087a5a185fd57dce6469095fb26ee42728b2e856f623b2643a3c15a9ed244197e6d0f
|
7
|
+
data.tar.gz: b1d1bd1c051b60188853436fd62ecec774cac2e303134e12dac203d166159b51e141f4a1a626ed952b6aa4591352f11e283bd17d371174bfdd25a6c3cac9745b
|
data/.gitignore
CHANGED
data/.travis.yml
CHANGED
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,7 @@
|
|
1
|
+
## 0.4.0 (July 18, 2018)
|
2
|
+
|
3
|
+
- make lock container configurable (non breaking change) - in case you would like to something else than `Thread.current` - now you easily can
|
4
|
+
|
1
5
|
## 0.3.1 (March 3, 2018)
|
2
6
|
|
3
7
|
- do not assume `ActiveSupport` is loaded / or old `Sidekiq` patches are present (add own symbolize keys logic)
|
data/README.md
CHANGED
@@ -3,14 +3,14 @@
|
|
3
3
|
[![Code Climate](https://codeclimate.com/github/emq/sidekiq-lock.png)](https://codeclimate.com/github/emq/sidekiq-lock)
|
4
4
|
[![Build Status](https://travis-ci.org/emq/sidekiq-lock.png?branch=master)](https://travis-ci.org/emq/sidekiq-lock)
|
5
5
|
[![Coverage Status](https://coveralls.io/repos/emq/sidekiq-lock/badge.png)](https://coveralls.io/r/emq/sidekiq-lock)
|
6
|
-
[![Dependency Status](https://gemnasium.com/emq/sidekiq-lock.png)](https://gemnasium.com/emq/sidekiq-lock)
|
7
6
|
[![Gem Version](https://badge.fury.io/rb/sidekiq-lock.png)](http://badge.fury.io/rb/sidekiq-lock)
|
8
7
|
|
9
8
|
**Note:** This is a _complete_ piece of software, it should work across all future sidekiq & ruby versions.
|
10
9
|
|
11
10
|
Redis-based simple locking mechanism for [sidekiq][2]. Uses [SET command][1] introduced in Redis 2.6.16.
|
12
11
|
|
13
|
-
It can be handy if you push a lot of jobs into the queue(s), but you don't want to execute specific jobs at the same
|
12
|
+
It can be handy if you push a lot of jobs into the queue(s), but you don't want to execute specific jobs at the same
|
13
|
+
time - it provides a `lock` method that you can use in whatever way you want.
|
14
14
|
|
15
15
|
## Installation
|
16
16
|
|
@@ -34,7 +34,8 @@ $ bundle
|
|
34
34
|
|
35
35
|
Sidekiq-lock is a middleware/module combination, let me go through my thought process here :).
|
36
36
|
|
37
|
-
In your worker class include `Sidekiq::Lock::Worker` module and provide `lock` attribute inside `sidekiq_options`,
|
37
|
+
In your worker class include `Sidekiq::Lock::Worker` module and provide `lock` attribute inside `sidekiq_options`,
|
38
|
+
for example:
|
38
39
|
|
39
40
|
``` ruby
|
40
41
|
class Worker
|
@@ -52,13 +53,17 @@ end
|
|
52
53
|
|
53
54
|
What will happen is:
|
54
55
|
|
55
|
-
- middleware will setup a `Sidekiq::Lock::RedisLock` object under `Thread.current[Sidekiq::Lock::THREAD_KEY]`
|
56
|
+
- middleware will setup a `Sidekiq::Lock::RedisLock` object under `Thread.current[Sidekiq::Lock::THREAD_KEY]`
|
57
|
+
(it should work in most use cases without any problems - but it's configurable, more below) - assuming you provided
|
58
|
+
`lock` options, otherwise it will do nothing, just execute your worker's code
|
56
59
|
|
57
|
-
- `Sidekiq::Lock::Worker` module provides a `lock` method that just simply points to that thread variable, just as
|
60
|
+
- `Sidekiq::Lock::Worker` module provides a `lock` method that just simply points to that thread variable, just as
|
61
|
+
a convenience
|
58
62
|
|
59
63
|
So now in your worker class you can call (whenever you need):
|
60
64
|
|
61
|
-
- `lock.acquire!` - will try to acquire the lock, if returns false on failure (that means some other process / thread
|
65
|
+
- `lock.acquire!` - will try to acquire the lock, if returns false on failure (that means some other process / thread
|
66
|
+
took the lock first)
|
62
67
|
- `lock.acquired?` - set to `true` when lock is successfully acquired
|
63
68
|
- `lock.release!` - deletes the lock (only if it's: acquired by current thread and not already expired)
|
64
69
|
|
@@ -115,9 +120,32 @@ Sidekiq.configure_server do |config|
|
|
115
120
|
end
|
116
121
|
```
|
117
122
|
|
123
|
+
### Customizing lock _container_
|
124
|
+
|
125
|
+
If you would like to change default behavior of storing lock instance in `Thread.current` for whatever reason you
|
126
|
+
can do that as well via server configuration:
|
127
|
+
|
128
|
+
``` ruby
|
129
|
+
# Any thread-safe class that implements .fetch and .store methods will do
|
130
|
+
class CustomStorage
|
131
|
+
def fetch
|
132
|
+
# returns stored lock instance
|
133
|
+
end
|
134
|
+
|
135
|
+
def store(lock_instance)
|
136
|
+
# store lock
|
137
|
+
end
|
138
|
+
end
|
139
|
+
|
140
|
+
Sidekiq.configure_server do |config|
|
141
|
+
config.lock_container = CustomStorage.new
|
142
|
+
end
|
143
|
+
```
|
144
|
+
|
118
145
|
### Inline testing
|
119
146
|
|
120
|
-
As you know middleware is not invoked when testing jobs inline, you can require in your test/spec helper file
|
147
|
+
As you know middleware is not invoked when testing jobs inline, you can require in your test/spec helper file
|
148
|
+
`sidekiq/lock/testing/inline` to include two methods that will help you setting / clearing up lock manually:
|
121
149
|
|
122
150
|
- `set_sidekiq_lock(worker_class, payload)` - note: payload should be an array of worker arguments
|
123
151
|
- `clear_sidekiq_lock`
|
data/lib/sidekiq/lock.rb
CHANGED
@@ -1,11 +1,20 @@
|
|
1
|
+
require 'sidekiq/lock/container'
|
1
2
|
require 'sidekiq/lock/middleware'
|
2
3
|
require 'sidekiq/lock/redis_lock'
|
3
4
|
require 'sidekiq/lock/version'
|
4
5
|
require 'sidekiq/lock/worker'
|
5
6
|
|
6
7
|
module Sidekiq
|
8
|
+
def self.lock_container
|
9
|
+
@lock_container ||= Lock::Container.new
|
10
|
+
end
|
11
|
+
|
7
12
|
def self.lock_method
|
8
|
-
@lock_method ||=
|
13
|
+
@lock_method ||= Lock::METHOD_NAME
|
14
|
+
end
|
15
|
+
|
16
|
+
def self.lock_container=(container)
|
17
|
+
@lock_container = container
|
9
18
|
end
|
10
19
|
|
11
20
|
def self.lock_method=(method)
|
@@ -14,6 +23,7 @@ module Sidekiq
|
|
14
23
|
|
15
24
|
module Lock
|
16
25
|
THREAD_KEY = :sidekiq_lock
|
26
|
+
METHOD_NAME = :lock
|
17
27
|
end
|
18
28
|
end
|
19
29
|
|
@@ -1,7 +1,7 @@
|
|
1
1
|
module Sidekiq
|
2
2
|
module Lock
|
3
3
|
class Middleware
|
4
|
-
def call(worker, msg,
|
4
|
+
def call(worker, msg, _queue)
|
5
5
|
options = lock_options(worker)
|
6
6
|
setup_lock(options, msg['args']) unless options.nil?
|
7
7
|
|
@@ -11,7 +11,7 @@ module Sidekiq
|
|
11
11
|
private
|
12
12
|
|
13
13
|
def setup_lock(options, payload)
|
14
|
-
|
14
|
+
Sidekiq.lock_container.store(RedisLock.new(options, payload))
|
15
15
|
end
|
16
16
|
|
17
17
|
def lock_options(worker)
|
@@ -1,8 +1,8 @@
|
|
1
1
|
def set_sidekiq_lock(worker_class, payload)
|
2
2
|
options = worker_class.get_sidekiq_options['lock']
|
3
|
-
|
3
|
+
Sidekiq.lock_container.store(Sidekiq::Lock::RedisLock.new(options, payload))
|
4
4
|
end
|
5
5
|
|
6
6
|
def clear_sidekiq_lock
|
7
|
-
|
7
|
+
Sidekiq.lock_container.store(nil)
|
8
8
|
end
|
data/lib/sidekiq/lock/version.rb
CHANGED
data/lib/sidekiq/lock/worker.rb
CHANGED
@@ -0,0 +1,23 @@
|
|
1
|
+
require 'test_helper'
|
2
|
+
require 'open3'
|
3
|
+
|
4
|
+
module Sidekiq
|
5
|
+
module Lock
|
6
|
+
describe Container do
|
7
|
+
it 'stores and fetches given value under Thread.current' do
|
8
|
+
begin
|
9
|
+
container = Sidekiq::Lock::Container.new
|
10
|
+
thread_key = Sidekiq::Lock::Container::THREAD_KEY
|
11
|
+
|
12
|
+
Thread.current[thread_key] = 'value'
|
13
|
+
assert_equal 'value', container.fetch
|
14
|
+
|
15
|
+
container.store 'new-value'
|
16
|
+
assert_equal Thread.current[thread_key], 'new-value'
|
17
|
+
ensure
|
18
|
+
Thread.current[thread_key] = nil
|
19
|
+
end
|
20
|
+
end
|
21
|
+
end
|
22
|
+
end
|
23
|
+
end
|
data/test/lib/middleware_test.rb
CHANGED
@@ -6,36 +6,35 @@ module Sidekiq
|
|
6
6
|
before do
|
7
7
|
Sidekiq.redis = REDIS
|
8
8
|
Sidekiq.redis { |c| c.flushdb }
|
9
|
-
|
9
|
+
reset_lock_variable!
|
10
10
|
end
|
11
11
|
|
12
|
-
let(:handler){ Sidekiq::Lock::Middleware.new }
|
12
|
+
let(:handler) { Sidekiq::Lock::Middleware.new }
|
13
13
|
|
14
14
|
it 'sets lock variable with provided static lock options' do
|
15
|
-
handler.call(LockWorker.new, {'class' => LockWorker, 'args' => []}, 'default') do
|
15
|
+
handler.call(LockWorker.new, { 'class' => LockWorker, 'args' => [] }, 'default') do
|
16
16
|
true
|
17
17
|
end
|
18
18
|
|
19
|
-
assert_kind_of RedisLock,
|
19
|
+
assert_kind_of RedisLock, lock_container_variable
|
20
20
|
end
|
21
21
|
|
22
22
|
it 'sets lock variable with provided dynamic options' do
|
23
|
-
handler.call(DynamicLockWorker.new, {'class' => DynamicLockWorker, 'args' => [1234, 1000]}, 'default') do
|
23
|
+
handler.call(DynamicLockWorker.new, { 'class' => DynamicLockWorker, 'args' => [1234, 1000] }, 'default') do
|
24
24
|
true
|
25
25
|
end
|
26
26
|
|
27
|
-
assert_equal "lock:1234",
|
28
|
-
assert_equal 2000,
|
27
|
+
assert_equal "lock:1234", lock_container_variable.name
|
28
|
+
assert_equal 2000, lock_container_variable.timeout
|
29
29
|
end
|
30
30
|
|
31
31
|
it 'sets nothing for workers without lock options' do
|
32
|
-
handler.call(RegularWorker.new, {'class' => RegularWorker, 'args' => []}, 'default') do
|
32
|
+
handler.call(RegularWorker.new, { 'class' => RegularWorker, 'args' => [] }, 'default') do
|
33
33
|
true
|
34
34
|
end
|
35
35
|
|
36
|
-
assert_nil
|
36
|
+
assert_nil lock_container_variable
|
37
37
|
end
|
38
|
-
|
39
38
|
end
|
40
39
|
end
|
41
40
|
end
|
@@ -2,20 +2,23 @@ require "test_helper"
|
|
2
2
|
require "sidekiq/lock/testing/inline"
|
3
3
|
|
4
4
|
describe "inline test helper" do
|
5
|
-
after {
|
5
|
+
after { reset_lock_variable! }
|
6
6
|
|
7
7
|
it "has helper fuction for setting lock" do
|
8
|
-
Sidekiq::Lock::RedisLock
|
8
|
+
Sidekiq::Lock::RedisLock
|
9
|
+
.expects(:new)
|
10
|
+
.with({ timeout: 1, name: 'lock-worker' }, 'worker argument')
|
11
|
+
.returns('lock set')
|
12
|
+
|
9
13
|
set_sidekiq_lock(LockWorker, 'worker argument')
|
10
|
-
assert_equal 'lock set',
|
14
|
+
assert_equal 'lock set', lock_container_variable
|
11
15
|
end
|
12
16
|
|
13
17
|
it "has helper fuction for clearing lock" do
|
14
18
|
set_lock_variable! "test"
|
15
|
-
assert_equal "test",
|
19
|
+
assert_equal "test", lock_container_variable
|
16
20
|
|
17
21
|
clear_sidekiq_lock
|
18
|
-
assert_nil
|
22
|
+
assert_nil lock_container_variable
|
19
23
|
end
|
20
|
-
|
21
24
|
end
|
data/test/lib/worker_test.rb
CHANGED
@@ -3,26 +3,64 @@ require 'test_helper'
|
|
3
3
|
module Sidekiq
|
4
4
|
module Lock
|
5
5
|
describe Worker do
|
6
|
-
after {
|
6
|
+
# after { }
|
7
7
|
|
8
|
-
|
9
|
-
|
10
|
-
|
8
|
+
class CustomContainer
|
9
|
+
def initialize
|
10
|
+
@lock = nil
|
11
|
+
end
|
12
|
+
|
13
|
+
def fetch
|
14
|
+
@lock
|
15
|
+
end
|
16
|
+
|
17
|
+
def store(lock)
|
18
|
+
@lock = lock
|
19
|
+
end
|
11
20
|
end
|
12
21
|
|
22
|
+
# it 'sets lock method that points to thread variable' do
|
23
|
+
# set_lock_variable! "test"
|
24
|
+
# assert_equal "test", LockWorker.new.lock
|
25
|
+
# end
|
26
|
+
|
13
27
|
it 'allows method name configuration' do
|
14
|
-
|
28
|
+
begin
|
29
|
+
Sidekiq.lock_method = :custom_lock_name
|
15
30
|
|
16
|
-
|
17
|
-
|
18
|
-
|
19
|
-
|
31
|
+
class WorkerWithCustomLockName
|
32
|
+
include Sidekiq::Worker
|
33
|
+
include Sidekiq::Lock::Worker
|
34
|
+
end
|
35
|
+
|
36
|
+
set_lock_variable! "custom_name"
|
20
37
|
|
21
|
-
|
38
|
+
assert_equal "custom_name", WorkerWithCustomLockName.new.custom_lock_name
|
22
39
|
|
23
|
-
|
40
|
+
reset_lock_variable!
|
41
|
+
ensure
|
42
|
+
|
43
|
+
Sidekiq.lock_method = Sidekiq::Lock::METHOD_NAME
|
44
|
+
end
|
24
45
|
end
|
25
46
|
|
47
|
+
it 'allows container configuration' do
|
48
|
+
begin
|
49
|
+
container = CustomContainer.new
|
50
|
+
Sidekiq.lock_container = container
|
51
|
+
|
52
|
+
class WorkerWithCustomContainer
|
53
|
+
include Sidekiq::Worker
|
54
|
+
include Sidekiq::Lock::Worker
|
55
|
+
end
|
56
|
+
|
57
|
+
container.store "lock-variable"
|
58
|
+
|
59
|
+
assert_equal "lock-variable", WorkerWithCustomContainer.new.lock
|
60
|
+
ensure
|
61
|
+
Sidekiq.lock_container = Sidekiq::Lock::Container.new
|
62
|
+
end
|
63
|
+
end
|
26
64
|
end
|
27
65
|
end
|
28
66
|
end
|
data/test/test_helper.rb
CHANGED
@@ -20,10 +20,14 @@ def redis(command, *args)
|
|
20
20
|
end
|
21
21
|
end
|
22
22
|
|
23
|
-
def set_lock_variable!(value
|
24
|
-
|
23
|
+
def set_lock_variable!(value)
|
24
|
+
Sidekiq.lock_container.store(value)
|
25
25
|
end
|
26
26
|
|
27
|
-
def
|
28
|
-
|
27
|
+
def reset_lock_variable!
|
28
|
+
set_lock_variable!(nil)
|
29
|
+
end
|
30
|
+
|
31
|
+
def lock_container_variable
|
32
|
+
Sidekiq.lock_container.fetch
|
29
33
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: sidekiq-lock
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.4.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Rafal Wojsznis
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2018-
|
11
|
+
date: 2018-07-18 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: sidekiq
|
@@ -157,12 +157,14 @@ files:
|
|
157
157
|
- gemfiles/sidekiq_5.gemfile
|
158
158
|
- lib/sidekiq-lock.rb
|
159
159
|
- lib/sidekiq/lock.rb
|
160
|
+
- lib/sidekiq/lock/container.rb
|
160
161
|
- lib/sidekiq/lock/middleware.rb
|
161
162
|
- lib/sidekiq/lock/redis_lock.rb
|
162
163
|
- lib/sidekiq/lock/testing/inline.rb
|
163
164
|
- lib/sidekiq/lock/version.rb
|
164
165
|
- lib/sidekiq/lock/worker.rb
|
165
166
|
- sidekiq-lock.gemspec
|
167
|
+
- test/lib/container_test.rb
|
166
168
|
- test/lib/lock_test.rb
|
167
169
|
- test/lib/middleware_test.rb
|
168
170
|
- test/lib/redis_lock_test.rb
|
@@ -195,6 +197,7 @@ signing_key:
|
|
195
197
|
specification_version: 4
|
196
198
|
summary: Simple redis-based lock mechanism for your sidekiq workers
|
197
199
|
test_files:
|
200
|
+
- test/lib/container_test.rb
|
198
201
|
- test/lib/lock_test.rb
|
199
202
|
- test/lib/middleware_test.rb
|
200
203
|
- test/lib/redis_lock_test.rb
|