activejob-locking 0.4.0 → 0.5.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/HISTORY.md +21 -16
- data/README.md +274 -247
- data/lib/activejob-locking.rb +24 -23
- data/lib/activejob/locking/adapters/base.rb +31 -31
- data/lib/activejob/locking/adapters/memory.rb +57 -57
- data/lib/activejob/locking/adapters/redis-semaphore.rb +26 -26
- data/lib/activejob/locking/adapters/redlock.rb +26 -26
- data/lib/activejob/locking/adapters/suo-redis.rb +25 -25
- data/lib/activejob/locking/base.rb +2 -2
- data/lib/activejob/locking/options.rb +16 -0
- data/lib/activejob/locking/serialized.rb +1 -1
- data/test/jobs/fail_job.rb +14 -14
- data/test/jobs/serial_job.rb +14 -14
- data/test/jobs/unique_job.rb +14 -14
- data/test/test_suite.rb +12 -12
- metadata +3 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: e574b3a41612fff773ca03309e55b6a2a4bb2db0
|
4
|
+
data.tar.gz: 3ef9de73fa5ac528c317fd8218be929ce3e975f8
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 28f284c0b260bfdc69387d901f5fb217d1afb512127b05a4a34a8399e0cf36718c33f0c60e857686572f0c4d340a9c83c8889ef3a50510a06114fe961e27b7df
|
7
|
+
data.tar.gz: 65c953f75b1795f7be08cd0d262b01584a6f850feb16ac4a7f2432fcedbd964e07820d7ebaa11e3a6c4a043cefbb2bdef315a2529b90069975b0778f13c23ab1
|
data/HISTORY.md
CHANGED
@@ -1,16 +1,21 @@
|
|
1
|
-
## 0.
|
2
|
-
|
3
|
-
-
|
4
|
-
|
5
|
-
|
6
|
-
|
7
|
-
|
8
|
-
|
9
|
-
|
10
|
-
|
11
|
-
|
12
|
-
-
|
13
|
-
|
14
|
-
## 0.
|
15
|
-
|
16
|
-
-
|
1
|
+
## 0.5.0 (2017-10-03)
|
2
|
+
|
3
|
+
- Add enqueue_time parameter
|
4
|
+
- Fix bug in setting enqueue time
|
5
|
+
|
6
|
+
## 0.4.0 (2017-06-21)
|
7
|
+
|
8
|
+
- Cleanup handling of hosts
|
9
|
+
|
10
|
+
## 0.3.0 (2017-06-21)
|
11
|
+
|
12
|
+
- Change lock_key signature to match perform signature
|
13
|
+
|
14
|
+
## 0.2.0 (2017-06-20)
|
15
|
+
|
16
|
+
- Bug fixes
|
17
|
+
- Improved tests
|
18
|
+
|
19
|
+
## 0.1.0 (2017-01-16)
|
20
|
+
|
21
|
+
- Initial release
|
data/README.md
CHANGED
@@ -1,247 +1,274 @@
|
|
1
|
-
ActiveJob Locking
|
2
|
-
===================
|
3
|
-
|
4
|
-
[![Build Status](https://secure.travis-ci.org/lantins/activejob-locking.png?branch=master)](http://travis-ci.org/cfis/activejob-locking)
|
5
|
-
[![Gem Version](https://badge.fury.io/rb/activejob-locking.png)](http://badge.fury.io/rb/activejob-locking)
|
6
|
-
|
7
|
-
activejob-locking lets you control how ActiveJobs are enqueued and performed:
|
8
|
-
|
9
|
-
* Allow only one job to be enqueued at a time - thus a "unique" job
|
10
|
-
* Allow only one job to be performed at a time - thus a "serialized" job
|
11
|
-
|
12
|
-
There are many other similar gems including [resque-lock-timeout](https://github.com/lantins/resque-lock-timeout),
|
13
|
-
[activejob-traffic-control](https://github.com/nickelser/activejob-traffic_control), [activejob-lock](https://github.com/idolweb/activejob-lock),
|
14
|
-
[activejob-locks](https://github.com/erickrause/activejob-locks). What is different about this gem is that it
|
15
|
-
is agnostic on the locking mechanism. In the same way that ActiveJob works with many apapters, ActiveJob Locking
|
16
|
-
works with a variety of locking gems.
|
17
|
-
|
18
|
-
Installation
|
19
|
-
------------
|
20
|
-
|
21
|
-
Add this line to your application's Gemfile:
|
22
|
-
|
23
|
-
```ruby
|
24
|
-
gem 'activejob-locking'
|
25
|
-
```
|
26
|
-
|
27
|
-
Unique Jobs
|
28
|
-
------------
|
29
|
-
Sometime you only want to enqueue one instance of a job. No other similar job should be enqueued until the first one
|
30
|
-
is completed.
|
31
|
-
|
32
|
-
```ruby
|
33
|
-
class UniqueJob < ActiveJob::Base
|
34
|
-
include ActiveJob::Locking::Unique
|
35
|
-
|
36
|
-
# Make sure the lock_key is always the same
|
37
|
-
def lock_key(object)
|
38
|
-
self.class.name
|
39
|
-
end
|
40
|
-
|
41
|
-
def perform(object)
|
42
|
-
# do some work
|
43
|
-
end
|
44
|
-
end
|
45
|
-
```
|
46
|
-
Only one instance of this job will ever be enqueued. If an additional job is enqueued, it will either be dropped and
|
47
|
-
never be enqueued or it will wait to the first job is performed. That is controlled by the job
|
48
|
-
[options](##options) described below.
|
49
|
-
|
50
|
-
|
51
|
-
Serialized Jobs
|
52
|
-
------------
|
53
|
-
Sometime you only want to perform one instance of a job at a time. No other similar job should be performed until the first one
|
54
|
-
is completed.
|
55
|
-
|
56
|
-
```ruby
|
57
|
-
class SerializedJob < ActiveJob::Base
|
58
|
-
include ActiveJob::Locking::Serialized
|
59
|
-
|
60
|
-
# Make sure the lock_key is always the same
|
61
|
-
def lock_key
|
62
|
-
self.class.name
|
63
|
-
end
|
64
|
-
|
65
|
-
def perform
|
66
|
-
# do some work
|
67
|
-
end
|
68
|
-
end
|
69
|
-
```
|
70
|
-
Only one instance of this job will ever be performed. If an additional job is enqueued, it will wait in its que until
|
71
|
-
to the first job is performed.
|
72
|
-
|
73
|
-
Locking
|
74
|
-
------------
|
75
|
-
Locks are used to control how jobs are enqueued and performed. The idea is that locks are stored in a distributed
|
76
|
-
system such as [Redis](https://redis.io/) or [Memcached](https://memcached.org/) so they can be used by
|
77
|
-
multiple servers to coordinate the enqueueing and performing of jobs.
|
78
|
-
|
79
|
-
The ActiveJob Locking gem does not include a locking implementation. Instead it provides adapters for
|
80
|
-
distributed locking gems.
|
81
|
-
|
82
|
-
Currently three gems are supported:
|
83
|
-
|
84
|
-
* [redis-semaphore](https://github.com/dv/redis-semaphore)
|
85
|
-
|
86
|
-
* [suo](https://github.com/nickelser/suo)
|
87
|
-
|
88
|
-
* [redlock-rb](https://github.com/leandromoreira/redlock-rb)
|
89
|
-
|
90
|
-
If you would like to have an additional locking mechanism supported, please feel free to send in a pull request.
|
91
|
-
|
92
|
-
Please see the [options](##options) section below on how to specify a locking adapter.
|
93
|
-
|
94
|
-
|
95
|
-
Lock Key
|
96
|
-
---------
|
97
|
-
|
98
|
-
Notice that the code samples above include a `lock_key` method. The return value of this method is used by the
|
99
|
-
gem to create locks behind the scenes. Thus it holds the key (pun intended) to controlling how jobs are enqueued
|
100
|
-
and performed.
|
101
|
-
|
102
|
-
By default the key is defined as:
|
103
|
-
|
104
|
-
```ruby
|
105
|
-
def lock_key(*args)
|
106
|
-
[self.class.name, serialize_arguments(self.arguments)].join('/')
|
107
|
-
end
|
108
|
-
```
|
109
|
-
Thus it has the format `<job class name>/<serialized_job_arguments>`
|
110
|
-
|
111
|
-
The args passed to the lock key method are the same that are passed to the job's perform method.
|
112
|
-
|
113
|
-
To use this gem, you will want to override this method per job.
|
114
|
-
|
115
|
-
### Examples
|
116
|
-
|
117
|
-
Allow only one job per queue to be enqueued or performed:
|
118
|
-
|
119
|
-
```ruby
|
120
|
-
def lock_key(*args)
|
121
|
-
self.queue
|
122
|
-
end
|
123
|
-
```
|
124
|
-
|
125
|
-
Allow only one instance of a job class to be enqueued of performed:
|
126
|
-
|
127
|
-
```ruby
|
128
|
-
def lock_key(*args)
|
129
|
-
self.class.name
|
130
|
-
end
|
131
|
-
```
|
132
|
-
|
133
|
-
Options
|
134
|
-
-------
|
135
|
-
The locking behavior can be dramatically changed by tweaking various options. There is a global set of options
|
136
|
-
available at:
|
137
|
-
|
138
|
-
```ruby
|
139
|
-
ActiveJob::Locking.options
|
140
|
-
```
|
141
|
-
This should be updated using a Rails initializer. Each job class can override individual options as it sees fit.
|
142
|
-
|
143
|
-
### Adapter
|
144
|
-
|
145
|
-
Use the adapter option to specify which locking gem to use.
|
146
|
-
|
147
|
-
Globally update:
|
148
|
-
|
149
|
-
```ruby
|
150
|
-
ActiveJob::Locking.options.adapter = ActiveJob::Locking::Adapters::SuoRedis
|
151
|
-
```
|
152
|
-
Locally update:
|
153
|
-
|
154
|
-
```ruby
|
155
|
-
class ExampleJob < ActiveJob::Base
|
156
|
-
include ActiveJob::Locking::Serialized
|
157
|
-
|
158
|
-
self.adapter = ActiveJob::Locking::Adapters::SuoRedis
|
159
|
-
end
|
160
|
-
```
|
161
|
-
|
162
|
-
### Hosts
|
163
|
-
|
164
|
-
An array of hosts for the distributed system. This format is dependent on the locking gem, but generally is a url or an existing Memcache or Redis
|
165
|
-
connection. Please refer to the appropriate locking gem's documentation documentation.
|
166
|
-
|
167
|
-
Globally update:
|
168
|
-
|
169
|
-
```ruby
|
170
|
-
ActiveJob::Locking.options.hosts = ['localhost']
|
171
|
-
```
|
172
|
-
Locally update:
|
173
|
-
|
174
|
-
```ruby
|
175
|
-
class ExampleJob < ActiveJob::Base
|
176
|
-
include ActiveJob::Locking::Serialized
|
177
|
-
|
178
|
-
self.hosts = ['localhost']
|
179
|
-
end
|
180
|
-
```
|
181
|
-
|
182
|
-
###
|
183
|
-
|
184
|
-
The is the
|
185
|
-
|
186
|
-
|
187
|
-
|
188
|
-
|
189
|
-
|
190
|
-
|
191
|
-
|
192
|
-
|
193
|
-
|
194
|
-
```
|
195
|
-
|
196
|
-
|
197
|
-
|
198
|
-
|
199
|
-
|
200
|
-
|
201
|
-
|
202
|
-
|
203
|
-
|
204
|
-
|
205
|
-
|
206
|
-
|
207
|
-
|
208
|
-
|
209
|
-
|
210
|
-
|
211
|
-
|
212
|
-
The value is specified in seconds and defaults to
|
213
|
-
|
214
|
-
|
215
|
-
|
216
|
-
|
217
|
-
|
218
|
-
|
219
|
-
|
220
|
-
|
221
|
-
|
222
|
-
|
223
|
-
|
224
|
-
|
225
|
-
|
226
|
-
|
227
|
-
|
228
|
-
|
229
|
-
|
230
|
-
|
231
|
-
|
232
|
-
|
233
|
-
|
234
|
-
|
235
|
-
|
236
|
-
|
237
|
-
|
238
|
-
|
239
|
-
|
240
|
-
|
241
|
-
|
242
|
-
|
243
|
-
|
244
|
-
|
245
|
-
|
246
|
-
|
247
|
-
|
1
|
+
ActiveJob Locking
|
2
|
+
===================
|
3
|
+
|
4
|
+
[![Build Status](https://secure.travis-ci.org/lantins/activejob-locking.png?branch=master)](http://travis-ci.org/cfis/activejob-locking)
|
5
|
+
[![Gem Version](https://badge.fury.io/rb/activejob-locking.png)](http://badge.fury.io/rb/activejob-locking)
|
6
|
+
|
7
|
+
activejob-locking lets you control how ActiveJobs are enqueued and performed:
|
8
|
+
|
9
|
+
* Allow only one job to be enqueued at a time - thus a "unique" job
|
10
|
+
* Allow only one job to be performed at a time - thus a "serialized" job
|
11
|
+
|
12
|
+
There are many other similar gems including [resque-lock-timeout](https://github.com/lantins/resque-lock-timeout),
|
13
|
+
[activejob-traffic-control](https://github.com/nickelser/activejob-traffic_control), [activejob-lock](https://github.com/idolweb/activejob-lock),
|
14
|
+
[activejob-locks](https://github.com/erickrause/activejob-locks). What is different about this gem is that it
|
15
|
+
is agnostic on the locking mechanism. In the same way that ActiveJob works with many apapters, ActiveJob Locking
|
16
|
+
works with a variety of locking gems.
|
17
|
+
|
18
|
+
Installation
|
19
|
+
------------
|
20
|
+
|
21
|
+
Add this line to your application's Gemfile:
|
22
|
+
|
23
|
+
```ruby
|
24
|
+
gem 'activejob-locking'
|
25
|
+
```
|
26
|
+
|
27
|
+
Unique Jobs
|
28
|
+
------------
|
29
|
+
Sometime you only want to enqueue one instance of a job. No other similar job should be enqueued until the first one
|
30
|
+
is completed.
|
31
|
+
|
32
|
+
```ruby
|
33
|
+
class UniqueJob < ActiveJob::Base
|
34
|
+
include ActiveJob::Locking::Unique
|
35
|
+
|
36
|
+
# Make sure the lock_key is always the same
|
37
|
+
def lock_key(object)
|
38
|
+
self.class.name
|
39
|
+
end
|
40
|
+
|
41
|
+
def perform(object)
|
42
|
+
# do some work
|
43
|
+
end
|
44
|
+
end
|
45
|
+
```
|
46
|
+
Only one instance of this job will ever be enqueued. If an additional job is enqueued, it will either be dropped and
|
47
|
+
never be enqueued or it will wait to the first job is performed. That is controlled by the job
|
48
|
+
[options](##options) described below.
|
49
|
+
|
50
|
+
|
51
|
+
Serialized Jobs
|
52
|
+
------------
|
53
|
+
Sometime you only want to perform one instance of a job at a time. No other similar job should be performed until the first one
|
54
|
+
is completed.
|
55
|
+
|
56
|
+
```ruby
|
57
|
+
class SerializedJob < ActiveJob::Base
|
58
|
+
include ActiveJob::Locking::Serialized
|
59
|
+
|
60
|
+
# Make sure the lock_key is always the same
|
61
|
+
def lock_key
|
62
|
+
self.class.name
|
63
|
+
end
|
64
|
+
|
65
|
+
def perform
|
66
|
+
# do some work
|
67
|
+
end
|
68
|
+
end
|
69
|
+
```
|
70
|
+
Only one instance of this job will ever be performed. If an additional job is enqueued, it will wait in its que until
|
71
|
+
to the first job is performed.
|
72
|
+
|
73
|
+
Locking
|
74
|
+
------------
|
75
|
+
Locks are used to control how jobs are enqueued and performed. The idea is that locks are stored in a distributed
|
76
|
+
system such as [Redis](https://redis.io/) or [Memcached](https://memcached.org/) so they can be used by
|
77
|
+
multiple servers to coordinate the enqueueing and performing of jobs.
|
78
|
+
|
79
|
+
The ActiveJob Locking gem does not include a locking implementation. Instead it provides adapters for
|
80
|
+
distributed locking gems.
|
81
|
+
|
82
|
+
Currently three gems are supported:
|
83
|
+
|
84
|
+
* [redis-semaphore](https://github.com/dv/redis-semaphore)
|
85
|
+
|
86
|
+
* [suo](https://github.com/nickelser/suo)
|
87
|
+
|
88
|
+
* [redlock-rb](https://github.com/leandromoreira/redlock-rb)
|
89
|
+
|
90
|
+
If you would like to have an additional locking mechanism supported, please feel free to send in a pull request.
|
91
|
+
|
92
|
+
Please see the [options](##options) section below on how to specify a locking adapter.
|
93
|
+
|
94
|
+
|
95
|
+
Lock Key
|
96
|
+
---------
|
97
|
+
|
98
|
+
Notice that the code samples above include a `lock_key` method. The return value of this method is used by the
|
99
|
+
gem to create locks behind the scenes. Thus it holds the key (pun intended) to controlling how jobs are enqueued
|
100
|
+
and performed.
|
101
|
+
|
102
|
+
By default the key is defined as:
|
103
|
+
|
104
|
+
```ruby
|
105
|
+
def lock_key(*args)
|
106
|
+
[self.class.name, serialize_arguments(self.arguments)].join('/')
|
107
|
+
end
|
108
|
+
```
|
109
|
+
Thus it has the format `<job class name>/<serialized_job_arguments>`
|
110
|
+
|
111
|
+
The args passed to the lock key method are the same that are passed to the job's perform method.
|
112
|
+
|
113
|
+
To use this gem, you will want to override this method per job.
|
114
|
+
|
115
|
+
### Examples
|
116
|
+
|
117
|
+
Allow only one job per queue to be enqueued or performed:
|
118
|
+
|
119
|
+
```ruby
|
120
|
+
def lock_key(*args)
|
121
|
+
self.queue
|
122
|
+
end
|
123
|
+
```
|
124
|
+
|
125
|
+
Allow only one instance of a job class to be enqueued of performed:
|
126
|
+
|
127
|
+
```ruby
|
128
|
+
def lock_key(*args)
|
129
|
+
self.class.name
|
130
|
+
end
|
131
|
+
```
|
132
|
+
|
133
|
+
Options
|
134
|
+
-------
|
135
|
+
The locking behavior can be dramatically changed by tweaking various options. There is a global set of options
|
136
|
+
available at:
|
137
|
+
|
138
|
+
```ruby
|
139
|
+
ActiveJob::Locking.options
|
140
|
+
```
|
141
|
+
This should be updated using a Rails initializer. Each job class can override individual options as it sees fit.
|
142
|
+
|
143
|
+
### Adapter
|
144
|
+
|
145
|
+
Use the adapter option to specify which locking gem to use.
|
146
|
+
|
147
|
+
Globally update:
|
148
|
+
|
149
|
+
```ruby
|
150
|
+
ActiveJob::Locking.options.adapter = ActiveJob::Locking::Adapters::SuoRedis
|
151
|
+
```
|
152
|
+
Locally update:
|
153
|
+
|
154
|
+
```ruby
|
155
|
+
class ExampleJob < ActiveJob::Base
|
156
|
+
include ActiveJob::Locking::Serialized
|
157
|
+
|
158
|
+
self.adapter = ActiveJob::Locking::Adapters::SuoRedis
|
159
|
+
end
|
160
|
+
```
|
161
|
+
|
162
|
+
### Hosts
|
163
|
+
|
164
|
+
An array of hosts for the distributed system. This format is dependent on the locking gem, but generally is a url or an existing Memcache or Redis
|
165
|
+
connection. Please refer to the appropriate locking gem's documentation documentation.
|
166
|
+
|
167
|
+
Globally update:
|
168
|
+
|
169
|
+
```ruby
|
170
|
+
ActiveJob::Locking.options.hosts = ['localhost']
|
171
|
+
```
|
172
|
+
Locally update:
|
173
|
+
|
174
|
+
```ruby
|
175
|
+
class ExampleJob < ActiveJob::Base
|
176
|
+
include ActiveJob::Locking::Serialized
|
177
|
+
|
178
|
+
self.hosts = ['localhost']
|
179
|
+
end
|
180
|
+
```
|
181
|
+
|
182
|
+
### lock_time
|
183
|
+
|
184
|
+
The is the time to live for any acquired locks. For most locking gems this is mapped to their concept of "stale" locks.
|
185
|
+
That means that if an attempt is made to access the lock after it is expired, it will be considered unlocked. That is in
|
186
|
+
contrast to aggressively removing locks for running jobs even if no other job has requested them.
|
187
|
+
|
188
|
+
The value is specified in seconds and defaults to 100.
|
189
|
+
|
190
|
+
Globally update:
|
191
|
+
|
192
|
+
```ruby
|
193
|
+
ActiveJob::Locking.options.lock_time = 100
|
194
|
+
```
|
195
|
+
Locally update:
|
196
|
+
|
197
|
+
```ruby
|
198
|
+
class ExampleJob < ActiveJob::Base
|
199
|
+
include ActiveJob::Locking::Serialized
|
200
|
+
|
201
|
+
self.lock_time = 100
|
202
|
+
end
|
203
|
+
```
|
204
|
+
|
205
|
+
You almost surely want the lock_time to be greater than the time it takes to execute the job. Otherwise, the lock will expire
|
206
|
+
and extra jobs will start to run. When the job finishes, or fails, the lock will be released. However, remember that the job
|
207
|
+
could be terminated by the operating system or a monitoring system (such as monit). In that case, the lock won't be released
|
208
|
+
and will remain in force until its lock_time expires.
|
209
|
+
|
210
|
+
### lock_acquire_time
|
211
|
+
|
212
|
+
The is the timeout for acquiring a lock. The value is specified in seconds and defaults to 1. It must
|
213
|
+
be greater than zero and cannot be nil.
|
214
|
+
|
215
|
+
Globally update:
|
216
|
+
|
217
|
+
```ruby
|
218
|
+
ActiveJob::Locking.options.lock_acquire_time = 1
|
219
|
+
```
|
220
|
+
Locally update:
|
221
|
+
|
222
|
+
```ruby
|
223
|
+
class ExampleJob < ActiveJob::Base
|
224
|
+
include ActiveJob::Locking::Unique
|
225
|
+
|
226
|
+
self.lock_acquire_time = 1
|
227
|
+
end
|
228
|
+
```
|
229
|
+
|
230
|
+
Remember that most locking gems block the current thread when trying to acquire a lock. Therefore you likely want
|
231
|
+
lock_acquire_time to be low. However, the lower it is the more likely that unique jobs that are enqueued will
|
232
|
+
expire and be dropped.
|
233
|
+
|
234
|
+
### enqueue_time
|
235
|
+
|
236
|
+
The is the time to re-enqueue a job if the lock_time has expired. Thus this value is only relevant for
|
237
|
+
serialized jobs since unique jobs will be dropped instead of enqueded.
|
238
|
+
|
239
|
+
The value is specified in seconds and defaults to 100.
|
240
|
+
|
241
|
+
Globally update:
|
242
|
+
|
243
|
+
```ruby
|
244
|
+
ActiveJob::Locking.options.enqueue_time = 100
|
245
|
+
```
|
246
|
+
Locally update:
|
247
|
+
|
248
|
+
```ruby
|
249
|
+
class ExampleJob < ActiveJob::Base
|
250
|
+
include ActiveJob::Locking::Serialized
|
251
|
+
|
252
|
+
self.enqueue_time = 100
|
253
|
+
end
|
254
|
+
```
|
255
|
+
|
256
|
+
### AdapterOptions
|
257
|
+
|
258
|
+
This is a hash table of options that should be sent to the lock gem when it is instantiated. Read the lock
|
259
|
+
gems documentation to find appropriate values.
|
260
|
+
|
261
|
+
Globally update:
|
262
|
+
|
263
|
+
```ruby
|
264
|
+
ActiveJob::Locking.options.adapter_options = {}
|
265
|
+
```
|
266
|
+
Locally update (notice the different method name to avoid potential conflicts):
|
267
|
+
|
268
|
+
```ruby
|
269
|
+
class ExampleJob < ActiveJob::Base
|
270
|
+
include ActiveJob::Locking::Unique
|
271
|
+
|
272
|
+
self.adapter_options = {}
|
273
|
+
end
|
274
|
+
```
|
data/lib/activejob-locking.rb
CHANGED
@@ -1,23 +1,24 @@
|
|
1
|
-
require 'activejob/locking/adapters/base'
|
2
|
-
require 'activejob/locking/adapters/memory'
|
3
|
-
|
4
|
-
require 'activejob/locking/base'
|
5
|
-
require 'activejob/locking/unique'
|
6
|
-
require 'activejob/locking/serialized'
|
7
|
-
|
8
|
-
require 'activejob/locking/options'
|
9
|
-
|
10
|
-
module ActiveJob
|
11
|
-
module Locking
|
12
|
-
@options = ActiveJob::Locking::Options.new(adapter: ActiveJob::Locking::Adapters::Memory,
|
13
|
-
hosts: ['localhost'],
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
|
22
|
-
end
|
23
|
-
|
1
|
+
require 'activejob/locking/adapters/base'
|
2
|
+
require 'activejob/locking/adapters/memory'
|
3
|
+
|
4
|
+
require 'activejob/locking/base'
|
5
|
+
require 'activejob/locking/unique'
|
6
|
+
require 'activejob/locking/serialized'
|
7
|
+
|
8
|
+
require 'activejob/locking/options'
|
9
|
+
|
10
|
+
module ActiveJob
|
11
|
+
module Locking
|
12
|
+
@options = ActiveJob::Locking::Options.new(adapter: ActiveJob::Locking::Adapters::Memory,
|
13
|
+
hosts: ['localhost'],
|
14
|
+
enqueue_time: 100, # seconds
|
15
|
+
lock_time: 100, # seconds
|
16
|
+
lock_acquire_time: 1, # seconds
|
17
|
+
adapter_options: {})
|
18
|
+
|
19
|
+
def self.options
|
20
|
+
@options
|
21
|
+
end
|
22
|
+
end
|
23
|
+
end
|
24
|
+
|
@@ -1,32 +1,32 @@
|
|
1
|
-
module ActiveJob
|
2
|
-
module Locking
|
3
|
-
module Adapters
|
4
|
-
class Base
|
5
|
-
attr_reader :key, :options, :lock_manager
|
6
|
-
attr_accessor :lock_token
|
7
|
-
|
8
|
-
def initialize(key, options)
|
9
|
-
@key = key
|
10
|
-
@options = options
|
11
|
-
@lock_manager = self.create_lock_manager
|
12
|
-
end
|
13
|
-
|
14
|
-
def create_lock_manager
|
15
|
-
raise('Subclass must implement')
|
16
|
-
end
|
17
|
-
|
18
|
-
def lock
|
19
|
-
raise('Subclass must implement')
|
20
|
-
end
|
21
|
-
|
22
|
-
def unlock
|
23
|
-
raise('Subclass must implement')
|
24
|
-
end
|
25
|
-
|
26
|
-
def refresh_lock!(refresh)
|
27
|
-
raise('Subclass must implement')
|
28
|
-
end
|
29
|
-
end
|
30
|
-
end
|
31
|
-
end
|
1
|
+
module ActiveJob
|
2
|
+
module Locking
|
3
|
+
module Adapters
|
4
|
+
class Base
|
5
|
+
attr_reader :key, :options, :lock_manager
|
6
|
+
attr_accessor :lock_token
|
7
|
+
|
8
|
+
def initialize(key, options)
|
9
|
+
@key = key
|
10
|
+
@options = options
|
11
|
+
@lock_manager = self.create_lock_manager
|
12
|
+
end
|
13
|
+
|
14
|
+
def create_lock_manager
|
15
|
+
raise('Subclass must implement')
|
16
|
+
end
|
17
|
+
|
18
|
+
def lock
|
19
|
+
raise('Subclass must implement')
|
20
|
+
end
|
21
|
+
|
22
|
+
def unlock
|
23
|
+
raise('Subclass must implement')
|
24
|
+
end
|
25
|
+
|
26
|
+
def refresh_lock!(refresh)
|
27
|
+
raise('Subclass must implement')
|
28
|
+
end
|
29
|
+
end
|
30
|
+
end
|
31
|
+
end
|
32
32
|
end
|
@@ -1,58 +1,58 @@
|
|
1
|
-
module ActiveJob
|
2
|
-
module Locking
|
3
|
-
module Adapters
|
4
|
-
class Memory < Base
|
5
|
-
@hash = Hash.new
|
6
|
-
@mutex = Mutex.new
|
7
|
-
|
8
|
-
def self.lock(key)
|
9
|
-
@mutex.synchronize do
|
10
|
-
if @hash[key]
|
11
|
-
false
|
12
|
-
else
|
13
|
-
@hash[key] = Time.now
|
14
|
-
end
|
15
|
-
end
|
16
|
-
end
|
17
|
-
|
18
|
-
def self.unlock(key)
|
19
|
-
@mutex.synchronize do
|
20
|
-
@hash.delete(key)
|
21
|
-
end
|
22
|
-
end
|
23
|
-
|
24
|
-
def self.locked?(key)
|
25
|
-
@mutex.synchronize do
|
26
|
-
@hash.include?(key)
|
27
|
-
end
|
28
|
-
end
|
29
|
-
|
30
|
-
def self.reset
|
31
|
-
@mutex.synchronize do
|
32
|
-
@hash = Hash.new
|
33
|
-
end
|
34
|
-
end
|
35
|
-
|
36
|
-
def create_lock_manager
|
37
|
-
end
|
38
|
-
|
39
|
-
def lock
|
40
|
-
finish = Time.now + self.options.lock_acquire_time
|
41
|
-
sleep_time = [5, self.options.lock_acquire_time / 5].min
|
42
|
-
|
43
|
-
begin
|
44
|
-
lock = self.class.lock(key)
|
45
|
-
return lock if lock
|
46
|
-
sleep(sleep_time)
|
47
|
-
end while Time.now < finish
|
48
|
-
|
49
|
-
false
|
50
|
-
end
|
51
|
-
|
52
|
-
def unlock
|
53
|
-
self.class.unlock(self.key)
|
54
|
-
end
|
55
|
-
end
|
56
|
-
end
|
57
|
-
end
|
1
|
+
module ActiveJob
|
2
|
+
module Locking
|
3
|
+
module Adapters
|
4
|
+
class Memory < Base
|
5
|
+
@hash = Hash.new
|
6
|
+
@mutex = Mutex.new
|
7
|
+
|
8
|
+
def self.lock(key)
|
9
|
+
@mutex.synchronize do
|
10
|
+
if @hash[key]
|
11
|
+
false
|
12
|
+
else
|
13
|
+
@hash[key] = Time.now
|
14
|
+
end
|
15
|
+
end
|
16
|
+
end
|
17
|
+
|
18
|
+
def self.unlock(key)
|
19
|
+
@mutex.synchronize do
|
20
|
+
@hash.delete(key)
|
21
|
+
end
|
22
|
+
end
|
23
|
+
|
24
|
+
def self.locked?(key)
|
25
|
+
@mutex.synchronize do
|
26
|
+
@hash.include?(key)
|
27
|
+
end
|
28
|
+
end
|
29
|
+
|
30
|
+
def self.reset
|
31
|
+
@mutex.synchronize do
|
32
|
+
@hash = Hash.new
|
33
|
+
end
|
34
|
+
end
|
35
|
+
|
36
|
+
def create_lock_manager
|
37
|
+
end
|
38
|
+
|
39
|
+
def lock
|
40
|
+
finish = Time.now + self.options.lock_acquire_time
|
41
|
+
sleep_time = [5, self.options.lock_acquire_time / 5].min
|
42
|
+
|
43
|
+
begin
|
44
|
+
lock = self.class.lock(key)
|
45
|
+
return lock if lock
|
46
|
+
sleep(sleep_time)
|
47
|
+
end while Time.now < finish
|
48
|
+
|
49
|
+
false
|
50
|
+
end
|
51
|
+
|
52
|
+
def unlock
|
53
|
+
self.class.unlock(self.key)
|
54
|
+
end
|
55
|
+
end
|
56
|
+
end
|
57
|
+
end
|
58
58
|
end
|
@@ -1,26 +1,26 @@
|
|
1
|
-
require 'redis-semaphore'
|
2
|
-
|
3
|
-
module ActiveJob
|
4
|
-
module Locking
|
5
|
-
module Adapters
|
6
|
-
class RedisSemaphore < Base
|
7
|
-
def create_lock_manager
|
8
|
-
mapped_options = {host: self.options.hosts.first,
|
9
|
-
resources: 1,
|
10
|
-
stale_client_timeout: self.options.lock_time}.merge(self.options.adapter_options)
|
11
|
-
|
12
|
-
Redis::Semaphore.new(self.key, mapped_options)
|
13
|
-
end
|
14
|
-
|
15
|
-
def lock
|
16
|
-
self.lock_token = self.lock_manager.lock(self.options.lock_acquire_time)
|
17
|
-
end
|
18
|
-
|
19
|
-
def unlock
|
20
|
-
self.lock_manager.signal(self.lock_token)
|
21
|
-
self.lock_token = nil
|
22
|
-
end
|
23
|
-
end
|
24
|
-
end
|
25
|
-
end
|
26
|
-
end
|
1
|
+
require 'redis-semaphore'
|
2
|
+
|
3
|
+
module ActiveJob
|
4
|
+
module Locking
|
5
|
+
module Adapters
|
6
|
+
class RedisSemaphore < Base
|
7
|
+
def create_lock_manager
|
8
|
+
mapped_options = {host: self.options.hosts.first,
|
9
|
+
resources: 1,
|
10
|
+
stale_client_timeout: self.options.lock_time}.merge(self.options.adapter_options)
|
11
|
+
|
12
|
+
Redis::Semaphore.new(self.key, mapped_options)
|
13
|
+
end
|
14
|
+
|
15
|
+
def lock
|
16
|
+
self.lock_token = self.lock_manager.lock(self.options.lock_acquire_time)
|
17
|
+
end
|
18
|
+
|
19
|
+
def unlock
|
20
|
+
self.lock_manager.signal(self.lock_token)
|
21
|
+
self.lock_token = nil
|
22
|
+
end
|
23
|
+
end
|
24
|
+
end
|
25
|
+
end
|
26
|
+
end
|
@@ -1,26 +1,26 @@
|
|
1
|
-
require 'redlock'
|
2
|
-
|
3
|
-
module ActiveJob
|
4
|
-
module Locking
|
5
|
-
module Adapters
|
6
|
-
class Redlock < Base
|
7
|
-
def create_lock_manager
|
8
|
-
mapped_options = self.options.adapter_options
|
9
|
-
mapped_options[:retry_count] = 2 # Try to get the lock and then try again when timeout is expiring--
|
10
|
-
mapped_options[:retry_delay] = self.options.lock_acquire_time * 1000 # convert from seconds to milliseconds
|
11
|
-
|
12
|
-
::Redlock::Client.new(self.options.hosts, mapped_options)
|
13
|
-
end
|
14
|
-
|
15
|
-
def lock
|
16
|
-
self.lock_token = self.lock_manager.lock(self.key, self.options.lock_time * 1000)
|
17
|
-
end
|
18
|
-
|
19
|
-
def unlock
|
20
|
-
self.lock_manager.unlock(self.lock_token)
|
21
|
-
self.lock_token = nil
|
22
|
-
end
|
23
|
-
end
|
24
|
-
end
|
25
|
-
end
|
26
|
-
end
|
1
|
+
require 'redlock'
|
2
|
+
|
3
|
+
module ActiveJob
|
4
|
+
module Locking
|
5
|
+
module Adapters
|
6
|
+
class Redlock < Base
|
7
|
+
def create_lock_manager
|
8
|
+
mapped_options = self.options.adapter_options
|
9
|
+
mapped_options[:retry_count] = 2 # Try to get the lock and then try again when timeout is expiring--
|
10
|
+
mapped_options[:retry_delay] = self.options.lock_acquire_time * 1000 # convert from seconds to milliseconds
|
11
|
+
|
12
|
+
::Redlock::Client.new(self.options.hosts, mapped_options)
|
13
|
+
end
|
14
|
+
|
15
|
+
def lock
|
16
|
+
self.lock_token = self.lock_manager.lock(self.key, self.options.lock_time * 1000)
|
17
|
+
end
|
18
|
+
|
19
|
+
def unlock
|
20
|
+
self.lock_manager.unlock(self.lock_token)
|
21
|
+
self.lock_token = nil
|
22
|
+
end
|
23
|
+
end
|
24
|
+
end
|
25
|
+
end
|
26
|
+
end
|
@@ -1,25 +1,25 @@
|
|
1
|
-
require 'suo'
|
2
|
-
|
3
|
-
module ActiveJob
|
4
|
-
module Locking
|
5
|
-
module Adapters
|
6
|
-
class SuoRedis < Base
|
7
|
-
def create_lock_manager
|
8
|
-
mapped_options = {connection: {host: self.options.hosts.first},
|
9
|
-
stale_lock_expiration: self.options.lock_time,
|
10
|
-
acquisition_timeout: self.options.lock_acquire_time}
|
11
|
-
|
12
|
-
Suo::Client::Redis.new(self.key, mapped_options)
|
13
|
-
end
|
14
|
-
|
15
|
-
def lock
|
16
|
-
self.lock_token = self.lock_manager.lock
|
17
|
-
end
|
18
|
-
|
19
|
-
def unlock
|
20
|
-
self.lock_manager.unlock(self.lock_token)
|
21
|
-
end
|
22
|
-
end
|
23
|
-
end
|
24
|
-
end
|
25
|
-
end
|
1
|
+
require 'suo'
|
2
|
+
|
3
|
+
module ActiveJob
|
4
|
+
module Locking
|
5
|
+
module Adapters
|
6
|
+
class SuoRedis < Base
|
7
|
+
def create_lock_manager
|
8
|
+
mapped_options = {connection: {host: self.options.hosts.first},
|
9
|
+
stale_lock_expiration: self.options.lock_time,
|
10
|
+
acquisition_timeout: self.options.lock_acquire_time}
|
11
|
+
|
12
|
+
Suo::Client::Redis.new(self.key, mapped_options)
|
13
|
+
end
|
14
|
+
|
15
|
+
def lock
|
16
|
+
self.lock_token = self.lock_manager.lock
|
17
|
+
end
|
18
|
+
|
19
|
+
def unlock
|
20
|
+
self.lock_manager.unlock(self.lock_token)
|
21
|
+
end
|
22
|
+
end
|
23
|
+
end
|
24
|
+
end
|
25
|
+
end
|
@@ -7,8 +7,8 @@ module ActiveJob
|
|
7
7
|
def lock_options
|
8
8
|
@lock_options ||= ActiveJob::Locking::Options.new
|
9
9
|
end
|
10
|
-
delegate :adapter, :hosts, :lock_time, :lock_acquire_time, :adapter_options, to: :lock_options
|
11
|
-
delegate :adapter=, :hosts=, :lock_time=, :lock_acquire_time=, :adapter_options=, to: :lock_options
|
10
|
+
delegate :adapter, :enqueue_time, :hosts, :lock_time, :lock_acquire_time, :adapter_options, to: :lock_options
|
11
|
+
delegate :adapter=, :enqueue_time=, :hosts=, :lock_time=, :lock_acquire_time=, :adapter_options=, to: :lock_options
|
12
12
|
end
|
13
13
|
|
14
14
|
included do
|
@@ -7,16 +7,31 @@ module ActiveJob
|
|
7
7
|
attr_accessor :hosts
|
8
8
|
attr_accessor :lock_time
|
9
9
|
attr_accessor :lock_acquire_time
|
10
|
+
attr_accessor :enqueue_time
|
10
11
|
attr_accessor :adapter_options
|
11
12
|
|
12
13
|
def initialize(options = {})
|
13
14
|
@adapter = options[:adapter]
|
14
15
|
@hosts = options[:hosts]
|
16
|
+
@enqueue_time = options[:enqueue_time]
|
15
17
|
@lock_time = options[:lock_time]
|
16
18
|
@lock_acquire_time = options[:lock_acquire_time]
|
17
19
|
@adapter_options = options[:adapter_options]
|
18
20
|
end
|
19
21
|
|
22
|
+
def enqueue_time=(value)
|
23
|
+
case value
|
24
|
+
when NilClass
|
25
|
+
raise(ArgumentError, 'Enqueue time must be set')
|
26
|
+
when ActiveSupport::Duration
|
27
|
+
@enqueue_time = value.value
|
28
|
+
when 0
|
29
|
+
raise(ArgumentError, 'Enqueue time must be greater than zero')
|
30
|
+
else
|
31
|
+
@enqueue_time = value
|
32
|
+
end
|
33
|
+
end
|
34
|
+
|
20
35
|
def lock_time=(value)
|
21
36
|
case value
|
22
37
|
when NilClass
|
@@ -47,6 +62,7 @@ module ActiveJob
|
|
47
62
|
result = self.dup
|
48
63
|
result.adapter = other.adapter if other.adapter
|
49
64
|
result.hosts = other.hosts if other.hosts
|
65
|
+
result.enqueue_time = other.enqueue_time if other.enqueue_time
|
50
66
|
result.lock_time = other.lock_time if other.lock_time
|
51
67
|
result.lock_acquire_time = other.lock_acquire_time if other.lock_acquire_time
|
52
68
|
result.adapter_options = other.adapter_options if other.adapter_options
|
data/test/jobs/fail_job.rb
CHANGED
@@ -1,15 +1,15 @@
|
|
1
|
-
class FailJob < ActiveJob::Base
|
2
|
-
include ActiveJob::Locking::Unique
|
3
|
-
|
4
|
-
self.lock_acquire_time = 2
|
5
|
-
|
6
|
-
# We want the job ids to be all the same for testing
|
7
|
-
def lock_key(index, sleep_time)
|
8
|
-
self.class.name
|
9
|
-
end
|
10
|
-
|
11
|
-
# Pass in index so we can distinguish different jobs
|
12
|
-
def perform(index, sleep_time)
|
13
|
-
raise(ArgumentError, 'Job failed')
|
14
|
-
end
|
1
|
+
class FailJob < ActiveJob::Base
|
2
|
+
include ActiveJob::Locking::Unique
|
3
|
+
|
4
|
+
self.lock_acquire_time = 2
|
5
|
+
|
6
|
+
# We want the job ids to be all the same for testing
|
7
|
+
def lock_key(index, sleep_time)
|
8
|
+
self.class.name
|
9
|
+
end
|
10
|
+
|
11
|
+
# Pass in index so we can distinguish different jobs
|
12
|
+
def perform(index, sleep_time)
|
13
|
+
raise(ArgumentError, 'Job failed')
|
14
|
+
end
|
15
15
|
end
|
data/test/jobs/serial_job.rb
CHANGED
@@ -1,15 +1,15 @@
|
|
1
|
-
class SerialJob < ActiveJob::Base
|
2
|
-
include ActiveJob::Locking::Serialized
|
3
|
-
|
4
|
-
self.lock_acquire_time = 2
|
5
|
-
|
6
|
-
# We want the job ids to be all the same for testing
|
7
|
-
def lock_key(index, sleep_time)
|
8
|
-
self.class.name
|
9
|
-
end
|
10
|
-
|
11
|
-
# Pass in index so we can distinguish different jobs
|
12
|
-
def perform(index, sleep_time)
|
13
|
-
sleep(sleep_time)
|
14
|
-
end
|
1
|
+
class SerialJob < ActiveJob::Base
|
2
|
+
include ActiveJob::Locking::Serialized
|
3
|
+
|
4
|
+
self.lock_acquire_time = 2
|
5
|
+
|
6
|
+
# We want the job ids to be all the same for testing
|
7
|
+
def lock_key(index, sleep_time)
|
8
|
+
self.class.name
|
9
|
+
end
|
10
|
+
|
11
|
+
# Pass in index so we can distinguish different jobs
|
12
|
+
def perform(index, sleep_time)
|
13
|
+
sleep(sleep_time)
|
14
|
+
end
|
15
15
|
end
|
data/test/jobs/unique_job.rb
CHANGED
@@ -1,15 +1,15 @@
|
|
1
|
-
class UniqueJob < ActiveJob::Base
|
2
|
-
include ActiveJob::Locking::Unique
|
3
|
-
|
4
|
-
self.lock_acquire_time = 2
|
5
|
-
|
6
|
-
# We want the job ids to be all the same for testing
|
7
|
-
def lock_key(index, sleep_time)
|
8
|
-
self.class.name
|
9
|
-
end
|
10
|
-
|
11
|
-
# Pass in index so we can distinguish different jobs
|
12
|
-
def perform(index, sleep_time)
|
13
|
-
sleep(sleep_time)
|
14
|
-
end
|
1
|
+
class UniqueJob < ActiveJob::Base
|
2
|
+
include ActiveJob::Locking::Unique
|
3
|
+
|
4
|
+
self.lock_acquire_time = 2
|
5
|
+
|
6
|
+
# We want the job ids to be all the same for testing
|
7
|
+
def lock_key(index, sleep_time)
|
8
|
+
self.class.name
|
9
|
+
end
|
10
|
+
|
11
|
+
# Pass in index so we can distinguish different jobs
|
12
|
+
def perform(index, sleep_time)
|
13
|
+
sleep(sleep_time)
|
14
|
+
end
|
15
15
|
end
|
data/test/test_suite.rb
CHANGED
@@ -1,12 +1,12 @@
|
|
1
|
-
%w(
|
2
|
-
test_serialized_memory
|
3
|
-
test_serialized_redis_semaphore
|
4
|
-
test_serialized_redlock
|
5
|
-
test_serialized_suo_redis
|
6
|
-
test_unique_memory
|
7
|
-
test_unique_redis_semaphore
|
8
|
-
test_unique_redlock
|
9
|
-
test_unique_suo_redis
|
10
|
-
).each do |test|
|
11
|
-
require File.expand_path("../#{test}", __FILE__)
|
12
|
-
end
|
1
|
+
%w(
|
2
|
+
test_serialized_memory
|
3
|
+
test_serialized_redis_semaphore
|
4
|
+
test_serialized_redlock
|
5
|
+
test_serialized_suo_redis
|
6
|
+
test_unique_memory
|
7
|
+
test_unique_redis_semaphore
|
8
|
+
test_unique_redlock
|
9
|
+
test_unique_suo_redis
|
10
|
+
).each do |test|
|
11
|
+
require File.expand_path("../#{test}", __FILE__)
|
12
|
+
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: activejob-locking
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.5.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Charlie Savage
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2017-
|
11
|
+
date: 2017-10-03 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activejob
|
@@ -154,7 +154,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
154
154
|
version: '0'
|
155
155
|
requirements: []
|
156
156
|
rubyforge_project:
|
157
|
-
rubygems_version: 2.6.
|
157
|
+
rubygems_version: 2.6.13
|
158
158
|
signing_key:
|
159
159
|
specification_version: 4
|
160
160
|
summary: ActiveJob locking to control how jobs are enqueued and performed.
|