activejob-locking 0.4.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/HISTORY.md +21 -16
- data/README.md +274 -247
- data/lib/activejob-locking.rb +24 -23
- data/lib/activejob/locking/adapters/base.rb +31 -31
- data/lib/activejob/locking/adapters/memory.rb +57 -57
- data/lib/activejob/locking/adapters/redis-semaphore.rb +26 -26
- data/lib/activejob/locking/adapters/redlock.rb +26 -26
- data/lib/activejob/locking/adapters/suo-redis.rb +25 -25
- data/lib/activejob/locking/base.rb +2 -2
- data/lib/activejob/locking/options.rb +16 -0
- data/lib/activejob/locking/serialized.rb +1 -1
- data/test/jobs/fail_job.rb +14 -14
- data/test/jobs/serial_job.rb +14 -14
- data/test/jobs/unique_job.rb +14 -14
- data/test/test_suite.rb +12 -12
- metadata +3 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: e574b3a41612fff773ca03309e55b6a2a4bb2db0
|
4
|
+
data.tar.gz: 3ef9de73fa5ac528c317fd8218be929ce3e975f8
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 28f284c0b260bfdc69387d901f5fb217d1afb512127b05a4a34a8399e0cf36718c33f0c60e857686572f0c4d340a9c83c8889ef3a50510a06114fe961e27b7df
|
7
|
+
data.tar.gz: 65c953f75b1795f7be08cd0d262b01584a6f850feb16ac4a7f2432fcedbd964e07820d7ebaa11e3a6c4a043cefbb2bdef315a2529b90069975b0778f13c23ab1
|
data/HISTORY.md
CHANGED
@@ -1,16 +1,21 @@
|
|
1
|
-
## 0.
|
2
|
-
|
3
|
-
-
|
4
|
-
|
5
|
-
|
6
|
-
|
7
|
-
|
8
|
-
|
9
|
-
|
10
|
-
|
11
|
-
|
12
|
-
-
|
13
|
-
|
14
|
-
## 0.
|
15
|
-
|
16
|
-
-
|
1
|
+
## 0.5.0 (2017-10-03)
|
2
|
+
|
3
|
+
- Add enqueue_time parameter
|
4
|
+
- Fix bug in setting enqueue time
|
5
|
+
|
6
|
+
## 0.4.0 (2017-06-21)
|
7
|
+
|
8
|
+
- Cleanup handling of hosts
|
9
|
+
|
10
|
+
## 0.3.0 (2017-06-21)
|
11
|
+
|
12
|
+
- Change lock_key signature to match perform signature
|
13
|
+
|
14
|
+
## 0.2.0 (2017-06-20)
|
15
|
+
|
16
|
+
- Bug fixes
|
17
|
+
- Improved tests
|
18
|
+
|
19
|
+
## 0.1.0 (2017-01-16)
|
20
|
+
|
21
|
+
- Initial release
|
data/README.md
CHANGED
@@ -1,247 +1,274 @@
|
|
1
|
-
ActiveJob Locking
|
2
|
-
===================
|
3
|
-
|
4
|
-
[](http://travis-ci.org/cfis/activejob-locking)
|
5
|
-
[](http://badge.fury.io/rb/activejob-locking)
|
6
|
-
|
7
|
-
activejob-locking lets you control how ActiveJobs are enqueued and performed:
|
8
|
-
|
9
|
-
* Allow only one job to be enqueued at a time - thus a "unique" job
|
10
|
-
* Allow only one job to be performed at a time - thus a "serialized" job
|
11
|
-
|
12
|
-
There are many other similar gems including [resque-lock-timeout](https://github.com/lantins/resque-lock-timeout),
|
13
|
-
[activejob-traffic-control](https://github.com/nickelser/activejob-traffic_control), [activejob-lock](https://github.com/idolweb/activejob-lock),
|
14
|
-
[activejob-locks](https://github.com/erickrause/activejob-locks). What is different about this gem is that it
|
15
|
-
is agnostic on the locking mechanism. In the same way that ActiveJob works with many apapters, ActiveJob Locking
|
16
|
-
works with a variety of locking gems.
|
17
|
-
|
18
|
-
Installation
|
19
|
-
------------
|
20
|
-
|
21
|
-
Add this line to your application's Gemfile:
|
22
|
-
|
23
|
-
```ruby
|
24
|
-
gem 'activejob-locking'
|
25
|
-
```
|
26
|
-
|
27
|
-
Unique Jobs
|
28
|
-
------------
|
29
|
-
Sometime you only want to enqueue one instance of a job. No other similar job should be enqueued until the first one
|
30
|
-
is completed.
|
31
|
-
|
32
|
-
```ruby
|
33
|
-
class UniqueJob < ActiveJob::Base
|
34
|
-
include ActiveJob::Locking::Unique
|
35
|
-
|
36
|
-
# Make sure the lock_key is always the same
|
37
|
-
def lock_key(object)
|
38
|
-
self.class.name
|
39
|
-
end
|
40
|
-
|
41
|
-
def perform(object)
|
42
|
-
# do some work
|
43
|
-
end
|
44
|
-
end
|
45
|
-
```
|
46
|
-
Only one instance of this job will ever be enqueued. If an additional job is enqueued, it will either be dropped and
|
47
|
-
never be enqueued or it will wait to the first job is performed. That is controlled by the job
|
48
|
-
[options](##options) described below.
|
49
|
-
|
50
|
-
|
51
|
-
Serialized Jobs
|
52
|
-
------------
|
53
|
-
Sometime you only want to perform one instance of a job at a time. No other similar job should be performed until the first one
|
54
|
-
is completed.
|
55
|
-
|
56
|
-
```ruby
|
57
|
-
class SerializedJob < ActiveJob::Base
|
58
|
-
include ActiveJob::Locking::Serialized
|
59
|
-
|
60
|
-
# Make sure the lock_key is always the same
|
61
|
-
def lock_key
|
62
|
-
self.class.name
|
63
|
-
end
|
64
|
-
|
65
|
-
def perform
|
66
|
-
# do some work
|
67
|
-
end
|
68
|
-
end
|
69
|
-
```
|
70
|
-
Only one instance of this job will ever be performed. If an additional job is enqueued, it will wait in its que until
|
71
|
-
to the first job is performed.
|
72
|
-
|
73
|
-
Locking
|
74
|
-
------------
|
75
|
-
Locks are used to control how jobs are enqueued and performed. The idea is that locks are stored in a distributed
|
76
|
-
system such as [Redis](https://redis.io/) or [Memcached](https://memcached.org/) so they can be used by
|
77
|
-
multiple servers to coordinate the enqueueing and performing of jobs.
|
78
|
-
|
79
|
-
The ActiveJob Locking gem does not include a locking implementation. Instead it provides adapters for
|
80
|
-
distributed locking gems.
|
81
|
-
|
82
|
-
Currently three gems are supported:
|
83
|
-
|
84
|
-
* [redis-semaphore](https://github.com/dv/redis-semaphore)
|
85
|
-
|
86
|
-
* [suo](https://github.com/nickelser/suo)
|
87
|
-
|
88
|
-
* [redlock-rb](https://github.com/leandromoreira/redlock-rb)
|
89
|
-
|
90
|
-
If you would like to have an additional locking mechanism supported, please feel free to send in a pull request.
|
91
|
-
|
92
|
-
Please see the [options](##options) section below on how to specify a locking adapter.
|
93
|
-
|
94
|
-
|
95
|
-
Lock Key
|
96
|
-
---------
|
97
|
-
|
98
|
-
Notice that the code samples above include a `lock_key` method. The return value of this method is used by the
|
99
|
-
gem to create locks behind the scenes. Thus it holds the key (pun intended) to controlling how jobs are enqueued
|
100
|
-
and performed.
|
101
|
-
|
102
|
-
By default the key is defined as:
|
103
|
-
|
104
|
-
```ruby
|
105
|
-
def lock_key(*args)
|
106
|
-
[self.class.name, serialize_arguments(self.arguments)].join('/')
|
107
|
-
end
|
108
|
-
```
|
109
|
-
Thus it has the format `<job class name>/<serialized_job_arguments>`
|
110
|
-
|
111
|
-
The args passed to the lock key method are the same that are passed to the job's perform method.
|
112
|
-
|
113
|
-
To use this gem, you will want to override this method per job.
|
114
|
-
|
115
|
-
### Examples
|
116
|
-
|
117
|
-
Allow only one job per queue to be enqueued or performed:
|
118
|
-
|
119
|
-
```ruby
|
120
|
-
def lock_key(*args)
|
121
|
-
self.queue
|
122
|
-
end
|
123
|
-
```
|
124
|
-
|
125
|
-
Allow only one instance of a job class to be enqueued of performed:
|
126
|
-
|
127
|
-
```ruby
|
128
|
-
def lock_key(*args)
|
129
|
-
self.class.name
|
130
|
-
end
|
131
|
-
```
|
132
|
-
|
133
|
-
Options
|
134
|
-
-------
|
135
|
-
The locking behavior can be dramatically changed by tweaking various options. There is a global set of options
|
136
|
-
available at:
|
137
|
-
|
138
|
-
```ruby
|
139
|
-
ActiveJob::Locking.options
|
140
|
-
```
|
141
|
-
This should be updated using a Rails initializer. Each job class can override individual options as it sees fit.
|
142
|
-
|
143
|
-
### Adapter
|
144
|
-
|
145
|
-
Use the adapter option to specify which locking gem to use.
|
146
|
-
|
147
|
-
Globally update:
|
148
|
-
|
149
|
-
```ruby
|
150
|
-
ActiveJob::Locking.options.adapter = ActiveJob::Locking::Adapters::SuoRedis
|
151
|
-
```
|
152
|
-
Locally update:
|
153
|
-
|
154
|
-
```ruby
|
155
|
-
class ExampleJob < ActiveJob::Base
|
156
|
-
include ActiveJob::Locking::Serialized
|
157
|
-
|
158
|
-
self.adapter = ActiveJob::Locking::Adapters::SuoRedis
|
159
|
-
end
|
160
|
-
```
|
161
|
-
|
162
|
-
### Hosts
|
163
|
-
|
164
|
-
An array of hosts for the distributed system. This format is dependent on the locking gem, but generally is a url or an existing Memcache or Redis
|
165
|
-
connection. Please refer to the appropriate locking gem's documentation documentation.
|
166
|
-
|
167
|
-
Globally update:
|
168
|
-
|
169
|
-
```ruby
|
170
|
-
ActiveJob::Locking.options.hosts = ['localhost']
|
171
|
-
```
|
172
|
-
Locally update:
|
173
|
-
|
174
|
-
```ruby
|
175
|
-
class ExampleJob < ActiveJob::Base
|
176
|
-
include ActiveJob::Locking::Serialized
|
177
|
-
|
178
|
-
self.hosts = ['localhost']
|
179
|
-
end
|
180
|
-
```
|
181
|
-
|
182
|
-
###
|
183
|
-
|
184
|
-
The is the
|
185
|
-
|
186
|
-
|
187
|
-
|
188
|
-
|
189
|
-
|
190
|
-
|
191
|
-
|
192
|
-
|
193
|
-
|
194
|
-
```
|
195
|
-
|
196
|
-
|
197
|
-
|
198
|
-
|
199
|
-
|
200
|
-
|
201
|
-
|
202
|
-
|
203
|
-
|
204
|
-
|
205
|
-
|
206
|
-
|
207
|
-
|
208
|
-
|
209
|
-
|
210
|
-
|
211
|
-
|
212
|
-
The value is specified in seconds and defaults to
|
213
|
-
|
214
|
-
|
215
|
-
|
216
|
-
|
217
|
-
|
218
|
-
|
219
|
-
|
220
|
-
|
221
|
-
|
222
|
-
|
223
|
-
|
224
|
-
|
225
|
-
|
226
|
-
|
227
|
-
|
228
|
-
|
229
|
-
|
230
|
-
|
231
|
-
|
232
|
-
|
233
|
-
|
234
|
-
|
235
|
-
|
236
|
-
|
237
|
-
|
238
|
-
|
239
|
-
|
240
|
-
|
241
|
-
|
242
|
-
|
243
|
-
|
244
|
-
|
245
|
-
|
246
|
-
|
247
|
-
|
1
|
+
ActiveJob Locking
|
2
|
+
===================
|
3
|
+
|
4
|
+
[](http://travis-ci.org/cfis/activejob-locking)
|
5
|
+
[](http://badge.fury.io/rb/activejob-locking)
|
6
|
+
|
7
|
+
activejob-locking lets you control how ActiveJobs are enqueued and performed:
|
8
|
+
|
9
|
+
* Allow only one job to be enqueued at a time - thus a "unique" job
|
10
|
+
* Allow only one job to be performed at a time - thus a "serialized" job
|
11
|
+
|
12
|
+
There are many other similar gems including [resque-lock-timeout](https://github.com/lantins/resque-lock-timeout),
|
13
|
+
[activejob-traffic-control](https://github.com/nickelser/activejob-traffic_control), [activejob-lock](https://github.com/idolweb/activejob-lock),
|
14
|
+
[activejob-locks](https://github.com/erickrause/activejob-locks). What is different about this gem is that it
|
15
|
+
is agnostic on the locking mechanism. In the same way that ActiveJob works with many apapters, ActiveJob Locking
|
16
|
+
works with a variety of locking gems.
|
17
|
+
|
18
|
+
Installation
|
19
|
+
------------
|
20
|
+
|
21
|
+
Add this line to your application's Gemfile:
|
22
|
+
|
23
|
+
```ruby
|
24
|
+
gem 'activejob-locking'
|
25
|
+
```
|
26
|
+
|
27
|
+
Unique Jobs
|
28
|
+
------------
|
29
|
+
Sometime you only want to enqueue one instance of a job. No other similar job should be enqueued until the first one
|
30
|
+
is completed.
|
31
|
+
|
32
|
+
```ruby
|
33
|
+
class UniqueJob < ActiveJob::Base
|
34
|
+
include ActiveJob::Locking::Unique
|
35
|
+
|
36
|
+
# Make sure the lock_key is always the same
|
37
|
+
def lock_key(object)
|
38
|
+
self.class.name
|
39
|
+
end
|
40
|
+
|
41
|
+
def perform(object)
|
42
|
+
# do some work
|
43
|
+
end
|
44
|
+
end
|
45
|
+
```
|
46
|
+
Only one instance of this job will ever be enqueued. If an additional job is enqueued, it will either be dropped and
|
47
|
+
never be enqueued or it will wait to the first job is performed. That is controlled by the job
|
48
|
+
[options](##options) described below.
|
49
|
+
|
50
|
+
|
51
|
+
Serialized Jobs
|
52
|
+
------------
|
53
|
+
Sometime you only want to perform one instance of a job at a time. No other similar job should be performed until the first one
|
54
|
+
is completed.
|
55
|
+
|
56
|
+
```ruby
|
57
|
+
class SerializedJob < ActiveJob::Base
|
58
|
+
include ActiveJob::Locking::Serialized
|
59
|
+
|
60
|
+
# Make sure the lock_key is always the same
|
61
|
+
def lock_key
|
62
|
+
self.class.name
|
63
|
+
end
|
64
|
+
|
65
|
+
def perform
|
66
|
+
# do some work
|
67
|
+
end
|
68
|
+
end
|
69
|
+
```
|
70
|
+
Only one instance of this job will ever be performed. If an additional job is enqueued, it will wait in its que until
|
71
|
+
to the first job is performed.
|
72
|
+
|
73
|
+
Locking
|
74
|
+
------------
|
75
|
+
Locks are used to control how jobs are enqueued and performed. The idea is that locks are stored in a distributed
|
76
|
+
system such as [Redis](https://redis.io/) or [Memcached](https://memcached.org/) so they can be used by
|
77
|
+
multiple servers to coordinate the enqueueing and performing of jobs.
|
78
|
+
|
79
|
+
The ActiveJob Locking gem does not include a locking implementation. Instead it provides adapters for
|
80
|
+
distributed locking gems.
|
81
|
+
|
82
|
+
Currently three gems are supported:
|
83
|
+
|
84
|
+
* [redis-semaphore](https://github.com/dv/redis-semaphore)
|
85
|
+
|
86
|
+
* [suo](https://github.com/nickelser/suo)
|
87
|
+
|
88
|
+
* [redlock-rb](https://github.com/leandromoreira/redlock-rb)
|
89
|
+
|
90
|
+
If you would like to have an additional locking mechanism supported, please feel free to send in a pull request.
|
91
|
+
|
92
|
+
Please see the [options](##options) section below on how to specify a locking adapter.
|
93
|
+
|
94
|
+
|
95
|
+
Lock Key
|
96
|
+
---------
|
97
|
+
|
98
|
+
Notice that the code samples above include a `lock_key` method. The return value of this method is used by the
|
99
|
+
gem to create locks behind the scenes. Thus it holds the key (pun intended) to controlling how jobs are enqueued
|
100
|
+
and performed.
|
101
|
+
|
102
|
+
By default the key is defined as:
|
103
|
+
|
104
|
+
```ruby
|
105
|
+
def lock_key(*args)
|
106
|
+
[self.class.name, serialize_arguments(self.arguments)].join('/')
|
107
|
+
end
|
108
|
+
```
|
109
|
+
Thus it has the format `<job class name>/<serialized_job_arguments>`
|
110
|
+
|
111
|
+
The args passed to the lock key method are the same that are passed to the job's perform method.
|
112
|
+
|
113
|
+
To use this gem, you will want to override this method per job.
|
114
|
+
|
115
|
+
### Examples
|
116
|
+
|
117
|
+
Allow only one job per queue to be enqueued or performed:
|
118
|
+
|
119
|
+
```ruby
|
120
|
+
def lock_key(*args)
|
121
|
+
self.queue
|
122
|
+
end
|
123
|
+
```
|
124
|
+
|
125
|
+
Allow only one instance of a job class to be enqueued of performed:
|
126
|
+
|
127
|
+
```ruby
|
128
|
+
def lock_key(*args)
|
129
|
+
self.class.name
|
130
|
+
end
|
131
|
+
```
|
132
|
+
|
133
|
+
Options
|
134
|
+
-------
|
135
|
+
The locking behavior can be dramatically changed by tweaking various options. There is a global set of options
|
136
|
+
available at:
|
137
|
+
|
138
|
+
```ruby
|
139
|
+
ActiveJob::Locking.options
|
140
|
+
```
|
141
|
+
This should be updated using a Rails initializer. Each job class can override individual options as it sees fit.
|
142
|
+
|
143
|
+
### Adapter
|
144
|
+
|
145
|
+
Use the adapter option to specify which locking gem to use.
|
146
|
+
|
147
|
+
Globally update:
|
148
|
+
|
149
|
+
```ruby
|
150
|
+
ActiveJob::Locking.options.adapter = ActiveJob::Locking::Adapters::SuoRedis
|
151
|
+
```
|
152
|
+
Locally update:
|
153
|
+
|
154
|
+
```ruby
|
155
|
+
class ExampleJob < ActiveJob::Base
|
156
|
+
include ActiveJob::Locking::Serialized
|
157
|
+
|
158
|
+
self.adapter = ActiveJob::Locking::Adapters::SuoRedis
|
159
|
+
end
|
160
|
+
```
|
161
|
+
|
162
|
+
### Hosts
|
163
|
+
|
164
|
+
An array of hosts for the distributed system. This format is dependent on the locking gem, but generally is a url or an existing Memcache or Redis
|
165
|
+
connection. Please refer to the appropriate locking gem's documentation documentation.
|
166
|
+
|
167
|
+
Globally update:
|
168
|
+
|
169
|
+
```ruby
|
170
|
+
ActiveJob::Locking.options.hosts = ['localhost']
|
171
|
+
```
|
172
|
+
Locally update:
|
173
|
+
|
174
|
+
```ruby
|
175
|
+
class ExampleJob < ActiveJob::Base
|
176
|
+
include ActiveJob::Locking::Serialized
|
177
|
+
|
178
|
+
self.hosts = ['localhost']
|
179
|
+
end
|
180
|
+
```
|
181
|
+
|
182
|
+
### lock_time
|
183
|
+
|
184
|
+
The is the time to live for any acquired locks. For most locking gems this is mapped to their concept of "stale" locks.
|
185
|
+
That means that if an attempt is made to access the lock after it is expired, it will be considered unlocked. That is in
|
186
|
+
contrast to aggressively removing locks for running jobs even if no other job has requested them.
|
187
|
+
|
188
|
+
The value is specified in seconds and defaults to 100.
|
189
|
+
|
190
|
+
Globally update:
|
191
|
+
|
192
|
+
```ruby
|
193
|
+
ActiveJob::Locking.options.lock_time = 100
|
194
|
+
```
|
195
|
+
Locally update:
|
196
|
+
|
197
|
+
```ruby
|
198
|
+
class ExampleJob < ActiveJob::Base
|
199
|
+
include ActiveJob::Locking::Serialized
|
200
|
+
|
201
|
+
self.lock_time = 100
|
202
|
+
end
|
203
|
+
```
|
204
|
+
|
205
|
+
You almost surely want the lock_time to be greater than the time it takes to execute the job. Otherwise, the lock will expire
|
206
|
+
and extra jobs will start to run. When the job finishes, or fails, the lock will be released. However, remember that the job
|
207
|
+
could be terminated by the operating system or a monitoring system (such as monit). In that case, the lock won't be released
|
208
|
+
and will remain in force until its lock_time expires.
|
209
|
+
|
210
|
+
### lock_acquire_time
|
211
|
+
|
212
|
+
The is the timeout for acquiring a lock. The value is specified in seconds and defaults to 1. It must
|
213
|
+
be greater than zero and cannot be nil.
|
214
|
+
|
215
|
+
Globally update:
|
216
|
+
|
217
|
+
```ruby
|
218
|
+
ActiveJob::Locking.options.lock_acquire_time = 1
|
219
|
+
```
|
220
|
+
Locally update:
|
221
|
+
|
222
|
+
```ruby
|
223
|
+
class ExampleJob < ActiveJob::Base
|
224
|
+
include ActiveJob::Locking::Unique
|
225
|
+
|
226
|
+
self.lock_acquire_time = 1
|
227
|
+
end
|
228
|
+
```
|
229
|
+
|
230
|
+
Remember that most locking gems block the current thread when trying to acquire a lock. Therefore you likely want
|
231
|
+
lock_acquire_time to be low. However, the lower it is the more likely that unique jobs that are enqueued will
|
232
|
+
expire and be dropped.
|
233
|
+
|
234
|
+
### enqueue_time
|
235
|
+
|
236
|
+
The is the time to re-enqueue a job if the lock_time has expired. Thus this value is only relevant for
|
237
|
+
serialized jobs since unique jobs will be dropped instead of enqueded.
|
238
|
+
|
239
|
+
The value is specified in seconds and defaults to 100.
|
240
|
+
|
241
|
+
Globally update:
|
242
|
+
|
243
|
+
```ruby
|
244
|
+
ActiveJob::Locking.options.enqueue_time = 100
|
245
|
+
```
|
246
|
+
Locally update:
|
247
|
+
|
248
|
+
```ruby
|
249
|
+
class ExampleJob < ActiveJob::Base
|
250
|
+
include ActiveJob::Locking::Serialized
|
251
|
+
|
252
|
+
self.enqueue_time = 100
|
253
|
+
end
|
254
|
+
```
|
255
|
+
|
256
|
+
### AdapterOptions
|
257
|
+
|
258
|
+
This is a hash table of options that should be sent to the lock gem when it is instantiated. Read the lock
|
259
|
+
gems documentation to find appropriate values.
|
260
|
+
|
261
|
+
Globally update:
|
262
|
+
|
263
|
+
```ruby
|
264
|
+
ActiveJob::Locking.options.adapter_options = {}
|
265
|
+
```
|
266
|
+
Locally update (notice the different method name to avoid potential conflicts):
|
267
|
+
|
268
|
+
```ruby
|
269
|
+
class ExampleJob < ActiveJob::Base
|
270
|
+
include ActiveJob::Locking::Unique
|
271
|
+
|
272
|
+
self.adapter_options = {}
|
273
|
+
end
|
274
|
+
```
|
data/lib/activejob-locking.rb
CHANGED
@@ -1,23 +1,24 @@
|
|
1
|
-
require 'activejob/locking/adapters/base'
|
2
|
-
require 'activejob/locking/adapters/memory'
|
3
|
-
|
4
|
-
require 'activejob/locking/base'
|
5
|
-
require 'activejob/locking/unique'
|
6
|
-
require 'activejob/locking/serialized'
|
7
|
-
|
8
|
-
require 'activejob/locking/options'
|
9
|
-
|
10
|
-
module ActiveJob
|
11
|
-
module Locking
|
12
|
-
@options = ActiveJob::Locking::Options.new(adapter: ActiveJob::Locking::Adapters::Memory,
|
13
|
-
hosts: ['localhost'],
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
|
22
|
-
end
|
23
|
-
|
1
|
+
require 'activejob/locking/adapters/base'
|
2
|
+
require 'activejob/locking/adapters/memory'
|
3
|
+
|
4
|
+
require 'activejob/locking/base'
|
5
|
+
require 'activejob/locking/unique'
|
6
|
+
require 'activejob/locking/serialized'
|
7
|
+
|
8
|
+
require 'activejob/locking/options'
|
9
|
+
|
10
|
+
module ActiveJob
|
11
|
+
module Locking
|
12
|
+
@options = ActiveJob::Locking::Options.new(adapter: ActiveJob::Locking::Adapters::Memory,
|
13
|
+
hosts: ['localhost'],
|
14
|
+
enqueue_time: 100, # seconds
|
15
|
+
lock_time: 100, # seconds
|
16
|
+
lock_acquire_time: 1, # seconds
|
17
|
+
adapter_options: {})
|
18
|
+
|
19
|
+
def self.options
|
20
|
+
@options
|
21
|
+
end
|
22
|
+
end
|
23
|
+
end
|
24
|
+
|
@@ -1,32 +1,32 @@
|
|
1
|
-
module ActiveJob
|
2
|
-
module Locking
|
3
|
-
module Adapters
|
4
|
-
class Base
|
5
|
-
attr_reader :key, :options, :lock_manager
|
6
|
-
attr_accessor :lock_token
|
7
|
-
|
8
|
-
def initialize(key, options)
|
9
|
-
@key = key
|
10
|
-
@options = options
|
11
|
-
@lock_manager = self.create_lock_manager
|
12
|
-
end
|
13
|
-
|
14
|
-
def create_lock_manager
|
15
|
-
raise('Subclass must implement')
|
16
|
-
end
|
17
|
-
|
18
|
-
def lock
|
19
|
-
raise('Subclass must implement')
|
20
|
-
end
|
21
|
-
|
22
|
-
def unlock
|
23
|
-
raise('Subclass must implement')
|
24
|
-
end
|
25
|
-
|
26
|
-
def refresh_lock!(refresh)
|
27
|
-
raise('Subclass must implement')
|
28
|
-
end
|
29
|
-
end
|
30
|
-
end
|
31
|
-
end
|
1
|
+
module ActiveJob
|
2
|
+
module Locking
|
3
|
+
module Adapters
|
4
|
+
class Base
|
5
|
+
attr_reader :key, :options, :lock_manager
|
6
|
+
attr_accessor :lock_token
|
7
|
+
|
8
|
+
def initialize(key, options)
|
9
|
+
@key = key
|
10
|
+
@options = options
|
11
|
+
@lock_manager = self.create_lock_manager
|
12
|
+
end
|
13
|
+
|
14
|
+
def create_lock_manager
|
15
|
+
raise('Subclass must implement')
|
16
|
+
end
|
17
|
+
|
18
|
+
def lock
|
19
|
+
raise('Subclass must implement')
|
20
|
+
end
|
21
|
+
|
22
|
+
def unlock
|
23
|
+
raise('Subclass must implement')
|
24
|
+
end
|
25
|
+
|
26
|
+
def refresh_lock!(refresh)
|
27
|
+
raise('Subclass must implement')
|
28
|
+
end
|
29
|
+
end
|
30
|
+
end
|
31
|
+
end
|
32
32
|
end
|
@@ -1,58 +1,58 @@
|
|
1
|
-
module ActiveJob
|
2
|
-
module Locking
|
3
|
-
module Adapters
|
4
|
-
class Memory < Base
|
5
|
-
@hash = Hash.new
|
6
|
-
@mutex = Mutex.new
|
7
|
-
|
8
|
-
def self.lock(key)
|
9
|
-
@mutex.synchronize do
|
10
|
-
if @hash[key]
|
11
|
-
false
|
12
|
-
else
|
13
|
-
@hash[key] = Time.now
|
14
|
-
end
|
15
|
-
end
|
16
|
-
end
|
17
|
-
|
18
|
-
def self.unlock(key)
|
19
|
-
@mutex.synchronize do
|
20
|
-
@hash.delete(key)
|
21
|
-
end
|
22
|
-
end
|
23
|
-
|
24
|
-
def self.locked?(key)
|
25
|
-
@mutex.synchronize do
|
26
|
-
@hash.include?(key)
|
27
|
-
end
|
28
|
-
end
|
29
|
-
|
30
|
-
def self.reset
|
31
|
-
@mutex.synchronize do
|
32
|
-
@hash = Hash.new
|
33
|
-
end
|
34
|
-
end
|
35
|
-
|
36
|
-
def create_lock_manager
|
37
|
-
end
|
38
|
-
|
39
|
-
def lock
|
40
|
-
finish = Time.now + self.options.lock_acquire_time
|
41
|
-
sleep_time = [5, self.options.lock_acquire_time / 5].min
|
42
|
-
|
43
|
-
begin
|
44
|
-
lock = self.class.lock(key)
|
45
|
-
return lock if lock
|
46
|
-
sleep(sleep_time)
|
47
|
-
end while Time.now < finish
|
48
|
-
|
49
|
-
false
|
50
|
-
end
|
51
|
-
|
52
|
-
def unlock
|
53
|
-
self.class.unlock(self.key)
|
54
|
-
end
|
55
|
-
end
|
56
|
-
end
|
57
|
-
end
|
1
|
+
module ActiveJob
|
2
|
+
module Locking
|
3
|
+
module Adapters
|
4
|
+
class Memory < Base
|
5
|
+
@hash = Hash.new
|
6
|
+
@mutex = Mutex.new
|
7
|
+
|
8
|
+
def self.lock(key)
|
9
|
+
@mutex.synchronize do
|
10
|
+
if @hash[key]
|
11
|
+
false
|
12
|
+
else
|
13
|
+
@hash[key] = Time.now
|
14
|
+
end
|
15
|
+
end
|
16
|
+
end
|
17
|
+
|
18
|
+
def self.unlock(key)
|
19
|
+
@mutex.synchronize do
|
20
|
+
@hash.delete(key)
|
21
|
+
end
|
22
|
+
end
|
23
|
+
|
24
|
+
def self.locked?(key)
|
25
|
+
@mutex.synchronize do
|
26
|
+
@hash.include?(key)
|
27
|
+
end
|
28
|
+
end
|
29
|
+
|
30
|
+
def self.reset
|
31
|
+
@mutex.synchronize do
|
32
|
+
@hash = Hash.new
|
33
|
+
end
|
34
|
+
end
|
35
|
+
|
36
|
+
def create_lock_manager
|
37
|
+
end
|
38
|
+
|
39
|
+
def lock
|
40
|
+
finish = Time.now + self.options.lock_acquire_time
|
41
|
+
sleep_time = [5, self.options.lock_acquire_time / 5].min
|
42
|
+
|
43
|
+
begin
|
44
|
+
lock = self.class.lock(key)
|
45
|
+
return lock if lock
|
46
|
+
sleep(sleep_time)
|
47
|
+
end while Time.now < finish
|
48
|
+
|
49
|
+
false
|
50
|
+
end
|
51
|
+
|
52
|
+
def unlock
|
53
|
+
self.class.unlock(self.key)
|
54
|
+
end
|
55
|
+
end
|
56
|
+
end
|
57
|
+
end
|
58
58
|
end
|
@@ -1,26 +1,26 @@
|
|
1
|
-
require 'redis-semaphore'
|
2
|
-
|
3
|
-
module ActiveJob
|
4
|
-
module Locking
|
5
|
-
module Adapters
|
6
|
-
class RedisSemaphore < Base
|
7
|
-
def create_lock_manager
|
8
|
-
mapped_options = {host: self.options.hosts.first,
|
9
|
-
resources: 1,
|
10
|
-
stale_client_timeout: self.options.lock_time}.merge(self.options.adapter_options)
|
11
|
-
|
12
|
-
Redis::Semaphore.new(self.key, mapped_options)
|
13
|
-
end
|
14
|
-
|
15
|
-
def lock
|
16
|
-
self.lock_token = self.lock_manager.lock(self.options.lock_acquire_time)
|
17
|
-
end
|
18
|
-
|
19
|
-
def unlock
|
20
|
-
self.lock_manager.signal(self.lock_token)
|
21
|
-
self.lock_token = nil
|
22
|
-
end
|
23
|
-
end
|
24
|
-
end
|
25
|
-
end
|
26
|
-
end
|
1
|
+
require 'redis-semaphore'
|
2
|
+
|
3
|
+
module ActiveJob
|
4
|
+
module Locking
|
5
|
+
module Adapters
|
6
|
+
class RedisSemaphore < Base
|
7
|
+
def create_lock_manager
|
8
|
+
mapped_options = {host: self.options.hosts.first,
|
9
|
+
resources: 1,
|
10
|
+
stale_client_timeout: self.options.lock_time}.merge(self.options.adapter_options)
|
11
|
+
|
12
|
+
Redis::Semaphore.new(self.key, mapped_options)
|
13
|
+
end
|
14
|
+
|
15
|
+
def lock
|
16
|
+
self.lock_token = self.lock_manager.lock(self.options.lock_acquire_time)
|
17
|
+
end
|
18
|
+
|
19
|
+
def unlock
|
20
|
+
self.lock_manager.signal(self.lock_token)
|
21
|
+
self.lock_token = nil
|
22
|
+
end
|
23
|
+
end
|
24
|
+
end
|
25
|
+
end
|
26
|
+
end
|
@@ -1,26 +1,26 @@
|
|
1
|
-
require 'redlock'
|
2
|
-
|
3
|
-
module ActiveJob
|
4
|
-
module Locking
|
5
|
-
module Adapters
|
6
|
-
class Redlock < Base
|
7
|
-
def create_lock_manager
|
8
|
-
mapped_options = self.options.adapter_options
|
9
|
-
mapped_options[:retry_count] = 2 # Try to get the lock and then try again when timeout is expiring--
|
10
|
-
mapped_options[:retry_delay] = self.options.lock_acquire_time * 1000 # convert from seconds to milliseconds
|
11
|
-
|
12
|
-
::Redlock::Client.new(self.options.hosts, mapped_options)
|
13
|
-
end
|
14
|
-
|
15
|
-
def lock
|
16
|
-
self.lock_token = self.lock_manager.lock(self.key, self.options.lock_time * 1000)
|
17
|
-
end
|
18
|
-
|
19
|
-
def unlock
|
20
|
-
self.lock_manager.unlock(self.lock_token)
|
21
|
-
self.lock_token = nil
|
22
|
-
end
|
23
|
-
end
|
24
|
-
end
|
25
|
-
end
|
26
|
-
end
|
1
|
+
require 'redlock'
|
2
|
+
|
3
|
+
module ActiveJob
|
4
|
+
module Locking
|
5
|
+
module Adapters
|
6
|
+
class Redlock < Base
|
7
|
+
def create_lock_manager
|
8
|
+
mapped_options = self.options.adapter_options
|
9
|
+
mapped_options[:retry_count] = 2 # Try to get the lock and then try again when timeout is expiring--
|
10
|
+
mapped_options[:retry_delay] = self.options.lock_acquire_time * 1000 # convert from seconds to milliseconds
|
11
|
+
|
12
|
+
::Redlock::Client.new(self.options.hosts, mapped_options)
|
13
|
+
end
|
14
|
+
|
15
|
+
def lock
|
16
|
+
self.lock_token = self.lock_manager.lock(self.key, self.options.lock_time * 1000)
|
17
|
+
end
|
18
|
+
|
19
|
+
def unlock
|
20
|
+
self.lock_manager.unlock(self.lock_token)
|
21
|
+
self.lock_token = nil
|
22
|
+
end
|
23
|
+
end
|
24
|
+
end
|
25
|
+
end
|
26
|
+
end
|
@@ -1,25 +1,25 @@
|
|
1
|
-
require 'suo'
|
2
|
-
|
3
|
-
module ActiveJob
|
4
|
-
module Locking
|
5
|
-
module Adapters
|
6
|
-
class SuoRedis < Base
|
7
|
-
def create_lock_manager
|
8
|
-
mapped_options = {connection: {host: self.options.hosts.first},
|
9
|
-
stale_lock_expiration: self.options.lock_time,
|
10
|
-
acquisition_timeout: self.options.lock_acquire_time}
|
11
|
-
|
12
|
-
Suo::Client::Redis.new(self.key, mapped_options)
|
13
|
-
end
|
14
|
-
|
15
|
-
def lock
|
16
|
-
self.lock_token = self.lock_manager.lock
|
17
|
-
end
|
18
|
-
|
19
|
-
def unlock
|
20
|
-
self.lock_manager.unlock(self.lock_token)
|
21
|
-
end
|
22
|
-
end
|
23
|
-
end
|
24
|
-
end
|
25
|
-
end
|
1
|
+
require 'suo'
|
2
|
+
|
3
|
+
module ActiveJob
|
4
|
+
module Locking
|
5
|
+
module Adapters
|
6
|
+
class SuoRedis < Base
|
7
|
+
def create_lock_manager
|
8
|
+
mapped_options = {connection: {host: self.options.hosts.first},
|
9
|
+
stale_lock_expiration: self.options.lock_time,
|
10
|
+
acquisition_timeout: self.options.lock_acquire_time}
|
11
|
+
|
12
|
+
Suo::Client::Redis.new(self.key, mapped_options)
|
13
|
+
end
|
14
|
+
|
15
|
+
def lock
|
16
|
+
self.lock_token = self.lock_manager.lock
|
17
|
+
end
|
18
|
+
|
19
|
+
def unlock
|
20
|
+
self.lock_manager.unlock(self.lock_token)
|
21
|
+
end
|
22
|
+
end
|
23
|
+
end
|
24
|
+
end
|
25
|
+
end
|
@@ -7,8 +7,8 @@ module ActiveJob
|
|
7
7
|
def lock_options
|
8
8
|
@lock_options ||= ActiveJob::Locking::Options.new
|
9
9
|
end
|
10
|
-
delegate :adapter, :hosts, :lock_time, :lock_acquire_time, :adapter_options, to: :lock_options
|
11
|
-
delegate :adapter=, :hosts=, :lock_time=, :lock_acquire_time=, :adapter_options=, to: :lock_options
|
10
|
+
delegate :adapter, :enqueue_time, :hosts, :lock_time, :lock_acquire_time, :adapter_options, to: :lock_options
|
11
|
+
delegate :adapter=, :enqueue_time=, :hosts=, :lock_time=, :lock_acquire_time=, :adapter_options=, to: :lock_options
|
12
12
|
end
|
13
13
|
|
14
14
|
included do
|
@@ -7,16 +7,31 @@ module ActiveJob
|
|
7
7
|
attr_accessor :hosts
|
8
8
|
attr_accessor :lock_time
|
9
9
|
attr_accessor :lock_acquire_time
|
10
|
+
attr_accessor :enqueue_time
|
10
11
|
attr_accessor :adapter_options
|
11
12
|
|
12
13
|
def initialize(options = {})
|
13
14
|
@adapter = options[:adapter]
|
14
15
|
@hosts = options[:hosts]
|
16
|
+
@enqueue_time = options[:enqueue_time]
|
15
17
|
@lock_time = options[:lock_time]
|
16
18
|
@lock_acquire_time = options[:lock_acquire_time]
|
17
19
|
@adapter_options = options[:adapter_options]
|
18
20
|
end
|
19
21
|
|
22
|
+
def enqueue_time=(value)
|
23
|
+
case value
|
24
|
+
when NilClass
|
25
|
+
raise(ArgumentError, 'Enqueue time must be set')
|
26
|
+
when ActiveSupport::Duration
|
27
|
+
@enqueue_time = value.value
|
28
|
+
when 0
|
29
|
+
raise(ArgumentError, 'Enqueue time must be greater than zero')
|
30
|
+
else
|
31
|
+
@enqueue_time = value
|
32
|
+
end
|
33
|
+
end
|
34
|
+
|
20
35
|
def lock_time=(value)
|
21
36
|
case value
|
22
37
|
when NilClass
|
@@ -47,6 +62,7 @@ module ActiveJob
|
|
47
62
|
result = self.dup
|
48
63
|
result.adapter = other.adapter if other.adapter
|
49
64
|
result.hosts = other.hosts if other.hosts
|
65
|
+
result.enqueue_time = other.enqueue_time if other.enqueue_time
|
50
66
|
result.lock_time = other.lock_time if other.lock_time
|
51
67
|
result.lock_acquire_time = other.lock_acquire_time if other.lock_acquire_time
|
52
68
|
result.adapter_options = other.adapter_options if other.adapter_options
|
data/test/jobs/fail_job.rb
CHANGED
@@ -1,15 +1,15 @@
|
|
1
|
-
class FailJob < ActiveJob::Base
|
2
|
-
include ActiveJob::Locking::Unique
|
3
|
-
|
4
|
-
self.lock_acquire_time = 2
|
5
|
-
|
6
|
-
# We want the job ids to be all the same for testing
|
7
|
-
def lock_key(index, sleep_time)
|
8
|
-
self.class.name
|
9
|
-
end
|
10
|
-
|
11
|
-
# Pass in index so we can distinguish different jobs
|
12
|
-
def perform(index, sleep_time)
|
13
|
-
raise(ArgumentError, 'Job failed')
|
14
|
-
end
|
1
|
+
class FailJob < ActiveJob::Base
|
2
|
+
include ActiveJob::Locking::Unique
|
3
|
+
|
4
|
+
self.lock_acquire_time = 2
|
5
|
+
|
6
|
+
# We want the job ids to be all the same for testing
|
7
|
+
def lock_key(index, sleep_time)
|
8
|
+
self.class.name
|
9
|
+
end
|
10
|
+
|
11
|
+
# Pass in index so we can distinguish different jobs
|
12
|
+
def perform(index, sleep_time)
|
13
|
+
raise(ArgumentError, 'Job failed')
|
14
|
+
end
|
15
15
|
end
|
data/test/jobs/serial_job.rb
CHANGED
@@ -1,15 +1,15 @@
|
|
1
|
-
class SerialJob < ActiveJob::Base
|
2
|
-
include ActiveJob::Locking::Serialized
|
3
|
-
|
4
|
-
self.lock_acquire_time = 2
|
5
|
-
|
6
|
-
# We want the job ids to be all the same for testing
|
7
|
-
def lock_key(index, sleep_time)
|
8
|
-
self.class.name
|
9
|
-
end
|
10
|
-
|
11
|
-
# Pass in index so we can distinguish different jobs
|
12
|
-
def perform(index, sleep_time)
|
13
|
-
sleep(sleep_time)
|
14
|
-
end
|
1
|
+
class SerialJob < ActiveJob::Base
|
2
|
+
include ActiveJob::Locking::Serialized
|
3
|
+
|
4
|
+
self.lock_acquire_time = 2
|
5
|
+
|
6
|
+
# We want the job ids to be all the same for testing
|
7
|
+
def lock_key(index, sleep_time)
|
8
|
+
self.class.name
|
9
|
+
end
|
10
|
+
|
11
|
+
# Pass in index so we can distinguish different jobs
|
12
|
+
def perform(index, sleep_time)
|
13
|
+
sleep(sleep_time)
|
14
|
+
end
|
15
15
|
end
|
data/test/jobs/unique_job.rb
CHANGED
@@ -1,15 +1,15 @@
|
|
1
|
-
class UniqueJob < ActiveJob::Base
|
2
|
-
include ActiveJob::Locking::Unique
|
3
|
-
|
4
|
-
self.lock_acquire_time = 2
|
5
|
-
|
6
|
-
# We want the job ids to be all the same for testing
|
7
|
-
def lock_key(index, sleep_time)
|
8
|
-
self.class.name
|
9
|
-
end
|
10
|
-
|
11
|
-
# Pass in index so we can distinguish different jobs
|
12
|
-
def perform(index, sleep_time)
|
13
|
-
sleep(sleep_time)
|
14
|
-
end
|
1
|
+
class UniqueJob < ActiveJob::Base
|
2
|
+
include ActiveJob::Locking::Unique
|
3
|
+
|
4
|
+
self.lock_acquire_time = 2
|
5
|
+
|
6
|
+
# We want the job ids to be all the same for testing
|
7
|
+
def lock_key(index, sleep_time)
|
8
|
+
self.class.name
|
9
|
+
end
|
10
|
+
|
11
|
+
# Pass in index so we can distinguish different jobs
|
12
|
+
def perform(index, sleep_time)
|
13
|
+
sleep(sleep_time)
|
14
|
+
end
|
15
15
|
end
|
data/test/test_suite.rb
CHANGED
@@ -1,12 +1,12 @@
|
|
1
|
-
%w(
|
2
|
-
test_serialized_memory
|
3
|
-
test_serialized_redis_semaphore
|
4
|
-
test_serialized_redlock
|
5
|
-
test_serialized_suo_redis
|
6
|
-
test_unique_memory
|
7
|
-
test_unique_redis_semaphore
|
8
|
-
test_unique_redlock
|
9
|
-
test_unique_suo_redis
|
10
|
-
).each do |test|
|
11
|
-
require File.expand_path("../#{test}", __FILE__)
|
12
|
-
end
|
1
|
+
%w(
|
2
|
+
test_serialized_memory
|
3
|
+
test_serialized_redis_semaphore
|
4
|
+
test_serialized_redlock
|
5
|
+
test_serialized_suo_redis
|
6
|
+
test_unique_memory
|
7
|
+
test_unique_redis_semaphore
|
8
|
+
test_unique_redlock
|
9
|
+
test_unique_suo_redis
|
10
|
+
).each do |test|
|
11
|
+
require File.expand_path("../#{test}", __FILE__)
|
12
|
+
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: activejob-locking
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.5.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Charlie Savage
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2017-
|
11
|
+
date: 2017-10-03 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activejob
|
@@ -154,7 +154,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
154
154
|
version: '0'
|
155
155
|
requirements: []
|
156
156
|
rubyforge_project:
|
157
|
-
rubygems_version: 2.6.
|
157
|
+
rubygems_version: 2.6.13
|
158
158
|
signing_key:
|
159
159
|
specification_version: 4
|
160
160
|
summary: ActiveJob locking to control how jobs are enqueued and performed.
|