activejob-uniqueness 0.1.4 → 0.2.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +12 -1
- data/README.md +93 -21
- data/lib/active_job/uniqueness.rb +2 -11
- data/lib/active_job/uniqueness/configuration.rb +7 -3
- data/lib/active_job/uniqueness/sidekiq_patch.rb +3 -0
- data/lib/active_job/uniqueness/version.rb +1 -1
- data/lib/generators/active_job/uniqueness/templates/config/initializers/active_job_uniqueness.rb +1 -1
- metadata +5 -5
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: a407572b8a6542f0303eecdedf53a58af589bef195ca15400880d417e510feb7
|
4
|
+
data.tar.gz: 65bd030916284341db0f3a0bde6c72a223664c05ed1b92a7637a8a7c436851d5
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 0e7adcf507b7e9f53ec3ed53a841ea460a24bd3851e6e46697c5bb2776f3a6c3aea0f54fbcd8669e57c36e6d6470a5edbacd9495ca315505690e6583d40ad2b5
|
7
|
+
data.tar.gz: 57074e6a7161fcaf5706598dd1d13fcaf878f70d3345344e1cc5b3d120928079a734608df3f4f5355545cf937864f31bc4b031d04d54ab191aa19601e93fd2ea
|
data/CHANGELOG.md
CHANGED
@@ -3,7 +3,18 @@ All notable changes to this project will be documented in this file.
|
|
3
3
|
|
4
4
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
|
5
5
|
|
6
|
-
## [Unreleased](https://github.com/veeqo/activejob-uniqueness/compare/v0.
|
6
|
+
## [Unreleased](https://github.com/veeqo/activejob-uniqueness/compare/v0.2.0...HEAD)
|
7
|
+
|
8
|
+
## [0.2.0](https://github.com/veeqo/activejob-uniqueness/compare/v0.1.4...v0.2.0) - 2021-05-09
|
9
|
+
|
10
|
+
### Added
|
11
|
+
- [#22](https://github.com/veeqo/activejob-uniqueness/pull/22) Test with ruby 3.0.1
|
12
|
+
|
13
|
+
### Changed
|
14
|
+
- [#20](https://github.com/veeqo/activejob-uniqueness/pull/20) **Breaking** Sidekiq patch is not applied automatically anymore
|
15
|
+
- [#21](https://github.com/veeqo/activejob-uniqueness/pull/21) Migrate from Travis to Github Actions
|
16
|
+
- [#24](https://github.com/veeqo/activejob-uniqueness/pull/24) The default value for `retry_count` of redlock is now 0
|
17
|
+
- Require ruby 2.5+
|
7
18
|
|
8
19
|
## [0.1.4](https://github.com/veeqo/activejob-uniqueness/compare/v0.1.3...v0.1.4) - 2020-09-22
|
9
20
|
|
data/README.md
CHANGED
@@ -1,5 +1,5 @@
|
|
1
1
|
# Job uniqueness for ActiveJob
|
2
|
-
[![Build Status](https://
|
2
|
+
[![Build Status](https://github.com/veeqo/activejob-uniqueness/actions/workflows/main.yml/badge.svg?branch=main)](https://github.com/veeqo/activejob-uniqueness/actions/workflows/main.yml) [![Gem Version](https://badge.fury.io/rb/activejob-uniqueness.svg)](https://badge.fury.io/rb/activejob-uniqueness)
|
3
3
|
|
4
4
|
The gem allows to protect job uniqueness with next strategies:
|
5
5
|
|
@@ -8,10 +8,16 @@ The gem allows to protect job uniqueness with next strategies:
|
|
8
8
|
| `until_executing` | when **pushed** to the queue | when **processing starts** |
|
9
9
|
| `until_executed` | when **pushed** to the queue | when the job is **processed successfully** |
|
10
10
|
| `until_expired` | when **pushed** to the queue | when the lock is **expired** |
|
11
|
-
| `until_and_while_executing` | when **pushed** to the queue | when **processing starts**<br>a runtime lock is acquired to **prevent simultaneous jobs
|
11
|
+
| `until_and_while_executing` | when **pushed** to the queue | when **processing starts**<br>a runtime lock is acquired to **prevent simultaneous jobs**<br>*has extra options: `runtime_lock_ttl`, `on_runtime_conflict`* |
|
12
12
|
| `while_executing` | when **processing starts** | when the job is **processed**<br>with any result including an error |
|
13
13
|
|
14
|
-
Inspired by [SidekiqUniqueJobs](https://github.com/mhenrixon/sidekiq-unique-jobs), uses [Redlock](https://github.com/leandromoreira/redlock-rb) under the hood
|
14
|
+
Inspired by [SidekiqUniqueJobs](https://github.com/mhenrixon/sidekiq-unique-jobs), uses [Redlock](https://github.com/leandromoreira/redlock-rb) under the hood.
|
15
|
+
|
16
|
+
<p align="center">
|
17
|
+
<a href="https://www.veeqo.com/" title="Sponsored by Veeqo">
|
18
|
+
<img src="https://static.veeqo.com/assets/sponsored_by_veeqo.png" width="360" />
|
19
|
+
</a>
|
20
|
+
</p>
|
15
21
|
|
16
22
|
## Installation
|
17
23
|
|
@@ -21,6 +27,11 @@ Add the `activejob-uniqueness` gem to your Gemfile.
|
|
21
27
|
gem 'activejob-uniqueness'
|
22
28
|
```
|
23
29
|
|
30
|
+
If you want jobs unlocking for Sidekiq Web UI, require the patch explicitly. [**Queues cleanup becomes slower!**](#sidekiq-api-support)
|
31
|
+
```ruby
|
32
|
+
gem 'activejob-uniqueness', require: 'active_job/uniqueness/sidekiq_patch'
|
33
|
+
```
|
34
|
+
|
24
35
|
And run `bundle install` command.
|
25
36
|
|
26
37
|
## Configuration
|
@@ -34,27 +45,83 @@ rails generate active_job:uniqueness:install
|
|
34
45
|
|
35
46
|
## Usage
|
36
47
|
|
37
|
-
|
48
|
+
|
49
|
+
### Make the job to be unique
|
38
50
|
|
39
51
|
```ruby
|
40
52
|
class MyJob < ActiveJob::Base
|
53
|
+
# new jobs with the same args will raise error until existing one is executed
|
41
54
|
unique :until_executed
|
42
55
|
|
43
|
-
|
44
|
-
|
56
|
+
def perform(args)
|
57
|
+
# work
|
58
|
+
end
|
59
|
+
end
|
60
|
+
```
|
61
|
+
|
62
|
+
### Tune uniqueness settings per job
|
63
|
+
|
64
|
+
```ruby
|
65
|
+
class MyJob < ActiveJob::Base
|
66
|
+
# new jobs with the same args will be logged within 3 hours or until existing one is being executing
|
67
|
+
unique :until_executing, lock_ttl: 3.hours, on_conflict: :log
|
68
|
+
|
69
|
+
def perform(args)
|
70
|
+
# work
|
71
|
+
end
|
72
|
+
end
|
73
|
+
```
|
74
|
+
|
75
|
+
You can set defaults globally with [the configuration](#configuration)
|
45
76
|
|
46
|
-
|
47
|
-
# unique :until_executed, on_conflict: :log
|
77
|
+
### Control lock conflicts
|
48
78
|
|
49
|
-
|
50
|
-
|
79
|
+
```ruby
|
80
|
+
class MyJob < ActiveJob::Base
|
81
|
+
# Proc gets the job instance including its arguments
|
82
|
+
unique :until_executing, on_conflict: ->(job) { job.logger.info "Oops: #{job.arguments}" }
|
51
83
|
|
52
|
-
|
53
|
-
|
84
|
+
def perform(args)
|
85
|
+
# work
|
86
|
+
end
|
54
87
|
end
|
55
88
|
```
|
56
89
|
|
57
|
-
|
90
|
+
### Control lock key arguments
|
91
|
+
|
92
|
+
```ruby
|
93
|
+
class MyJob < ActiveJob::Base
|
94
|
+
unique :until_executed
|
95
|
+
|
96
|
+
def perform(foo, bar, baz)
|
97
|
+
# work
|
98
|
+
end
|
99
|
+
|
100
|
+
def lock_key_arguments
|
101
|
+
arguments.first(2) # baz is ignored
|
102
|
+
end
|
103
|
+
end
|
104
|
+
```
|
105
|
+
|
106
|
+
### Control the lock key
|
107
|
+
|
108
|
+
```ruby
|
109
|
+
class MyJob < ActiveJob::Base
|
110
|
+
unique :until_executed
|
111
|
+
|
112
|
+
def perform(foo, bar, baz)
|
113
|
+
# work
|
114
|
+
end
|
115
|
+
|
116
|
+
def lock_key
|
117
|
+
'qux' # completely custom lock key
|
118
|
+
end
|
119
|
+
end
|
120
|
+
```
|
121
|
+
|
122
|
+
### Unlock jobs manually
|
123
|
+
|
124
|
+
The selected strategy automatically unlocks jobs, but in some cases (e.g. the queue is purged) it is handy to unlock jobs manually.
|
58
125
|
|
59
126
|
```ruby
|
60
127
|
# Remove the lock for particular arguments:
|
@@ -91,7 +158,7 @@ ActiveJob::Uniqueness instruments `ActiveSupport::Notifications` with next event
|
|
91
158
|
|
92
159
|
And then writes to `ActiveJob::Base.logger`.
|
93
160
|
|
94
|
-
|
161
|
+
**ActiveJob prior to version `6.1` will always log `Enqueued MyJob (Job ID) ...` even if the callback chain is halted. [Details](https://github.com/rails/rails/pull/37830)**
|
95
162
|
|
96
163
|
## Testing
|
97
164
|
|
@@ -107,15 +174,14 @@ bundle
|
|
107
174
|
rake
|
108
175
|
```
|
109
176
|
|
110
|
-
|
111
|
-
```sh
|
112
|
-
gem install wwtd
|
113
|
-
wwtd
|
114
|
-
```
|
177
|
+
## Sidekiq API support
|
115
178
|
|
116
|
-
|
179
|
+
ActiveJob::Uniqueness supports Sidekiq API to unset job locks on queues cleanup (e.g. via Sidekiq Web UI). Starting Sidekiq 5.1 job death also triggers locks cleanup.
|
180
|
+
Take into account that **[big queues clanup becomes much slower](https://github.com/veeqo/activejob-uniqueness/issues/16)** because each job is being unlocked individually. In order to activate Sidekiq API patch require it explicitly in your Gemfile:
|
117
181
|
|
118
|
-
|
182
|
+
```ruby
|
183
|
+
gem 'activejob-uniqueness', require: 'active_job/uniqueness/sidekiq_patch'
|
184
|
+
```
|
119
185
|
|
120
186
|
## Contributing
|
121
187
|
|
@@ -124,3 +190,9 @@ Bug reports and pull requests are welcome on GitHub at https://github.com/veeqo/
|
|
124
190
|
## License
|
125
191
|
|
126
192
|
The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
|
193
|
+
|
194
|
+
## About [Veeqo](https://www.veeqo.com)
|
195
|
+
|
196
|
+
At Veeqo, our team of Engineers is on a mission to create a world-class Inventory and Shipping platform, built to the highest standards in best coding practices. We are a growing team, looking for other passionate developers to [join us](https://veeqo-ltd.breezy.hr/) on our journey. If you're looking for a career working for one of the most exciting tech companies in ecommerce, we want to hear from you.
|
197
|
+
|
198
|
+
[Veeqo developers blog](https://devs.veeqo.com)
|
@@ -1,22 +1,13 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
|
-
require '
|
4
|
-
require 'active_support/core_ext'
|
3
|
+
require 'active_job'
|
5
4
|
require 'redlock'
|
6
|
-
|
5
|
+
|
7
6
|
require 'active_job/uniqueness/version'
|
8
7
|
require 'active_job/uniqueness/errors'
|
9
8
|
require 'active_job/uniqueness/log_subscriber'
|
10
9
|
require 'active_job/uniqueness/active_job_patch'
|
11
10
|
|
12
|
-
begin
|
13
|
-
require 'sidekiq/api'
|
14
|
-
rescue LoadError
|
15
|
-
# ¯\_(ツ)_/¯
|
16
|
-
else
|
17
|
-
require 'active_job/uniqueness/sidekiq_patch'
|
18
|
-
end
|
19
|
-
|
20
11
|
module ActiveJob
|
21
12
|
module Uniqueness
|
22
13
|
extend ActiveSupport::Autoload
|
@@ -11,14 +11,18 @@ module ActiveJob
|
|
11
11
|
class Configuration
|
12
12
|
include ActiveSupport::Configurable
|
13
13
|
|
14
|
-
config_accessor(:lock_ttl) { 1.day
|
14
|
+
config_accessor(:lock_ttl) { 86_400 } # 1.day
|
15
15
|
config_accessor(:lock_prefix) { 'activejob_uniqueness' }
|
16
16
|
config_accessor(:on_conflict) { :raise }
|
17
|
-
config_accessor(:digest_method) { OpenSSL::Digest::MD5 }
|
18
17
|
config_accessor(:redlock_servers) { [ENV.fetch('REDIS_URL', 'redis://localhost:6379')] }
|
19
|
-
config_accessor(:redlock_options) { {} }
|
18
|
+
config_accessor(:redlock_options) { { retry_count: 0 } }
|
20
19
|
config_accessor(:lock_strategies) { {} }
|
21
20
|
|
21
|
+
config_accessor(:digest_method) do
|
22
|
+
require 'openssl'
|
23
|
+
OpenSSL::Digest::MD5
|
24
|
+
end
|
25
|
+
|
22
26
|
def on_conflict=(action)
|
23
27
|
validate_on_conflict_action!(action)
|
24
28
|
|
data/lib/generators/active_job/uniqueness/templates/config/initializers/active_job_uniqueness.rb
CHANGED
@@ -31,7 +31,7 @@ ActiveJob::Uniqueness.configure do |config|
|
|
31
31
|
# Custom options for Redlock.
|
32
32
|
# Read more at https://github.com/leandromoreira/redlock-rb#redlock-configuration
|
33
33
|
#
|
34
|
-
# config.redlock_options = {}
|
34
|
+
# config.redlock_options = { retry_count: 0 }
|
35
35
|
|
36
36
|
# Custom strategies.
|
37
37
|
# config.lock_strategies = { my_strategy: MyStrategy }
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: activejob-uniqueness
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.2.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Rustam Sharshenov
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2021-05-09 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activejob
|
@@ -142,7 +142,7 @@ licenses:
|
|
142
142
|
metadata:
|
143
143
|
homepage_uri: https://github.com/veeqo/activejob-uniqueness
|
144
144
|
source_code_uri: https://github.com/veeqo/activejob-uniqueness
|
145
|
-
changelog_uri: https://github.com/veeqo/activejob-uniqueness/blob/
|
145
|
+
changelog_uri: https://github.com/veeqo/activejob-uniqueness/blob/main/CHANGELOG.md
|
146
146
|
post_install_message:
|
147
147
|
rdoc_options: []
|
148
148
|
require_paths:
|
@@ -151,14 +151,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
151
151
|
requirements:
|
152
152
|
- - ">="
|
153
153
|
- !ruby/object:Gem::Version
|
154
|
-
version: '
|
154
|
+
version: '2.5'
|
155
155
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
156
156
|
requirements:
|
157
157
|
- - ">="
|
158
158
|
- !ruby/object:Gem::Version
|
159
159
|
version: '0'
|
160
160
|
requirements: []
|
161
|
-
rubygems_version: 3.
|
161
|
+
rubygems_version: 3.2.15
|
162
162
|
signing_key:
|
163
163
|
specification_version: 4
|
164
164
|
summary: Ensure uniqueness of your ActiveJob jobs
|