sidekiq-unique-jobs 7.0.1 → 7.0.6
Sign up to get free protection for your applications and to get access to all the features.
Potentially problematic release.
This version of sidekiq-unique-jobs might be problematic. Click here for more details.
- checksums.yaml +4 -4
- data/CHANGELOG.md +118 -37
- data/README.md +33 -19
- data/lib/sidekiq_unique_jobs.rb +3 -1
- data/lib/sidekiq_unique_jobs/batch_delete.rb +1 -1
- data/lib/sidekiq_unique_jobs/json.rb +7 -1
- data/lib/sidekiq_unique_jobs/lock.rb +31 -1
- data/lib/sidekiq_unique_jobs/lua/lock.lua +9 -9
- data/lib/sidekiq_unique_jobs/lua/shared/_find_digest_in_process_set.lua +2 -2
- data/lib/sidekiq_unique_jobs/orphans/manager.rb +13 -3
- data/lib/sidekiq_unique_jobs/orphans/ruby_reaper.rb +22 -8
- data/lib/sidekiq_unique_jobs/redis/entity.rb +7 -1
- data/lib/sidekiq_unique_jobs/sidekiq_unique_jobs.rb +1 -1
- data/lib/sidekiq_unique_jobs/timer_task.rb +78 -0
- data/lib/sidekiq_unique_jobs/version.rb +1 -1
- metadata +6 -5
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 15ac11b6577e3aec68065e79d9d5b61d67b88e958c23097b04409869e4a6c18f
|
|
4
|
+
data.tar.gz: d6d8f8244abf8b4dacfc28a313b94d89edee88002052618f018ea81fc29cb32a
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: dfc16d255733a016d93b1a5699f740732c4e2095ecbe20e9562512c4a2ab0dd51c221a681b7ee9e6ccdb9ce9b90a1dcd2a8902d26089658b3936f2e0667ca1b3
|
|
7
|
+
data.tar.gz: dd52aa2aa2ec53260e8ba8569827bc825e926e3245585731e17130693ef588c8482ad0539f74fa6ce43436d6476b38cf8e0303990b6689783b2ce6277f05f1c4
|
data/CHANGELOG.md
CHANGED
|
@@ -2,32 +2,104 @@
|
|
|
2
2
|
|
|
3
3
|
## [Unreleased](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/HEAD)
|
|
4
4
|
|
|
5
|
-
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.
|
|
5
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.5...HEAD)
|
|
6
6
|
|
|
7
|
+
**Fixed bugs:**
|
|
7
8
|
|
|
9
|
+
- Deprecation warning for redis behaviour change in 5.0 [\#579](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/579)
|
|
8
10
|
|
|
9
|
-
## [v7.0.
|
|
11
|
+
## [v7.0.5](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.5) (2021-03-18)
|
|
12
|
+
|
|
13
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.4...v7.0.5)
|
|
14
|
+
|
|
15
|
+
**Implemented enhancements:**
|
|
16
|
+
|
|
17
|
+
- Improve compatibility with redis-namespace [\#581](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/581) ([Wolfer](https://github.com/Wolfer))
|
|
18
|
+
|
|
19
|
+
**Fixed bugs:**
|
|
20
|
+
|
|
21
|
+
- RubyReaper treats runtime lock as orphan and delete it [\#580](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/580)
|
|
22
|
+
- Prefer conn.exists? when possible [\#583](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/583) ([mhenrixon](https://github.com/mhenrixon))
|
|
23
|
+
- Don't reap :RUN keys when active [\#582](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/582) ([mhenrixon](https://github.com/mhenrixon))
|
|
24
|
+
|
|
25
|
+
**Closed issues:**
|
|
26
|
+
|
|
27
|
+
- redis-namespace asks to use admistrative commands directly [\#578](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/578)
|
|
28
|
+
|
|
29
|
+
## [v7.0.4](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.4) (2021-02-17)
|
|
30
|
+
|
|
31
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.3...v7.0.4)
|
|
32
|
+
|
|
33
|
+
**Fixed bugs:**
|
|
34
|
+
|
|
35
|
+
- Fix uninitialized scheduled task [\#577](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/577) ([ArturT](https://github.com/ArturT))
|
|
36
|
+
|
|
37
|
+
## [v7.0.3](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.3) (2021-02-17)
|
|
38
|
+
|
|
39
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.2...v7.0.3)
|
|
40
|
+
|
|
41
|
+
**Fixed bugs:**
|
|
42
|
+
|
|
43
|
+
- Reduce reaper threads [\#576](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/576) ([mhenrixon](https://github.com/mhenrixon))
|
|
44
|
+
|
|
45
|
+
**Merged pull requests:**
|
|
46
|
+
|
|
47
|
+
- Fix typo in README.md \[ci skip\] [\#575](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/575) ([yujideveloper](https://github.com/yujideveloper))
|
|
48
|
+
|
|
49
|
+
## [v7.0.2](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.2) (2021-02-08)
|
|
50
|
+
|
|
51
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.1...v7.0.2)
|
|
10
52
|
|
|
11
|
-
|
|
53
|
+
**Fixed bugs:**
|
|
54
|
+
|
|
55
|
+
- Lock not getting properly cleared for some jobs [\#560](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/560)
|
|
56
|
+
- while\_executing + raise let non-unique jobs to be executed [\#534](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/534)
|
|
57
|
+
- delete\_by\_digest does not work with the msg\['unique\_digest'\] value available in sidekiq\_retries\_exhausted [\#532](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/532)
|
|
58
|
+
- Multiple jobs running at the same time [\#531](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/531)
|
|
59
|
+
- Unable to setup in standalone Ruby project [\#523](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/523)
|
|
60
|
+
- v7.0.0.beta15 Can't push new jobs to queue [\#501](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/501)
|
|
61
|
+
- Reaper doesn't work - lua or ruby [\#498](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/498)
|
|
62
|
+
- Tasks run once, and then there is no launch [\#464](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/464)
|
|
63
|
+
- Jobs executing and immediately returning [\#418](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/418)
|
|
64
|
+
- until\_and\_while\_executing + sidekiq retry mechanism [\#395](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/395)
|
|
65
|
+
- Failed jobs waiting to be retried are not considered when fetching uniqueness [\#394](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/394)
|
|
66
|
+
- Fix that :PRIMED keys are seemingly not removed [\#574](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/574) ([mhenrixon](https://github.com/mhenrixon))
|
|
67
|
+
|
|
68
|
+
**Closed issues:**
|
|
69
|
+
|
|
70
|
+
- Just some clarification on the documentation. [\#530](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/530)
|
|
71
|
+
- Unique Digests dashboard not filtering the full set of results [\#529](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/529)
|
|
72
|
+
- after\_unlock isn't called unless it's a class method [\#526](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/526)
|
|
73
|
+
- The job gets JID and goes to dead right away [\#522](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/522)
|
|
74
|
+
- Able to assign customise Redis setup [\#509](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/509)
|
|
75
|
+
- SidekiqUniqueJobs::UniqueArgs\#create\_digest is getting called twice [\#391](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/391)
|
|
76
|
+
|
|
77
|
+
**Merged pull requests:**
|
|
78
|
+
|
|
79
|
+
- Fix example url in documentation [\#572](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/572) ([yboulkaid](https://github.com/yboulkaid))
|
|
80
|
+
|
|
81
|
+
## [v7.0.1](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.1) (2021-01-22)
|
|
82
|
+
|
|
83
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0...v7.0.1)
|
|
12
84
|
|
|
13
85
|
**Implemented enhancements:**
|
|
14
86
|
|
|
87
|
+
- Any way to manually clear/reset the changelog history? [\#568](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/568)
|
|
88
|
+
- Present the entire changelog in its own view [\#569](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/569) ([mhenrixon](https://github.com/mhenrixon))
|
|
15
89
|
- Give user full control over adding middleware [\#566](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/566) ([mhenrixon](https://github.com/mhenrixon))
|
|
16
90
|
|
|
17
91
|
**Fixed bugs:**
|
|
18
92
|
|
|
19
|
-
-
|
|
20
|
-
- Fix until and while executed and improve documentation [\#397](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/397)
|
|
21
|
-
- Fix 532 [\#549](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/549) ([mhenrixon](https://github.com/mhenrixon))
|
|
93
|
+
- Fix configuration [\#570](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/570) ([mhenrixon](https://github.com/mhenrixon))
|
|
22
94
|
|
|
23
95
|
**Closed issues:**
|
|
24
96
|
|
|
97
|
+
- undefined method 'delete\_by\_digest' for SidekiqUniqueJobs::Digests:Class [\#567](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/567)
|
|
25
98
|
- Rejected jobs are still displayed as 'Queued' with gem 'sidekiq-status' on /sidekiq/statuses [\#564](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/564)
|
|
26
|
-
- Is it possible to have a :until\_executed lock with an expiration time? [\#524](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/524)
|
|
27
99
|
|
|
28
|
-
## [v7.0.0
|
|
100
|
+
## [v7.0.0](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0) (2021-01-20)
|
|
29
101
|
|
|
30
|
-
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.
|
|
102
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta29...v7.0.0)
|
|
31
103
|
|
|
32
104
|
**Implemented enhancements:**
|
|
33
105
|
|
|
@@ -35,6 +107,20 @@
|
|
|
35
107
|
|
|
36
108
|
**Fixed bugs:**
|
|
37
109
|
|
|
110
|
+
- Race condition in ruby reaper [\#559](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/559)
|
|
111
|
+
- Fix until and while executed and improve documentation [\#397](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/397)
|
|
112
|
+
- Fix race condition to avoid reaping active jobs [\#563](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/563) ([mhenrixon](https://github.com/mhenrixon))
|
|
113
|
+
|
|
114
|
+
**Closed issues:**
|
|
115
|
+
|
|
116
|
+
- Is it possible to have a :until\_executed lock with an expiration time? [\#524](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/524)
|
|
117
|
+
|
|
118
|
+
## [v7.0.0.beta29](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta29) (2021-01-16)
|
|
119
|
+
|
|
120
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta28...v7.0.0.beta29)
|
|
121
|
+
|
|
122
|
+
**Fixed bugs:**
|
|
123
|
+
|
|
38
124
|
- Ruby Reaper active check incorrect [\#557](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/557)
|
|
39
125
|
- Routes with authentication should work with web [\#562](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/562) ([mhenrixon](https://github.com/mhenrixon))
|
|
40
126
|
|
|
@@ -50,7 +136,6 @@
|
|
|
50
136
|
|
|
51
137
|
- lock\_args does not work when you define the lock\_args argument and default lock\_args function at the same time. [\#548](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/548)
|
|
52
138
|
- incorrect `:while\_executing` behavior [\#432](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/432)
|
|
53
|
-
- Fix race condition to avoid reaping active jobs [\#563](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/563) ([mhenrixon](https://github.com/mhenrixon))
|
|
54
139
|
- Fix active check for the worker hash [\#558](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/558) ([mhenrixon](https://github.com/mhenrixon))
|
|
55
140
|
- Prefer lock\_prefix not unique\_prefix [\#554](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/554) ([mhenrixon](https://github.com/mhenrixon))
|
|
56
141
|
- Fix issue 432 [\#552](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/552) ([mhenrixon](https://github.com/mhenrixon))
|
|
@@ -72,6 +157,10 @@
|
|
|
72
157
|
- Documentation incorrect for `delete\_by\_digest` [\#547](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/547)
|
|
73
158
|
- Locked jobs after kill -9 with while\_executing lock [\#546](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/546)
|
|
74
159
|
|
|
160
|
+
**Merged pull requests:**
|
|
161
|
+
|
|
162
|
+
- Support apartment [\#540](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/540) ([mhenrixon](https://github.com/mhenrixon))
|
|
163
|
+
|
|
75
164
|
## [v7.0.0.beta26](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta26) (2020-10-28)
|
|
76
165
|
|
|
77
166
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v6.0.25...v7.0.0.beta26)
|
|
@@ -111,10 +200,6 @@
|
|
|
111
200
|
|
|
112
201
|
- ConnectionPool::TimeoutError and :until\_executed [\#535](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/535)
|
|
113
202
|
|
|
114
|
-
**Merged pull requests:**
|
|
115
|
-
|
|
116
|
-
- Support apartment [\#540](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/540) ([mhenrixon](https://github.com/mhenrixon))
|
|
117
|
-
|
|
118
203
|
## [v7.0.0.beta24](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta24) (2020-09-27)
|
|
119
204
|
|
|
120
205
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta23...v7.0.0.beta24)
|
|
@@ -165,7 +250,6 @@
|
|
|
165
250
|
**Fixed bugs:**
|
|
166
251
|
|
|
167
252
|
- Prevent indefinitely looping entries [\#516](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/516) ([mhenrixon](https://github.com/mhenrixon))
|
|
168
|
-
- Don't bypass lock\_args if arguments are empty [\#511](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/511) ([nhippenmeyer](https://github.com/nhippenmeyer))
|
|
169
253
|
|
|
170
254
|
**Closed issues:**
|
|
171
255
|
|
|
@@ -219,6 +303,7 @@
|
|
|
219
303
|
|
|
220
304
|
- Deprecate configuration options with `default\_` [\#504](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/504) ([mhenrixon](https://github.com/mhenrixon))
|
|
221
305
|
- Fix access to both server and client conflict [\#503](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/503) ([mhenrixon](https://github.com/mhenrixon))
|
|
306
|
+
- Use thread-safe digest creation mechanism [\#484](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/484) ([mhenrixon](https://github.com/mhenrixon))
|
|
222
307
|
|
|
223
308
|
**Closed issues:**
|
|
224
309
|
|
|
@@ -247,6 +332,7 @@
|
|
|
247
332
|
- Brpoplpush::RedisScript::LuaError: WRONGTYPE Operation against a key holding the wrong kind of value [\#491](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/491)
|
|
248
333
|
- Lua script bug [\#489](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/489)
|
|
249
334
|
- Reaper will delete locks for running jobs [\#488](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/488)
|
|
335
|
+
- Fix access to hash members [\#496](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/496) ([mhenrixon](https://github.com/mhenrixon))
|
|
250
336
|
- Fix cursor assignment [\#494](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/494) ([mhenrixon](https://github.com/mhenrixon))
|
|
251
337
|
- Prevent reaping of active jobs [\#493](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/493) ([mhenrixon](https://github.com/mhenrixon))
|
|
252
338
|
|
|
@@ -262,11 +348,6 @@
|
|
|
262
348
|
|
|
263
349
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta13...v6.0.21)
|
|
264
350
|
|
|
265
|
-
**Fixed bugs:**
|
|
266
|
-
|
|
267
|
-
- Fix access to hash members [\#496](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/496) ([mhenrixon](https://github.com/mhenrixon))
|
|
268
|
-
- Use thread-safe digest creation mechanism [\#484](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/484) ([mhenrixon](https://github.com/mhenrixon))
|
|
269
|
-
|
|
270
351
|
## [v7.0.0.beta13](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta13) (2020-03-26)
|
|
271
352
|
|
|
272
353
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta12...v7.0.0.beta13)
|
|
@@ -284,6 +365,10 @@
|
|
|
284
365
|
- until\_expired is not setting TTL [\#468](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/468)
|
|
285
366
|
- Fix bug where expiration wasn't set until unlock [\#481](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/481) ([mhenrixon](https://github.com/mhenrixon))
|
|
286
367
|
|
|
368
|
+
**Merged pull requests:**
|
|
369
|
+
|
|
370
|
+
- Allow v7 configuration to work [\#479](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/479) ([mhenrixon](https://github.com/mhenrixon))
|
|
371
|
+
|
|
287
372
|
## [v6.0.20](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.20) (2020-03-22)
|
|
288
373
|
|
|
289
374
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta11...v6.0.20)
|
|
@@ -297,10 +382,6 @@
|
|
|
297
382
|
- Please keep some recent versions on rubygems.org [\#478](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/478)
|
|
298
383
|
- validate\_worker! throws error [\#466](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/466)
|
|
299
384
|
|
|
300
|
-
**Merged pull requests:**
|
|
301
|
-
|
|
302
|
-
- Allow v7 configuration to work [\#479](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/479) ([mhenrixon](https://github.com/mhenrixon))
|
|
303
|
-
|
|
304
385
|
## [v7.0.0.beta11](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.0.beta11) (2020-03-21)
|
|
305
386
|
|
|
306
387
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.0.beta10...v7.0.0.beta11)
|
|
@@ -463,6 +544,7 @@
|
|
|
463
544
|
|
|
464
545
|
- Redis is busy running script and script never terminates [\#441](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/441)
|
|
465
546
|
- Make the ruby reaper plain ruby [\#443](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/443) ([mhenrixon](https://github.com/mhenrixon))
|
|
547
|
+
- Fix sidekiq develop [\#426](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/426) ([mhenrixon](https://github.com/mhenrixon))
|
|
466
548
|
|
|
467
549
|
**Closed issues:**
|
|
468
550
|
|
|
@@ -500,11 +582,10 @@
|
|
|
500
582
|
|
|
501
583
|
**Fixed bugs:**
|
|
502
584
|
|
|
503
|
-
- Include redis directory [\#430](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/430) ([mhenrixon](https://github.com/mhenrixon))
|
|
504
585
|
- Allow Sidekiq::Context to be used for logging [\#429](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/429) ([mhenrixon](https://github.com/mhenrixon))
|
|
505
|
-
- Fix sidekiq develop [\#426](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/426) ([mhenrixon](https://github.com/mhenrixon))
|
|
506
586
|
- Reap as many orphans as advertised [\#403](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/403) ([mhenrixon](https://github.com/mhenrixon))
|
|
507
587
|
- Reaper should remove :INFO keys [\#399](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/399) ([mhenrixon](https://github.com/mhenrixon))
|
|
588
|
+
- Convert v5 locks when needed [\#375](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/375) ([mhenrixon](https://github.com/mhenrixon))
|
|
508
589
|
|
|
509
590
|
**Merged pull requests:**
|
|
510
591
|
|
|
@@ -527,6 +608,7 @@
|
|
|
527
608
|
|
|
528
609
|
- Duplicate job was pushed \( v6.0.13 \) [\#414](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/414)
|
|
529
610
|
- Constant SidekiqUniqueJobs::Web::Digests not found [\#396](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/396)
|
|
611
|
+
- Include redis directory [\#430](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/430) ([mhenrixon](https://github.com/mhenrixon))
|
|
530
612
|
- Fix v6 [\#428](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/428) ([mhenrixon](https://github.com/mhenrixon))
|
|
531
613
|
|
|
532
614
|
**Closed issues:**
|
|
@@ -582,7 +664,6 @@
|
|
|
582
664
|
- Version 6: lets you schedule job with missing arguments [\#351](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/351)
|
|
583
665
|
- Version 6 Ignores Jobs Enqueued in Version 5 [\#345](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/345)
|
|
584
666
|
- Job will not enqueue even with no existing match [\#342](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/342)
|
|
585
|
-
- Convert v5 locks when needed [\#375](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/375) ([mhenrixon](https://github.com/mhenrixon))
|
|
586
667
|
|
|
587
668
|
**Closed issues:**
|
|
588
669
|
|
|
@@ -647,7 +728,7 @@
|
|
|
647
728
|
|
|
648
729
|
- Version 5: Job ID Hash Entries Not Removed if Unique Key Expires [\#346](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/346)
|
|
649
730
|
- Move the lpush last [\#354](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/354) ([mhenrixon](https://github.com/mhenrixon))
|
|
650
|
-
-
|
|
731
|
+
- Convert expiration time to integer [\#330](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/330) ([dareddov](https://github.com/dareddov))
|
|
651
732
|
|
|
652
733
|
**Closed issues:**
|
|
653
734
|
|
|
@@ -669,7 +750,6 @@
|
|
|
669
750
|
**Fixed bugs:**
|
|
670
751
|
|
|
671
752
|
- Always Remove Job ID from uniquejobs Hash [\#347](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/347) ([chadrschroeder](https://github.com/chadrschroeder))
|
|
672
|
-
- Convert expiration time to integer [\#330](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/330) ([dareddov](https://github.com/dareddov))
|
|
673
753
|
|
|
674
754
|
**Closed issues:**
|
|
675
755
|
|
|
@@ -710,6 +790,10 @@
|
|
|
710
790
|
|
|
711
791
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v6.0.3...v6.0.4)
|
|
712
792
|
|
|
793
|
+
**Fixed bugs:**
|
|
794
|
+
|
|
795
|
+
- Fix the broken expiration [\#316](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/316) ([mhenrixon](https://github.com/mhenrixon))
|
|
796
|
+
|
|
713
797
|
**Closed issues:**
|
|
714
798
|
|
|
715
799
|
- Question about until\_timeout with 6.0.0 [\#303](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/303)
|
|
@@ -757,7 +841,6 @@
|
|
|
757
841
|
|
|
758
842
|
- Dead jobs [\#308](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/308) ([mhenrixon](https://github.com/mhenrixon))
|
|
759
843
|
- Fix require path for sidekiq\_unique\_jobs/web [\#305](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/305) ([soundasleep](https://github.com/soundasleep))
|
|
760
|
-
- Update README.md [\#300](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/300) ([pirj](https://github.com/pirj))
|
|
761
844
|
|
|
762
845
|
## [v6.0.0](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.0) (2018-07-27)
|
|
763
846
|
|
|
@@ -776,6 +859,7 @@
|
|
|
776
859
|
**Merged pull requests:**
|
|
777
860
|
|
|
778
861
|
- Add more details about testing uniqueness [\#301](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/301) ([mhenrixon](https://github.com/mhenrixon))
|
|
862
|
+
- Update README.md [\#300](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/300) ([pirj](https://github.com/pirj))
|
|
779
863
|
|
|
780
864
|
## [v6.0.0.rc7](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.0.rc7) (2018-07-23)
|
|
781
865
|
|
|
@@ -820,6 +904,7 @@
|
|
|
820
904
|
**Fixed bugs:**
|
|
821
905
|
|
|
822
906
|
- bundle exec jobs console does not work [\#253](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/253)
|
|
907
|
+
- Rename command line binary [\#287](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/287) ([mhenrixon](https://github.com/mhenrixon))
|
|
823
908
|
|
|
824
909
|
## [v6.0.0.rc4](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.0.rc4) (2018-06-30)
|
|
825
910
|
|
|
@@ -830,10 +915,6 @@
|
|
|
830
915
|
- Prepare for v6 [\#286](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/286) ([mhenrixon](https://github.com/mhenrixon))
|
|
831
916
|
- Only unlock not delete [\#285](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/285) ([mhenrixon](https://github.com/mhenrixon))
|
|
832
917
|
|
|
833
|
-
**Fixed bugs:**
|
|
834
|
-
|
|
835
|
-
- Rename command line binary [\#287](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/287) ([mhenrixon](https://github.com/mhenrixon))
|
|
836
|
-
|
|
837
918
|
## [v6.0.0.rc3](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.0.rc3) (2018-06-29)
|
|
838
919
|
|
|
839
920
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v6.0.0.rc2...v6.0.0.rc3)
|
|
@@ -897,6 +978,7 @@
|
|
|
897
978
|
**Implemented enhancements:**
|
|
898
979
|
|
|
899
980
|
- Code smells [\#275](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/275) ([mhenrixon](https://github.com/mhenrixon))
|
|
981
|
+
- Reject while scheduling [\#273](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/273) ([mhenrixon](https://github.com/mhenrixon))
|
|
900
982
|
- Improve testing [\#272](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/272) ([mhenrixon](https://github.com/mhenrixon))
|
|
901
983
|
|
|
902
984
|
## [v6.0.0.beta](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.0.beta) (2018-06-17)
|
|
@@ -905,7 +987,6 @@
|
|
|
905
987
|
|
|
906
988
|
**Implemented enhancements:**
|
|
907
989
|
|
|
908
|
-
- Reject while scheduling [\#273](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/273) ([mhenrixon](https://github.com/mhenrixon))
|
|
909
990
|
- Until and while executing [\#271](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/271) ([mhenrixon](https://github.com/mhenrixon))
|
|
910
991
|
- Solidify master [\#270](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/270) ([mhenrixon](https://github.com/mhenrixon))
|
|
911
992
|
- Minor adjustments [\#268](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/268) ([mhenrixon](https://github.com/mhenrixon))
|
|
@@ -1320,7 +1401,6 @@
|
|
|
1320
1401
|
- Do not unlock on sidekiq shutdown [\#87](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/87) ([deltaroe](https://github.com/deltaroe))
|
|
1321
1402
|
- Remove no-op code, protect global space from test code [\#86](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/86) ([stevenjonescgm](https://github.com/stevenjonescgm))
|
|
1322
1403
|
- Remove unique lock when executing and clearing jobs in sidekiq fake mode [\#83](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/83) ([crberube](https://github.com/crberube))
|
|
1323
|
-
- Fix tests. Tests with latest sidekiq versions and ruby versions [\#82](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/82) ([simonoff](https://github.com/simonoff))
|
|
1324
1404
|
- Duplicate Payload logging configuration [\#81](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/81) ([jprincipe](https://github.com/jprincipe))
|
|
1325
1405
|
- output log if not unique [\#79](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/79) ([sonots](https://github.com/sonots))
|
|
1326
1406
|
- Checking Sidekiq::Testing.inline? on testing strategy and connector [\#75](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/75) ([Draiken](https://github.com/Draiken))
|
|
@@ -1336,6 +1416,7 @@
|
|
|
1336
1416
|
|
|
1337
1417
|
**Merged pull requests:**
|
|
1338
1418
|
|
|
1419
|
+
- Fix tests. Tests with latest sidekiq versions and ruby versions [\#82](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/82) ([simonoff](https://github.com/simonoff))
|
|
1339
1420
|
- Use ConnectionPool blocks to ensure exclusive connection. Closes \#66. [\#67](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/67) ([adstage-david](https://github.com/adstage-david))
|
|
1340
1421
|
|
|
1341
1422
|
## [v3.0.10](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v3.0.10) (2014-11-19)
|
|
@@ -1417,6 +1498,7 @@
|
|
|
1417
1498
|
- Clarify README about unique expiration [\#36](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/36) ([spacemunkay](https://github.com/spacemunkay))
|
|
1418
1499
|
- Add option to make jobs unique on all queues [\#32](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/32) ([robinmessage](https://github.com/robinmessage))
|
|
1419
1500
|
- Fix homepage in gemspec [\#29](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/29) ([tmaier](https://github.com/tmaier))
|
|
1501
|
+
- Compatibility with Sidekiq 2.12.1 Scheduled Jobs [\#16](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/16) ([lsimoneau](https://github.com/lsimoneau))
|
|
1420
1502
|
|
|
1421
1503
|
## [v2.7.0](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v2.7.0) (2013-11-24)
|
|
1422
1504
|
|
|
@@ -1435,7 +1517,6 @@
|
|
|
1435
1517
|
- Make unlock/yield order configurable. [\#21](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/21) ([endofunky](https://github.com/endofunky))
|
|
1436
1518
|
- Rely on Sidekiq's String\#constantize extension instead of rolling our own [\#19](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/19) ([disbelief](https://github.com/disbelief))
|
|
1437
1519
|
- Attempt to constantize String `worker\_class` arguments passed to client middleware [\#17](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/17) ([disbelief](https://github.com/disbelief))
|
|
1438
|
-
- Compatibility with Sidekiq 2.12.1 Scheduled Jobs [\#16](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/16) ([lsimoneau](https://github.com/lsimoneau))
|
|
1439
1520
|
- Allow worker to specify which arguments to include in uniquing hash [\#12](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/12) ([sax](https://github.com/sax))
|
|
1440
1521
|
- Add support for unique when using Sidekiq's delay function [\#11](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/11) ([eduardosasso](https://github.com/eduardosasso))
|
|
1441
1522
|
- Adding the unique prefix option [\#8](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/8) ([KensoDev](https://github.com/KensoDev))
|
data/README.md
CHANGED
|
@@ -100,7 +100,7 @@ Before v7, the middleware was configured automatically. Since some people report
|
|
|
100
100
|
|
|
101
101
|
*NOTE* if you want to use the reaper you also need to configure the server middleware.
|
|
102
102
|
|
|
103
|
-
[A full
|
|
103
|
+
[A full example](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/myapp/config/initializers/sidekiq.rb#L12)
|
|
104
104
|
|
|
105
105
|
```ruby
|
|
106
106
|
Sidekiq.configure_server do |config|
|
|
@@ -185,16 +185,16 @@ Configure SidekiqUniqueJobs in an initializer or the sidekiq initializer on appl
|
|
|
185
185
|
|
|
186
186
|
```ruby
|
|
187
187
|
SidekiqUniqueJobs.configure do |config|
|
|
188
|
-
config.
|
|
189
|
-
config.
|
|
190
|
-
config.
|
|
191
|
-
config.
|
|
192
|
-
config.
|
|
193
|
-
config.max_history =
|
|
194
|
-
config.reaper = :lua
|
|
195
|
-
config.reaper_count =
|
|
196
|
-
config.reaper_interval = 10
|
|
197
|
-
config.reaper_timeout = 5
|
|
188
|
+
config.logger = Sidekiq.logger # default, change at your own discretion
|
|
189
|
+
config.debug_lua = false # Turn on when debugging
|
|
190
|
+
config.lock_info = false # Turn on when debugging
|
|
191
|
+
config.lock_ttl = 600 # Expire locks after 10 minutes
|
|
192
|
+
config.lock_timeout = nil # turn off lock timeout
|
|
193
|
+
config.max_history = 0 # Turn on when debugging
|
|
194
|
+
config.reaper = :ruby # :ruby, :lua or :none/nil
|
|
195
|
+
config.reaper_count = 1000 # Stop reaping after this many keys
|
|
196
|
+
config.reaper_interval = 600 # Reap orphans every 10 minutes
|
|
197
|
+
config.reaper_timeout = 150 # Timeout reaper after 2.5 minutes
|
|
198
198
|
end
|
|
199
199
|
```
|
|
200
200
|
|
|
@@ -709,7 +709,7 @@ class MyWorker
|
|
|
709
709
|
end
|
|
710
710
|
```
|
|
711
711
|
|
|
712
|
-
Starting in v5.1, Sidekiq can also fire a global callback when a job dies:
|
|
712
|
+
Starting in v5.1, Sidekiq can also fire a global callback when a job dies: In version 7, this is handled automatically for you. You don't need to add a death handler, if you configure v7 like in [Add the middleware](#add-the-middleware) you don't have to worry about the below.
|
|
713
713
|
|
|
714
714
|
```ruby
|
|
715
715
|
Sidekiq.configure_server do |config|
|
|
@@ -765,14 +765,28 @@ The reason for this is that the global id needs to be set before the unique jobs
|
|
|
765
765
|
It was reported in [#564](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/564) that the order of the middleware needs to be as follows.
|
|
766
766
|
|
|
767
767
|
```ruby
|
|
768
|
-
|
|
769
|
-
|
|
770
|
-
|
|
768
|
+
# Thanks to @ArturT for the correction
|
|
769
|
+
|
|
770
|
+
Sidekiq.configure_server do |config|
|
|
771
|
+
config.client_middleware do |chain|
|
|
772
|
+
chain.add SidekiqUniqueJobs::Middleware::Client
|
|
773
|
+
chain.add Sidekiq::Status::ClientMiddleware, expiration: 30.minutes
|
|
774
|
+
end
|
|
775
|
+
|
|
776
|
+
config.server_middleware do |chain|
|
|
777
|
+
chain.add Sidekiq::Status::ServerMiddleware, expiration: 30.minutes
|
|
778
|
+
chain.add SidekiqUniqueJobs::Middleware::Server
|
|
779
|
+
end
|
|
780
|
+
|
|
781
|
+
SidekiqUniqueJobs::Server.configure(config)
|
|
771
782
|
end
|
|
772
783
|
|
|
773
|
-
|
|
774
|
-
|
|
775
|
-
|
|
784
|
+
|
|
785
|
+
Sidekiq.configure_client do |config|
|
|
786
|
+
config.client_middleware do |chain|
|
|
787
|
+
chain.add SidekiqUniqueJobs::Middleware::Client
|
|
788
|
+
chain.add Sidekiq::Status::ClientMiddleware, expiration: 30.minutes
|
|
789
|
+
end
|
|
776
790
|
end
|
|
777
791
|
```
|
|
778
792
|
|
|
@@ -889,7 +903,7 @@ RSpec.describe Workers::CoolOne do
|
|
|
889
903
|
end
|
|
890
904
|
```
|
|
891
905
|
|
|
892
|
-
It is
|
|
906
|
+
It is recommended to leave the uniqueness testing to the gem maintainers. If you care about how the gem is integration tested have a look at the following specs:
|
|
893
907
|
|
|
894
908
|
- [spec/integration/sidekiq_unique_jobs/lock/until_and_while_executing_spec.rb](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/spec/integration/sidekiq_unique_jobs/lock/until_and_while_executing_spec.rb)
|
|
895
909
|
- [spec/integration/sidekiq_unique_jobs/lock/until_executed_spec.rb](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/spec/integration/sidekiq_unique_jobs/lock/until_executed_spec.rb)
|
data/lib/sidekiq_unique_jobs.rb
CHANGED
|
@@ -3,9 +3,10 @@
|
|
|
3
3
|
require "brpoplpush/redis_script"
|
|
4
4
|
require "concurrent/future"
|
|
5
5
|
require "concurrent/promises"
|
|
6
|
-
require "concurrent/timer_task"
|
|
7
6
|
require "concurrent/map"
|
|
8
7
|
require "concurrent/mutable_struct"
|
|
8
|
+
require "concurrent/timer_task"
|
|
9
|
+
require "concurrent/executor/ruby_single_thread_executor"
|
|
9
10
|
require "digest"
|
|
10
11
|
require "digest/sha1"
|
|
11
12
|
require "erb"
|
|
@@ -14,6 +15,7 @@ require "json"
|
|
|
14
15
|
require "pathname"
|
|
15
16
|
require "sidekiq"
|
|
16
17
|
|
|
18
|
+
require "sidekiq_unique_jobs/timer_task"
|
|
17
19
|
require "sidekiq_unique_jobs/version"
|
|
18
20
|
require "sidekiq_unique_jobs/version_check"
|
|
19
21
|
require "sidekiq_unique_jobs/constants"
|
|
@@ -15,11 +15,17 @@ module SidekiqUniqueJobs
|
|
|
15
15
|
# @return [Object]
|
|
16
16
|
#
|
|
17
17
|
def load_json(string)
|
|
18
|
-
return
|
|
18
|
+
return if string.nil? || string.empty?
|
|
19
19
|
|
|
20
20
|
::JSON.parse(string)
|
|
21
21
|
end
|
|
22
22
|
|
|
23
|
+
def safe_load_json(string)
|
|
24
|
+
return string if string.is_a?(Hash)
|
|
25
|
+
|
|
26
|
+
load_json(string)
|
|
27
|
+
end
|
|
28
|
+
|
|
23
29
|
#
|
|
24
30
|
# Dumps an object into a JSON string
|
|
25
31
|
#
|
|
@@ -53,7 +53,7 @@ module SidekiqUniqueJobs
|
|
|
53
53
|
#
|
|
54
54
|
# Locks a job_id
|
|
55
55
|
#
|
|
56
|
-
# @note intended only for testing
|
|
56
|
+
# @note intended only for testing purposes
|
|
57
57
|
#
|
|
58
58
|
# @param [String] job_id a sidekiq JID
|
|
59
59
|
# @param [Hash] lock_info information about the lock
|
|
@@ -73,6 +73,36 @@ module SidekiqUniqueJobs
|
|
|
73
73
|
end
|
|
74
74
|
end
|
|
75
75
|
|
|
76
|
+
#
|
|
77
|
+
# Create the :QUEUED key
|
|
78
|
+
#
|
|
79
|
+
# @note intended only for testing purposes
|
|
80
|
+
#
|
|
81
|
+
# @param [String] job_id a sidekiq JID
|
|
82
|
+
#
|
|
83
|
+
# @return [void]
|
|
84
|
+
#
|
|
85
|
+
def queue(job_id)
|
|
86
|
+
redis do |conn|
|
|
87
|
+
conn.lpush(key.queued, job_id)
|
|
88
|
+
end
|
|
89
|
+
end
|
|
90
|
+
|
|
91
|
+
#
|
|
92
|
+
# Create the :PRIMED key
|
|
93
|
+
#
|
|
94
|
+
# @note intended only for testing purposes
|
|
95
|
+
#
|
|
96
|
+
# @param [String] job_id a sidekiq JID
|
|
97
|
+
#
|
|
98
|
+
# @return [void]
|
|
99
|
+
#
|
|
100
|
+
def prime(job_id)
|
|
101
|
+
redis do |conn|
|
|
102
|
+
conn.lpush(key.primed, job_id)
|
|
103
|
+
end
|
|
104
|
+
end
|
|
105
|
+
|
|
76
106
|
#
|
|
77
107
|
# Unlock a specific job_id
|
|
78
108
|
#
|
|
@@ -74,19 +74,19 @@ if pttl and pttl > 0 then
|
|
|
74
74
|
log_debug("PEXPIRE", digest, pttl)
|
|
75
75
|
redis.call("PEXPIRE", digest, pttl)
|
|
76
76
|
|
|
77
|
-
log_debug("PEXPIRE", queued, pttl)
|
|
78
|
-
redis.call("PEXPIRE", queued, pttl)
|
|
79
|
-
|
|
80
|
-
log_debug("PEXPIRE", primed, pttl)
|
|
81
|
-
redis.call("PEXPIRE", primed, pttl)
|
|
82
|
-
|
|
83
77
|
log_debug("PEXPIRE", locked, pttl)
|
|
84
78
|
redis.call("PEXPIRE", locked, pttl)
|
|
85
|
-
|
|
86
|
-
log_debug("PEXPIRE", info, pttl)
|
|
87
|
-
redis.call("PEXPIRE", info, pttl)
|
|
88
79
|
end
|
|
89
80
|
|
|
81
|
+
log_debug("PEXPIRE", queued, 1000)
|
|
82
|
+
redis.call("PEXPIRE", queued, 1000)
|
|
83
|
+
|
|
84
|
+
log_debug("PEXPIRE", primed, 1000)
|
|
85
|
+
redis.call("PEXPIRE", primed, 1000)
|
|
86
|
+
|
|
87
|
+
log_debug("PEXPIRE", info, 1000)
|
|
88
|
+
redis.call("PEXPIRE", info, 1000)
|
|
89
|
+
|
|
90
90
|
log("Locked")
|
|
91
91
|
log_debug("END lock digest:", digest, "job_id:", job_id)
|
|
92
92
|
return job_id
|
|
@@ -27,14 +27,14 @@ local function find_digest_in_process_set(digest, threshold)
|
|
|
27
27
|
else
|
|
28
28
|
for i = 1, #jobs, 2 do
|
|
29
29
|
local jobstr = jobs[i +1]
|
|
30
|
-
if string.find(jobstr, digest) then
|
|
30
|
+
if string.find(string.gsub(jobstr, ':RUN', ''), string.gsub(digest, ':RUN', '')) then
|
|
31
31
|
log_debug("Found digest", digest, "in:", workers_key)
|
|
32
32
|
found = true
|
|
33
33
|
break
|
|
34
34
|
end
|
|
35
35
|
|
|
36
36
|
local job = cjson.decode(jobstr)
|
|
37
|
-
if job.created_at > threshold then
|
|
37
|
+
if job.payload.created_at > threshold then
|
|
38
38
|
found = true
|
|
39
39
|
break
|
|
40
40
|
end
|
|
@@ -20,12 +20,14 @@ module SidekiqUniqueJobs
|
|
|
20
20
|
# Starts a separate thread that periodically reaps orphans
|
|
21
21
|
#
|
|
22
22
|
#
|
|
23
|
-
# @return [
|
|
23
|
+
# @return [SidekiqUniqueJobs::TimerTask] the task that was started
|
|
24
24
|
#
|
|
25
|
-
def start # rubocop:disable
|
|
25
|
+
def start(test_task = nil) # rubocop:disable
|
|
26
26
|
return if disabled?
|
|
27
27
|
return if registered?
|
|
28
28
|
|
|
29
|
+
self.task = test_task || default_task
|
|
30
|
+
|
|
29
31
|
with_logging_context do
|
|
30
32
|
register_reaper_process
|
|
31
33
|
log_info("Starting Reaper")
|
|
@@ -59,7 +61,11 @@ module SidekiqUniqueJobs
|
|
|
59
61
|
# @return [<type>] <description>
|
|
60
62
|
#
|
|
61
63
|
def task
|
|
62
|
-
@task ||=
|
|
64
|
+
@task ||= default_task
|
|
65
|
+
end
|
|
66
|
+
|
|
67
|
+
def default_task
|
|
68
|
+
SidekiqUniqueJobs::TimerTask.new(timer_task_options) do
|
|
63
69
|
with_logging_context do
|
|
64
70
|
redis do |conn|
|
|
65
71
|
refresh_reaper_mutex
|
|
@@ -69,6 +75,10 @@ module SidekiqUniqueJobs
|
|
|
69
75
|
end
|
|
70
76
|
end
|
|
71
77
|
|
|
78
|
+
def task=(task)
|
|
79
|
+
@task = task
|
|
80
|
+
end
|
|
81
|
+
|
|
72
82
|
#
|
|
73
83
|
# Arguments passed on to the timer task
|
|
74
84
|
#
|
|
@@ -10,6 +10,7 @@ module SidekiqUniqueJobs
|
|
|
10
10
|
# @author Mikael Henriksson <mikael@mhenrixon.com>
|
|
11
11
|
#
|
|
12
12
|
class RubyReaper < Reaper
|
|
13
|
+
RUN_SUFFIX = ":RUN"
|
|
13
14
|
#
|
|
14
15
|
# @!attribute [r] digests
|
|
15
16
|
# @return [SidekiqUniqueJobs::Digests] digest collection
|
|
@@ -52,11 +53,11 @@ module SidekiqUniqueJobs
|
|
|
52
53
|
# @return [Array<String>] an array of orphaned digests
|
|
53
54
|
#
|
|
54
55
|
def orphans
|
|
55
|
-
conn.zrevrange(digests.key, 0, -1).each_with_object([]) do |digest,
|
|
56
|
+
conn.zrevrange(digests.key, 0, -1).each_with_object([]) do |digest, memo|
|
|
56
57
|
next if belongs_to_job?(digest)
|
|
57
58
|
|
|
58
|
-
|
|
59
|
-
break if
|
|
59
|
+
memo << digest
|
|
60
|
+
break if memo.size >= reaper_count
|
|
60
61
|
end
|
|
61
62
|
end
|
|
62
63
|
|
|
@@ -117,14 +118,19 @@ module SidekiqUniqueJobs
|
|
|
117
118
|
end
|
|
118
119
|
end
|
|
119
120
|
|
|
120
|
-
def active?(digest) # rubocop:disable Metrics/MethodLength, Metrics/CyclomaticComplexity
|
|
121
|
+
def active?(digest) # rubocop:disable Metrics/MethodLength, Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity
|
|
121
122
|
Sidekiq.redis do |conn|
|
|
122
123
|
procs = conn.sscan_each("processes").to_a
|
|
123
124
|
return false if procs.empty?
|
|
124
125
|
|
|
125
126
|
procs.sort.each do |key|
|
|
126
127
|
valid, workers = conn.pipelined do
|
|
127
|
-
|
|
128
|
+
# TODO: Remove the if statement in the future
|
|
129
|
+
if conn.respond_to?(:exists?)
|
|
130
|
+
conn.exists?(key)
|
|
131
|
+
else
|
|
132
|
+
conn.exists(key)
|
|
133
|
+
end
|
|
128
134
|
conn.hgetall("#{key}:workers")
|
|
129
135
|
end
|
|
130
136
|
|
|
@@ -132,10 +138,12 @@ module SidekiqUniqueJobs
|
|
|
132
138
|
next unless workers.any?
|
|
133
139
|
|
|
134
140
|
workers.each_pair do |_tid, job|
|
|
135
|
-
item =
|
|
141
|
+
next unless (item = safe_load_json(job))
|
|
136
142
|
|
|
137
|
-
|
|
138
|
-
|
|
143
|
+
payload = safe_load_json(item[PAYLOAD])
|
|
144
|
+
|
|
145
|
+
return true if match?(digest, payload[LOCK_DIGEST])
|
|
146
|
+
return true if considered_active?(payload[CREATED_AT])
|
|
139
147
|
end
|
|
140
148
|
end
|
|
141
149
|
|
|
@@ -143,6 +151,12 @@ module SidekiqUniqueJobs
|
|
|
143
151
|
end
|
|
144
152
|
end
|
|
145
153
|
|
|
154
|
+
def match?(key_one, key_two)
|
|
155
|
+
return false if key_one.nil? || key_two.nil?
|
|
156
|
+
|
|
157
|
+
key_one.delete_suffix(RUN_SUFFIX) == key_two.delete_suffix(RUN_SUFFIX)
|
|
158
|
+
end
|
|
159
|
+
|
|
146
160
|
def considered_active?(time_f)
|
|
147
161
|
(Time.now - reaper_timeout).to_f < time_f
|
|
148
162
|
end
|
|
@@ -47,7 +47,13 @@ module SidekiqUniqueJobs
|
|
|
47
47
|
#
|
|
48
48
|
def exist?
|
|
49
49
|
redis do |conn|
|
|
50
|
-
|
|
50
|
+
# TODO: Remove the if statement in the future
|
|
51
|
+
value =
|
|
52
|
+
if conn.respond_to?(:exists?)
|
|
53
|
+
conn.exists?(key)
|
|
54
|
+
else
|
|
55
|
+
conn.exists(key)
|
|
56
|
+
end
|
|
51
57
|
|
|
52
58
|
return value if boolean?(value)
|
|
53
59
|
|
|
@@ -0,0 +1,78 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module SidekiqUniqueJobs
|
|
4
|
+
# @see [Concurrent::TimerTask] https://www.rubydoc.info/gems/concurrent-ruby/Concurrent/TimerTask
|
|
5
|
+
#
|
|
6
|
+
class TimerTask < ::Concurrent::TimerTask
|
|
7
|
+
private
|
|
8
|
+
|
|
9
|
+
def ns_initialize(opts, &task)
|
|
10
|
+
set_deref_options(opts)
|
|
11
|
+
|
|
12
|
+
self.execution_interval = opts[:execution] || opts[:execution_interval] || EXECUTION_INTERVAL
|
|
13
|
+
self.timeout_interval = opts[:timeout] || opts[:timeout_interval] || TIMEOUT_INTERVAL
|
|
14
|
+
@run_now = opts[:now] || opts[:run_now]
|
|
15
|
+
@executor = Concurrent::RubySingleThreadExecutor.new
|
|
16
|
+
@running = Concurrent::AtomicBoolean.new(false)
|
|
17
|
+
@task = task
|
|
18
|
+
@value = nil
|
|
19
|
+
|
|
20
|
+
self.observers = Concurrent::Collection::CopyOnNotifyObserverSet.new
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
def schedule_next_task(interval = execution_interval)
|
|
24
|
+
exec_task = ->(completion) { execute_task(completion) }
|
|
25
|
+
Concurrent::ScheduledTask.execute(interval, args: [Concurrent::Event.new], &exec_task)
|
|
26
|
+
nil
|
|
27
|
+
end
|
|
28
|
+
|
|
29
|
+
# @!visibility private
|
|
30
|
+
def execute_task(completion) # rubocop:disable Metrics/MethodLength
|
|
31
|
+
return nil unless @running.true?
|
|
32
|
+
|
|
33
|
+
timeout_task = -> { timeout_task(completion) }
|
|
34
|
+
|
|
35
|
+
Concurrent::ScheduledTask.execute(
|
|
36
|
+
timeout_interval,
|
|
37
|
+
args: [completion],
|
|
38
|
+
&timeout_task
|
|
39
|
+
)
|
|
40
|
+
@thread_completed = Concurrent::Event.new
|
|
41
|
+
|
|
42
|
+
@value = @reason = nil
|
|
43
|
+
@executor.post do
|
|
44
|
+
@value = @task.call(self)
|
|
45
|
+
rescue Exception => ex # rubocop:disable Lint/RescueException
|
|
46
|
+
@reason = ex
|
|
47
|
+
ensure
|
|
48
|
+
@thread_completed.set
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
@thread_completed.wait
|
|
52
|
+
|
|
53
|
+
if completion.try?
|
|
54
|
+
schedule_next_task
|
|
55
|
+
time = Time.now
|
|
56
|
+
observers.notify_observers do
|
|
57
|
+
[time, value, @reason]
|
|
58
|
+
end
|
|
59
|
+
end
|
|
60
|
+
nil
|
|
61
|
+
end
|
|
62
|
+
|
|
63
|
+
# @!visibility private
|
|
64
|
+
def timeout_task(completion)
|
|
65
|
+
return unless @running.true?
|
|
66
|
+
return unless completion.try?
|
|
67
|
+
|
|
68
|
+
@executor.kill
|
|
69
|
+
@executor.wait_for_termination
|
|
70
|
+
@executor = Concurrent::RubySingleThreadExecutor.new
|
|
71
|
+
|
|
72
|
+
@thread_completed.set
|
|
73
|
+
|
|
74
|
+
schedule_next_task
|
|
75
|
+
observers.notify_observers(Time.now, nil, Concurrent::TimeoutError.new)
|
|
76
|
+
end
|
|
77
|
+
end
|
|
78
|
+
end
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: sidekiq-unique-jobs
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 7.0.
|
|
4
|
+
version: 7.0.6
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Mikael Henriksson
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2021-
|
|
11
|
+
date: 2021-03-19 00:00:00.000000000 Z
|
|
12
12
|
dependencies:
|
|
13
13
|
- !ruby/object:Gem::Dependency
|
|
14
14
|
name: brpoplpush-redis_script
|
|
@@ -16,7 +16,7 @@ dependencies:
|
|
|
16
16
|
requirements:
|
|
17
17
|
- - ">"
|
|
18
18
|
- !ruby/object:Gem::Version
|
|
19
|
-
version: 0.
|
|
19
|
+
version: 0.1.1
|
|
20
20
|
- - "<="
|
|
21
21
|
- !ruby/object:Gem::Version
|
|
22
22
|
version: 2.0.0
|
|
@@ -26,7 +26,7 @@ dependencies:
|
|
|
26
26
|
requirements:
|
|
27
27
|
- - ">"
|
|
28
28
|
- !ruby/object:Gem::Version
|
|
29
|
-
version: 0.
|
|
29
|
+
version: 0.1.1
|
|
30
30
|
- - "<="
|
|
31
31
|
- !ruby/object:Gem::Version
|
|
32
32
|
version: 2.0.0
|
|
@@ -193,6 +193,7 @@ files:
|
|
|
193
193
|
- lib/sidekiq_unique_jobs/sidekiq_unique_jobs.rb
|
|
194
194
|
- lib/sidekiq_unique_jobs/sidekiq_worker_methods.rb
|
|
195
195
|
- lib/sidekiq_unique_jobs/testing.rb
|
|
196
|
+
- lib/sidekiq_unique_jobs/timer_task.rb
|
|
196
197
|
- lib/sidekiq_unique_jobs/timing.rb
|
|
197
198
|
- lib/sidekiq_unique_jobs/unlockable.rb
|
|
198
199
|
- lib/sidekiq_unique_jobs/update_version.rb
|
|
@@ -256,7 +257,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
256
257
|
- !ruby/object:Gem::Version
|
|
257
258
|
version: '0'
|
|
258
259
|
requirements: []
|
|
259
|
-
rubygems_version: 3.2.
|
|
260
|
+
rubygems_version: 3.2.6
|
|
260
261
|
signing_key:
|
|
261
262
|
specification_version: 4
|
|
262
263
|
summary: Sidekiq middleware that prevents duplicates jobs
|