sidekiq-unique-jobs 7.1.5 → 7.1.25
Sign up to get free protection for your applications and to get access to all the features.
Potentially problematic release.
This version of sidekiq-unique-jobs might be problematic. Click here for more details.
- checksums.yaml +4 -4
- data/CHANGELOG.md +244 -4
- data/README.md +16 -14
- data/lib/sidekiq_unique_jobs/batch_delete.rb +6 -6
- data/lib/sidekiq_unique_jobs/changelog.rb +3 -3
- data/lib/sidekiq_unique_jobs/config.rb +43 -4
- data/lib/sidekiq_unique_jobs/constants.rb +44 -44
- data/lib/sidekiq_unique_jobs/deprecation.rb +30 -0
- data/lib/sidekiq_unique_jobs/digests.rb +6 -9
- data/lib/sidekiq_unique_jobs/exceptions.rb +4 -3
- data/lib/sidekiq_unique_jobs/json.rb +7 -0
- data/lib/sidekiq_unique_jobs/lock/base_lock.rb +30 -22
- data/lib/sidekiq_unique_jobs/lock/until_and_while_executing.rb +9 -3
- data/lib/sidekiq_unique_jobs/lock/until_executed.rb +14 -4
- data/lib/sidekiq_unique_jobs/lock/until_executing.rb +14 -4
- data/lib/sidekiq_unique_jobs/lock/until_expired.rb +13 -5
- data/lib/sidekiq_unique_jobs/lock/while_executing.rb +9 -4
- data/lib/sidekiq_unique_jobs/lock.rb +10 -10
- data/lib/sidekiq_unique_jobs/lock_args.rb +19 -15
- data/lib/sidekiq_unique_jobs/lock_config.rb +6 -6
- data/lib/sidekiq_unique_jobs/lock_digest.rb +6 -6
- data/lib/sidekiq_unique_jobs/lock_info.rb +2 -2
- data/lib/sidekiq_unique_jobs/lock_timeout.rb +4 -4
- data/lib/sidekiq_unique_jobs/lock_ttl.rb +5 -5
- data/lib/sidekiq_unique_jobs/locksmith.rb +5 -4
- data/lib/sidekiq_unique_jobs/logging.rb +9 -0
- data/lib/sidekiq_unique_jobs/lua/lock.lua +3 -3
- data/lib/sidekiq_unique_jobs/lua/shared/_common.lua +1 -1
- data/lib/sidekiq_unique_jobs/lua/unlock.lua +12 -5
- data/lib/sidekiq_unique_jobs/middleware/client.rb +3 -1
- data/lib/sidekiq_unique_jobs/middleware/server.rb +2 -0
- data/lib/sidekiq_unique_jobs/middleware.rb +4 -4
- data/lib/sidekiq_unique_jobs/on_conflict/reject.rb +4 -4
- data/lib/sidekiq_unique_jobs/on_conflict/reschedule.rb +3 -3
- data/lib/sidekiq_unique_jobs/options_with_fallback.rb +5 -7
- data/lib/sidekiq_unique_jobs/orphans/manager.rb +38 -10
- data/lib/sidekiq_unique_jobs/orphans/ruby_reaper.rb +77 -14
- data/lib/sidekiq_unique_jobs/redis/string.rb +3 -1
- data/lib/sidekiq_unique_jobs/reflectable.rb +11 -2
- data/lib/sidekiq_unique_jobs/reflections.rb +12 -1
- data/lib/sidekiq_unique_jobs/server.rb +13 -1
- data/lib/sidekiq_unique_jobs/sidekiq_unique_ext.rb +35 -13
- data/lib/sidekiq_unique_jobs/sidekiq_worker_methods.rb +40 -21
- data/lib/sidekiq_unique_jobs/testing.rb +45 -20
- data/lib/sidekiq_unique_jobs/timer_task.rb +266 -45
- data/lib/sidekiq_unique_jobs/timing.rb +1 -1
- data/lib/sidekiq_unique_jobs/upgrade_locks.rb +6 -6
- data/lib/sidekiq_unique_jobs/version.rb +1 -1
- data/lib/sidekiq_unique_jobs/web/helpers.rb +1 -1
- data/lib/tasks/changelog.rake +16 -16
- metadata +11 -15
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 4a5c0d9d30f11e61d3cc7599343951d242c5ccba2c93ee2a46138eab1037db81
|
4
|
+
data.tar.gz: a0721afd3b1a4e98655f9d1aba34e609d9ddd71bf6fd02d2b6f35bfbf4f9bece
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 7751e266ae6df52925ba2b0b3fbc72e7fff5e2e979ae00dcc8fb7af433aac22627901a650cdd53f64479af440e68a814ad3d67f446509e4aef4351e6eed71da0
|
7
|
+
data.tar.gz: 210d306f47c106bab3ae6e109c24d728baf5a81cc6caa3ee309a7c96f547c5cd8503e99cb7f814671c1feccfdaa48ea992581c9f978bb2bfd662c51946e13ffd
|
data/CHANGELOG.md
CHANGED
@@ -1,10 +1,252 @@
|
|
1
1
|
# Changelog
|
2
2
|
|
3
|
+
## [v7.1.24](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.24) (2022-06-09)
|
4
|
+
|
5
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.23...v7.1.24)
|
6
|
+
|
7
|
+
**Implemented enhancements:**
|
8
|
+
|
9
|
+
- Chore: Sidekiq 6.5 compatibility [\#715](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/715) ([mhenrixon](https://github.com/mhenrixon))
|
10
|
+
|
11
|
+
**Merged pull requests:**
|
12
|
+
|
13
|
+
- Use sidekiq/testing `Worker.clear` API in sidekiq\_unique\_jobs/testing [\#713](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/713) ([dsander](https://github.com/dsander))
|
14
|
+
|
15
|
+
## [v7.1.23](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.23) (2022-05-23)
|
16
|
+
|
17
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.22...v7.1.23)
|
18
|
+
|
19
|
+
**Fixed bugs:**
|
20
|
+
|
21
|
+
- fix: raise on error [\#712](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/712) ([mhenrixon](https://github.com/mhenrixon))
|
22
|
+
|
23
|
+
## [v7.1.22](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.22) (2022-05-04)
|
24
|
+
|
25
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.21...v7.1.22)
|
26
|
+
|
27
|
+
**Fixed bugs:**
|
28
|
+
|
29
|
+
- Failed jobs waiting to be retried are not considered when fetching uniqueness [\#394](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/394)
|
30
|
+
- fix\(locksmith\): execute to yield without arguments [\#710](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/710) ([mhenrixon](https://github.com/mhenrixon))
|
31
|
+
- fix: re:lock until\_executing on worker failure [\#709](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/709) ([mhenrixon](https://github.com/mhenrixon))
|
32
|
+
|
33
|
+
**Closed issues:**
|
34
|
+
|
35
|
+
- Reviwing: Failed jobs waiting to be retried are not considered when fetching uniqueness [\#708](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/708)
|
36
|
+
|
37
|
+
## [v7.1.21](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.21) (2022-04-23)
|
38
|
+
|
39
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.20...v7.1.21)
|
40
|
+
|
41
|
+
**Implemented enhancements:**
|
42
|
+
|
43
|
+
- Prepare for Sidekiq v7 [\#707](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/707) ([mhenrixon](https://github.com/mhenrixon))
|
44
|
+
|
45
|
+
**Closed issues:**
|
46
|
+
|
47
|
+
- DEPRECATION WARNING: default\_worker\_options is deprecated and will be removed from Sidekiq 7.0 \(use default\_job\_options instead\) \(called from notify\_agents at /Users/hackeron/Development/Tether/timeline/app/models/user.rb:303\) [\#705](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/705)
|
48
|
+
|
49
|
+
## [v7.1.20](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.20) (2022-04-22)
|
50
|
+
|
51
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.19...v7.1.20)
|
52
|
+
|
53
|
+
**Implemented enhancements:**
|
54
|
+
|
55
|
+
- Manually handle timeouts [\#706](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/706) ([mhenrixon](https://github.com/mhenrixon))
|
56
|
+
|
57
|
+
**Merged pull requests:**
|
58
|
+
|
59
|
+
- improve README wrt. middleware config [\#704](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/704) ([slhck](https://github.com/slhck))
|
60
|
+
|
61
|
+
## [v7.1.19](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.19) (2022-04-09)
|
62
|
+
|
63
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.18...v7.1.19)
|
64
|
+
|
65
|
+
**Fixed bugs:**
|
66
|
+
|
67
|
+
- concurrent-ruby 1.1.10 spikes volume of jobs [\#701](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/701)
|
68
|
+
- Reimplement the entire TimerTask as it was [\#702](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/702) ([mhenrixon](https://github.com/mhenrixon))
|
69
|
+
|
70
|
+
## [v7.1.18](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.18) (2022-04-05)
|
71
|
+
|
72
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.17...v7.1.18)
|
73
|
+
|
74
|
+
**Implemented enhancements:**
|
75
|
+
|
76
|
+
- Make sure we reflect on execution failure [\#700](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/700) ([mhenrixon](https://github.com/mhenrixon))
|
77
|
+
|
78
|
+
## [v7.1.17](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.17) (2022-04-05)
|
79
|
+
|
80
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.16...v7.1.17)
|
81
|
+
|
82
|
+
## [v7.1.16](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.16) (2022-04-02)
|
83
|
+
|
84
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.15...v7.1.16)
|
85
|
+
|
86
|
+
**Implemented enhancements:**
|
87
|
+
|
88
|
+
- Abort Ruby Reaper when sidekiq queues are full [\#690](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/690) ([francesmcmullin](https://github.com/francesmcmullin))
|
89
|
+
- Quote '3.0' to ensure CI uses Ruby 3.0.x for the 3.0 entry [\#689](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/689) ([petergoldstein](https://github.com/petergoldstein))
|
90
|
+
|
91
|
+
**Fixed bugs:**
|
92
|
+
|
93
|
+
- Hotfix: Ensure consistent lock args [\#699](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/699) ([mhenrixon](https://github.com/mhenrixon))
|
94
|
+
- Expire older changelog entries first [\#698](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/698) ([mhenrixon](https://github.com/mhenrixon))
|
95
|
+
- Fix drift [\#688](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/688) ([mhenrixon](https://github.com/mhenrixon))
|
96
|
+
|
97
|
+
**Closed issues:**
|
98
|
+
|
99
|
+
- concurrent-ruby has dropped support for TimerTask timeouts [\#697](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/697)
|
100
|
+
- Most recent changelogs are removed first [\#696](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/696)
|
101
|
+
- Improve README slightly [\#694](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/694)
|
102
|
+
- locksmith.rb:327: NoMethodError: undefined method `+' for nil:NilClass [\#686](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/686)
|
103
|
+
- lock\_timeout cannot be nil [\#675](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/675)
|
104
|
+
- Skip reaping when queues are too large [\#670](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/670)
|
105
|
+
|
106
|
+
**Merged pull requests:**
|
107
|
+
|
108
|
+
- Improve readme [\#695](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/695) ([fwolfst](https://github.com/fwolfst))
|
109
|
+
- Add funding\_uri to gemspec [\#693](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/693) ([fwolfst](https://github.com/fwolfst))
|
110
|
+
- Fix worker validator [\#685](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/685) ([victorfgs](https://github.com/victorfgs))
|
111
|
+
|
112
|
+
## [v7.1.15](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.15) (2022-02-10)
|
113
|
+
|
114
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.14...v7.1.15)
|
115
|
+
|
116
|
+
**Merged pull requests:**
|
117
|
+
|
118
|
+
- Fixing reschedule when using a non default queue [\#679](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/679) ([bigzed](https://github.com/bigzed))
|
119
|
+
|
120
|
+
## [v7.1.14](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.14) (2022-02-04)
|
121
|
+
|
122
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.13...v7.1.14)
|
123
|
+
|
124
|
+
**Implemented enhancements:**
|
125
|
+
|
126
|
+
- Fix the remaining deprecation warnings [\#681](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/681) ([mhenrixon](https://github.com/mhenrixon))
|
127
|
+
|
128
|
+
## [v7.1.13](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.13) (2022-02-03)
|
129
|
+
|
130
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.12...v7.1.13)
|
131
|
+
|
132
|
+
**Implemented enhancements:**
|
133
|
+
|
134
|
+
- Prepare for redis 5.0.0 [\#680](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/680) ([mhenrixon](https://github.com/mhenrixon))
|
135
|
+
|
136
|
+
**Fixed bugs:**
|
137
|
+
|
138
|
+
- Fix homepage url [\#667](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/667) ([dal-ioki](https://github.com/dal-ioki))
|
139
|
+
|
140
|
+
**Closed issues:**
|
141
|
+
|
142
|
+
- Job finished, but lock is not cleared [\#677](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/677)
|
143
|
+
- sidekiq\_options lock conflicts with sidekiq-lock gem lock option [\#669](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/669)
|
144
|
+
- Slow evalsha causing timeouts [\#668](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/668)
|
145
|
+
- Inconsistent documentation for config validation [\#647](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/647)
|
146
|
+
|
147
|
+
**Merged pull requests:**
|
148
|
+
|
149
|
+
- Bump bundler and friends [\#674](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/674) ([mhenrixon](https://github.com/mhenrixon))
|
150
|
+
- readme: fix minitest assertion. [\#672](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/672) ([crondaemon](https://github.com/crondaemon))
|
151
|
+
- Pass `item` in `after_unlock` callback [\#665](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/665) ([piloos](https://github.com/piloos))
|
152
|
+
|
153
|
+
## [v7.1.12](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.12) (2021-12-01)
|
154
|
+
|
155
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.11...v7.1.12)
|
156
|
+
|
157
|
+
**Implemented enhancements:**
|
158
|
+
|
159
|
+
- Improve Ruby Reaper performance under heavy load [\#663](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/663)
|
160
|
+
- Improve reaper performance under heavy load [\#666](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/666) ([mhenrixon](https://github.com/mhenrixon))
|
161
|
+
|
162
|
+
## [v7.1.11](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.11) (2021-11-30)
|
163
|
+
|
164
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.10...v7.1.11)
|
165
|
+
|
166
|
+
**Fixed bugs:**
|
167
|
+
|
168
|
+
- Fix ruby reaper edge case [\#661](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/661) ([mhenrixon](https://github.com/mhenrixon))
|
169
|
+
|
170
|
+
**Closed issues:**
|
171
|
+
|
172
|
+
- Question: Wait instead of cancelling if it is executing? [\#655](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/655)
|
173
|
+
- Expired Locks remain in zset of digests \[using "until\_expired" lock\] [\#653](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/653)
|
174
|
+
- \[Q&A\] Performance & Dead Locks [\#652](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/652)
|
175
|
+
- Sidekiq 6.3.0 includes Job module that clashes with sidekiq\_unique\_ext.rb class Job [\#651](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/651)
|
176
|
+
|
177
|
+
## [v7.1.10](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.10) (2021-10-18)
|
178
|
+
|
179
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.8...v7.1.10)
|
180
|
+
|
181
|
+
**Fixed bugs:**
|
182
|
+
|
183
|
+
- "IndexError: string not matched" when job is replaced on client [\#635](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/635)
|
184
|
+
|
185
|
+
**Merged pull requests:**
|
186
|
+
|
187
|
+
- Update URL for Sidekiq's Enterprise unique jobs [\#648](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/648) ([jsantos](https://github.com/jsantos))
|
188
|
+
|
189
|
+
## [v7.1.8](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.8) (2021-10-08)
|
190
|
+
|
191
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.13...v7.1.8)
|
192
|
+
|
193
|
+
**Fixed bugs:**
|
194
|
+
|
195
|
+
- undefined method `delete' for class `Sidekiq::Job' [\#634](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/634)
|
196
|
+
- INFO keys not persisted when job is enqueued [\#602](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/602)
|
197
|
+
- lock\_info set to true but no lock info showing up in web ui [\#589](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/589)
|
198
|
+
- Prevent too eager cleanup of lock info [\#645](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/645) ([mhenrixon](https://github.com/mhenrixon))
|
199
|
+
|
200
|
+
**Closed issues:**
|
201
|
+
|
202
|
+
- Compatibility with unreleased Sidekiq 6.3.0 [\#636](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/636)
|
203
|
+
|
204
|
+
**Merged pull requests:**
|
205
|
+
|
206
|
+
- Update docs [\#644](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/644) ([andypple](https://github.com/andypple))
|
207
|
+
|
208
|
+
## [v7.0.13](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.13) (2021-09-27)
|
209
|
+
|
210
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.7...v7.0.13)
|
211
|
+
|
212
|
+
## [v7.1.7](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.7) (2021-09-27)
|
213
|
+
|
214
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.6...v7.1.7)
|
215
|
+
|
216
|
+
**Implemented enhancements:**
|
217
|
+
|
218
|
+
- Styles [\#642](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/642) ([mhenrixon](https://github.com/mhenrixon))
|
219
|
+
|
220
|
+
**Fixed bugs:**
|
221
|
+
|
222
|
+
- OnConflict::Replace: yield when lock was achieved [\#640](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/640) ([mhenrixon](https://github.com/mhenrixon))
|
223
|
+
|
224
|
+
## [v7.1.6](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.6) (2021-09-21)
|
225
|
+
|
226
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.5...v7.1.6)
|
227
|
+
|
228
|
+
**Closed issues:**
|
229
|
+
|
230
|
+
- until\_and\_while\_executing is not running the job at all in sidekiq-unique-jobs 7.1.4 [\#626](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/626)
|
231
|
+
|
232
|
+
**Merged pull requests:**
|
233
|
+
|
234
|
+
- Necessary upgrades for Sidekiq v6.2.2 [\#639](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/639) ([mhenrixon](https://github.com/mhenrixon))
|
235
|
+
- Tese to these in README.md [\#633](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/633) ([carrickr](https://github.com/carrickr))
|
236
|
+
|
237
|
+
## [v7.1.5](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.5) (2021-07-28)
|
238
|
+
|
239
|
+
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.4...v7.1.5)
|
240
|
+
|
241
|
+
**Fixed bugs:**
|
242
|
+
|
243
|
+
- Fix: UntilAndWhileExecuting [\#627](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/627) ([mhenrixon](https://github.com/mhenrixon))
|
244
|
+
|
3
245
|
## [v7.1.4](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.4) (2021-07-21)
|
4
246
|
|
5
247
|
[Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.3...v7.1.4)
|
6
248
|
|
7
|
-
**
|
249
|
+
**Fixed bugs:**
|
8
250
|
|
9
251
|
- Pass lock timeout to primed\_async [\#624](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/624) ([millerjs](https://github.com/millerjs))
|
10
252
|
|
@@ -53,7 +295,7 @@
|
|
53
295
|
**Implemented enhancements:**
|
54
296
|
|
55
297
|
- Reflections [\#611](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/611) ([mhenrixon](https://github.com/mhenrixon))
|
56
|
-
- Start new orphan reaper process if orphan reaper is not running [\#604](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/604) ([
|
298
|
+
- Start new orphan reaper process if orphan reaper is not running [\#604](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/604) ([AlexFlint73](https://github.com/AlexFlint73))
|
57
299
|
|
58
300
|
**Fixed bugs:**
|
59
301
|
|
@@ -104,7 +346,6 @@
|
|
104
346
|
|
105
347
|
**Fixed bugs:**
|
106
348
|
|
107
|
-
- lock\_info set to true but no lock info showing up in web ui [\#589](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/589)
|
108
349
|
- Fix recording lock\_info [\#599](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/599) ([mhenrixon](https://github.com/mhenrixon))
|
109
350
|
|
110
351
|
## [v7.0.8](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.8) (2021-04-14)
|
@@ -200,7 +441,6 @@
|
|
200
441
|
- Tasks run once, and then there is no launch [\#464](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/464)
|
201
442
|
- Jobs executing and immediately returning [\#418](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/418)
|
202
443
|
- until\_and\_while\_executing + sidekiq retry mechanism [\#395](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/395)
|
203
|
-
- Failed jobs waiting to be retried are not considered when fetching uniqueness [\#394](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/394)
|
204
444
|
- Fix that :PRIMED keys are seemingly not removed [\#574](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/574) ([mhenrixon](https://github.com/mhenrixon))
|
205
445
|
|
206
446
|
**Closed issues:**
|
data/README.md
CHANGED
@@ -90,12 +90,13 @@ Want to show me some ❤️ for the hard work I do on this gem? You can use the
|
|
90
90
|
|
91
91
|
This gem adds unique constraints to sidekiq jobs. The uniqueness is achieved by creating a set of keys in redis based off of `queue`, `class`, `args` (in the sidekiq job hash).
|
92
92
|
|
93
|
-
By default, only one lock for a given hash can be acquired. What happens when a lock can't be acquired is governed by a chosen [Conflict Strategy](#conflict-strategy) strategy. Unless a conflict strategy is chosen
|
93
|
+
By default, only one lock for a given hash can be acquired. What happens when a lock can't be acquired is governed by a chosen [Conflict Strategy](#conflict-strategy) strategy. Unless a conflict strategy is chosen (?)
|
94
94
|
|
95
|
-
This is the documentation for the
|
95
|
+
This is the documentation for the `main` branch. You can find the documentation for each release by navigating to its tag.
|
96
96
|
|
97
97
|
Here are links to some of the old versions
|
98
98
|
|
99
|
+
- [v7.0.12](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.12)
|
99
100
|
- [v6.0.25](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.25)
|
100
101
|
- [v5.0.10](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v5.0.10)
|
101
102
|
- [v4.0.18](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v4.0.18)
|
@@ -122,9 +123,11 @@ Before v7, the middleware was configured automatically. Since some people report
|
|
122
123
|
|
123
124
|
*NOTE* if you want to use the reaper you also need to configure the server middleware.
|
124
125
|
|
125
|
-
[
|
126
|
+
The following shows how to modify your `config/initializers/sidekiq.rb` file to use the middleware. [Here is a full example.](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/myapp/config/initializers/sidekiq.rb#L12)
|
126
127
|
|
127
128
|
```ruby
|
129
|
+
require "sidekiq-unique-jobs"
|
130
|
+
|
128
131
|
Sidekiq.configure_server do |config|
|
129
132
|
config.redis = { url: ENV["REDIS_URL"], driver: :hiredis }
|
130
133
|
|
@@ -150,7 +153,7 @@ end
|
|
150
153
|
|
151
154
|
### Your first worker
|
152
155
|
|
153
|
-
The most likely to be
|
156
|
+
The lock type most likely to be is `:until_executed`. This type of lock creates a lock from when `UntilExecutedWorker.perform_async` is called until right after `UntilExecutedWorker.new.perform` has been called.
|
154
157
|
|
155
158
|
```ruby
|
156
159
|
# frozen_string_literal: true
|
@@ -158,8 +161,7 @@ The most likely to be used worker is `:until_executed`. This type of lock create
|
|
158
161
|
class UntilExecutedWorker
|
159
162
|
include Sidekiq::Worker
|
160
163
|
|
161
|
-
sidekiq_options
|
162
|
-
lock: :until_executed
|
164
|
+
sidekiq_options lock: :until_executed
|
163
165
|
|
164
166
|
def perform
|
165
167
|
logger.info("cowboy")
|
@@ -268,7 +270,7 @@ end
|
|
268
270
|
|
269
271
|
### While Executing
|
270
272
|
|
271
|
-
|
273
|
+
These locks are put on a queue without any type of locking mechanism, the locking doesn't happen until Sidekiq pops the job from the queue and starts processing it.
|
272
274
|
|
273
275
|
#### Example worker
|
274
276
|
|
@@ -345,9 +347,9 @@ Please not that if you try to override a default lock, an `ArgumentError` will b
|
|
345
347
|
|
346
348
|
## Conflict Strategy
|
347
349
|
|
348
|
-
Decides how we handle conflict. We can either reject the job to the dead queue or reschedule it. Both are useful for jobs that absolutely need to run and have been configured to use the lock `WhileExecuting` that is used only by the sidekiq server process.
|
350
|
+
Decides how we handle conflict. We can either `reject` the job to the dead queue or `reschedule` it. Both are useful for jobs that absolutely need to run and have been configured to use the lock `WhileExecuting` that is used only by the sidekiq server process.
|
349
351
|
|
350
|
-
|
352
|
+
Furthermore, `log` can be be used with the lock `UntilExecuted` and `UntilExpired`. Now we write a log entry saying the job could not be pushed because it is a duplicate of another job with the same arguments.
|
351
353
|
|
352
354
|
It is possible for locks to have different conflict strategy for the client and server. This is useful for `:until_and_while_executing`.
|
353
355
|
|
@@ -393,7 +395,7 @@ queue and retry the lock again.
|
|
393
395
|
This is slightly dangerous and should probably only be used for jobs that are
|
394
396
|
always scheduled in the future. Currently only attempting to retry one time.
|
395
397
|
|
396
|
-
###
|
398
|
+
### reschedule
|
397
399
|
|
398
400
|
```ruby
|
399
401
|
sidekiq_options on_conflict: :reschedule
|
@@ -567,7 +569,7 @@ The reason this happens is that the server couldn't find a valid sidekiq worker
|
|
567
569
|
|
568
570
|
### Validating Worker Configuration
|
569
571
|
|
570
|
-
Since v7 it is possible to perform some simple validation against your workers sidekiq_options
|
572
|
+
Since v7 it is possible to perform some simple validation against your workers `sidekiq_options`. What it does is scan for some issues that are known to cause problems in production.
|
571
573
|
|
572
574
|
Let's take a _bad_ worker:
|
573
575
|
|
@@ -598,12 +600,12 @@ Expected BadWorker to have valid sidekiq options but found the following problem
|
|
598
600
|
If you are not using RSpec (a lot of people prefer minitest or test unit) you can do something like:
|
599
601
|
|
600
602
|
```ruby
|
601
|
-
|
603
|
+
assert_raise(InvalidWorker){ SidekiqUniqueJobs.validate_worker!(BadWorker.get_sidekiq_options) }
|
602
604
|
```
|
603
605
|
|
604
606
|
### Uniqueness
|
605
607
|
|
606
|
-
This has been probably the most confusing part of this gem. People get really confused with how unreliable the unique jobs have been. I there for decided to do what Mike is doing for sidekiq enterprise. Read the section about unique jobs: [Enterprise unique jobs][]
|
608
|
+
This has been probably the most confusing part of this gem. People get really confused with how unreliable the unique jobs have been. I there for decided to do what Mike is doing for sidekiq enterprise. Read the section about unique jobs: [Enterprise unique jobs][](?)
|
607
609
|
|
608
610
|
```ruby
|
609
611
|
SidekiqUniqueJobs.configure do |config|
|
@@ -1042,7 +1044,7 @@ There is a [![Join the chat at https://gitter.im/mhenrixon/sidekiq-unique-jobs](
|
|
1042
1044
|
|
1043
1045
|
You can find a list of contributors over on [Contributors][]
|
1044
1046
|
|
1045
|
-
[Enterprise unique jobs]: https://
|
1047
|
+
[Enterprise unique jobs]: https://github.com/mperham/sidekiq/wiki/Ent-Unique-Jobs
|
1046
1048
|
[Contributors]: https://github.com/mhenrixon/sidekiq-unique-jobs/graphs/contributors
|
1047
1049
|
[v4.0.18]: https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v4.0.18
|
1048
1050
|
[v5.0.10]: https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v5.0.10.
|
@@ -87,10 +87,10 @@ module SidekiqUniqueJobs
|
|
87
87
|
#
|
88
88
|
def batch_delete(conn)
|
89
89
|
digests.each_slice(BATCH_SIZE) do |chunk|
|
90
|
-
conn.pipelined do
|
90
|
+
conn.pipelined do |pipeline|
|
91
91
|
chunk.each do |digest|
|
92
|
-
del_digest(
|
93
|
-
|
92
|
+
del_digest(pipeline, digest)
|
93
|
+
pipeline.zrem(SidekiqUniqueJobs::DIGESTS, digest)
|
94
94
|
@count += 1
|
95
95
|
end
|
96
96
|
end
|
@@ -99,13 +99,13 @@ module SidekiqUniqueJobs
|
|
99
99
|
@count
|
100
100
|
end
|
101
101
|
|
102
|
-
def del_digest(
|
102
|
+
def del_digest(pipeline, digest)
|
103
103
|
removable_keys = keys_for_digest(digest)
|
104
104
|
|
105
105
|
if VersionCheck.satisfied?(redis_version, ">= 4.0.0")
|
106
|
-
|
106
|
+
pipeline.unlink(*removable_keys)
|
107
107
|
else
|
108
|
-
|
108
|
+
pipeline.del(*removable_keys)
|
109
109
|
end
|
110
110
|
end
|
111
111
|
|
@@ -62,9 +62,9 @@ module SidekiqUniqueJobs
|
|
62
62
|
#
|
63
63
|
def page(cursor: 0, pattern: "*", page_size: 100)
|
64
64
|
redis do |conn|
|
65
|
-
total_size, result = conn.multi do
|
66
|
-
|
67
|
-
|
65
|
+
total_size, result = conn.multi do |pipeline|
|
66
|
+
pipeline.zcard(key)
|
67
|
+
pipeline.zscan(key, cursor, match: pattern, count: page_size)
|
68
68
|
end
|
69
69
|
|
70
70
|
[
|
@@ -196,26 +196,65 @@ module SidekiqUniqueJobs
|
|
196
196
|
)
|
197
197
|
end
|
198
198
|
|
199
|
+
#
|
200
|
+
# Set the default_lock_ttl
|
201
|
+
# @deprecated
|
202
|
+
#
|
203
|
+
# @param [Integer] obj value to set (seconds)
|
204
|
+
#
|
205
|
+
# @return [<type>] <description>
|
206
|
+
#
|
199
207
|
def default_lock_ttl=(obj)
|
200
|
-
warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated.
|
208
|
+
warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated." \
|
209
|
+
" Please use `#{class_name}#lock_ttl=` instead."
|
201
210
|
self.lock_ttl = obj
|
202
211
|
end
|
203
212
|
|
213
|
+
#
|
214
|
+
# Set new value for default_lock_timeout
|
215
|
+
# @deprecated
|
216
|
+
#
|
217
|
+
# @param [Integer] obj value to set (seconds)
|
218
|
+
#
|
219
|
+
# @return [Integer]
|
220
|
+
#
|
204
221
|
def default_lock_timeout=(obj)
|
205
|
-
warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated.
|
222
|
+
warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated." \
|
223
|
+
" Please use `#{class_name}#lock_timeout=` instead."
|
206
224
|
self.lock_timeout = obj
|
207
225
|
end
|
208
226
|
|
227
|
+
#
|
228
|
+
# Default lock TTL (Time To Live)
|
229
|
+
# @deprecated
|
230
|
+
#
|
231
|
+
# @return [nil, Integer] configured value or nil
|
232
|
+
#
|
209
233
|
def default_lock_ttl
|
210
|
-
warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated.
|
234
|
+
warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated." \
|
235
|
+
" Please use `#{class_name}#lock_ttl` instead."
|
211
236
|
lock_ttl
|
212
237
|
end
|
213
238
|
|
239
|
+
#
|
240
|
+
# Default Lock Timeout
|
241
|
+
# @deprecated
|
242
|
+
#
|
243
|
+
#
|
244
|
+
# @return [nil, Integer] configured value or nil
|
245
|
+
#
|
214
246
|
def default_lock_timeout
|
215
|
-
warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated.
|
247
|
+
warn "[DEPRECATION] `#{class_name}##{__method__}` is deprecated." \
|
248
|
+
" Please use `#{class_name}#lock_timeout` instead."
|
216
249
|
lock_timeout
|
217
250
|
end
|
218
251
|
|
252
|
+
#
|
253
|
+
# Memoized variable to get the class name
|
254
|
+
#
|
255
|
+
#
|
256
|
+
# @return [String] name of the class
|
257
|
+
#
|
219
258
|
def class_name
|
220
259
|
@class_name ||= self.class.name
|
221
260
|
end
|
@@ -6,48 +6,48 @@
|
|
6
6
|
# @author Mikael Henriksson <mikael@mhenrixon.com>
|
7
7
|
#
|
8
8
|
module SidekiqUniqueJobs
|
9
|
-
ARGS
|
10
|
-
APARTMENT
|
11
|
-
AT
|
12
|
-
CHANGELOGS
|
13
|
-
CLASS
|
14
|
-
CREATED_AT
|
15
|
-
DEAD_VERSION
|
16
|
-
DIGESTS
|
17
|
-
ERRORS
|
18
|
-
JID
|
19
|
-
LIMIT
|
20
|
-
LIVE_VERSION
|
21
|
-
LOCK
|
22
|
-
LOCK_ARGS
|
23
|
-
LOCK_ARGS_METHOD
|
24
|
-
LOCK_DIGEST
|
25
|
-
LOCK_EXPIRATION
|
26
|
-
LOCK_INFO
|
27
|
-
LOCK_LIMIT
|
28
|
-
LOCK_PREFIX
|
29
|
-
LOCK_TIMEOUT
|
30
|
-
LOCK_TTL
|
31
|
-
LOCK_TYPE
|
32
|
-
ON_CLIENT_CONFLICT
|
33
|
-
ON_CONFLICT
|
34
|
-
ON_SERVER_CONFLICT
|
35
|
-
PAYLOAD
|
36
|
-
PROCESSES
|
37
|
-
QUEUE
|
38
|
-
RETRY
|
39
|
-
SCHEDULE
|
40
|
-
TIME
|
41
|
-
TIMEOUT
|
42
|
-
TTL
|
43
|
-
TYPE
|
44
|
-
UNIQUE
|
45
|
-
UNIQUE_ACROSS_QUEUES
|
46
|
-
UNIQUE_ACROSS_WORKERS
|
47
|
-
UNIQUE_ARGS
|
48
|
-
UNIQUE_ARGS_METHOD
|
49
|
-
UNIQUE_DIGEST
|
50
|
-
UNIQUE_PREFIX
|
51
|
-
UNIQUE_REAPER
|
52
|
-
WORKER
|
9
|
+
ARGS = "args"
|
10
|
+
APARTMENT = "apartment"
|
11
|
+
AT = "at"
|
12
|
+
CHANGELOGS = "uniquejobs:changelog"
|
13
|
+
CLASS = "class"
|
14
|
+
CREATED_AT = "created_at"
|
15
|
+
DEAD_VERSION = "uniquejobs:dead"
|
16
|
+
DIGESTS = "uniquejobs:digests"
|
17
|
+
ERRORS = "errors"
|
18
|
+
JID = "jid"
|
19
|
+
LIMIT = "limit"
|
20
|
+
LIVE_VERSION = "uniquejobs:live"
|
21
|
+
LOCK = "lock"
|
22
|
+
LOCK_ARGS = "lock_args"
|
23
|
+
LOCK_ARGS_METHOD = "lock_args_method"
|
24
|
+
LOCK_DIGEST = "lock_digest"
|
25
|
+
LOCK_EXPIRATION = "lock_expiration"
|
26
|
+
LOCK_INFO = "lock_info"
|
27
|
+
LOCK_LIMIT = "lock_limit"
|
28
|
+
LOCK_PREFIX = "lock_prefix"
|
29
|
+
LOCK_TIMEOUT = "lock_timeout"
|
30
|
+
LOCK_TTL = "lock_ttl"
|
31
|
+
LOCK_TYPE = "lock_type"
|
32
|
+
ON_CLIENT_CONFLICT = "on_client_conflict"
|
33
|
+
ON_CONFLICT = "on_conflict"
|
34
|
+
ON_SERVER_CONFLICT = "on_server_conflict"
|
35
|
+
PAYLOAD = "payload"
|
36
|
+
PROCESSES = "processes"
|
37
|
+
QUEUE = "queue"
|
38
|
+
RETRY = "retry"
|
39
|
+
SCHEDULE = "schedule"
|
40
|
+
TIME = "time"
|
41
|
+
TIMEOUT = "timeout"
|
42
|
+
TTL = "ttl"
|
43
|
+
TYPE = "type"
|
44
|
+
UNIQUE = "unique"
|
45
|
+
UNIQUE_ACROSS_QUEUES = "unique_across_queues"
|
46
|
+
UNIQUE_ACROSS_WORKERS = "unique_across_workers"
|
47
|
+
UNIQUE_ARGS = "unique_args"
|
48
|
+
UNIQUE_ARGS_METHOD = "unique_args_method"
|
49
|
+
UNIQUE_DIGEST = "unique_digest"
|
50
|
+
UNIQUE_PREFIX = "unique_prefix"
|
51
|
+
UNIQUE_REAPER = "uniquejobs:reaper"
|
52
|
+
WORKER = "worker"
|
53
53
|
end
|