sidekiq-unique-jobs 7.1.6 → 7.1.30

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.

This version of sidekiq-unique-jobs might be problematic. Click here for more details.

Files changed (66) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +312 -3
  3. data/README.md +19 -15
  4. data/lib/sidekiq_unique_jobs/batch_delete.rb +8 -7
  5. data/lib/sidekiq_unique_jobs/changelog.rb +3 -3
  6. data/lib/sidekiq_unique_jobs/cli.rb +33 -8
  7. data/lib/sidekiq_unique_jobs/config.rb +48 -4
  8. data/lib/sidekiq_unique_jobs/constants.rb +45 -44
  9. data/lib/sidekiq_unique_jobs/core_ext.rb +1 -1
  10. data/lib/sidekiq_unique_jobs/deprecation.rb +30 -0
  11. data/lib/sidekiq_unique_jobs/digests.rb +8 -11
  12. data/lib/sidekiq_unique_jobs/exceptions.rb +4 -3
  13. data/lib/sidekiq_unique_jobs/expiring_digests.rb +14 -0
  14. data/lib/sidekiq_unique_jobs/job.rb +5 -0
  15. data/lib/sidekiq_unique_jobs/json.rb +7 -0
  16. data/lib/sidekiq_unique_jobs/key.rb +13 -8
  17. data/lib/sidekiq_unique_jobs/lock/base_lock.rb +30 -23
  18. data/lib/sidekiq_unique_jobs/lock/until_and_while_executing.rb +9 -3
  19. data/lib/sidekiq_unique_jobs/lock/until_executed.rb +15 -4
  20. data/lib/sidekiq_unique_jobs/lock/until_executing.rb +14 -4
  21. data/lib/sidekiq_unique_jobs/lock/until_expired.rb +13 -5
  22. data/lib/sidekiq_unique_jobs/lock/while_executing.rb +10 -6
  23. data/lib/sidekiq_unique_jobs/lock.rb +27 -10
  24. data/lib/sidekiq_unique_jobs/lock_args.rb +19 -15
  25. data/lib/sidekiq_unique_jobs/lock_config.rb +6 -6
  26. data/lib/sidekiq_unique_jobs/lock_digest.rb +7 -7
  27. data/lib/sidekiq_unique_jobs/lock_info.rb +2 -2
  28. data/lib/sidekiq_unique_jobs/lock_timeout.rb +4 -4
  29. data/lib/sidekiq_unique_jobs/lock_ttl.rb +5 -5
  30. data/lib/sidekiq_unique_jobs/lock_type.rb +37 -0
  31. data/lib/sidekiq_unique_jobs/locksmith.rb +41 -10
  32. data/lib/sidekiq_unique_jobs/logging.rb +23 -0
  33. data/lib/sidekiq_unique_jobs/lua/lock.lua +18 -12
  34. data/lib/sidekiq_unique_jobs/lua/lock_until_expired.lua +92 -0
  35. data/lib/sidekiq_unique_jobs/lua/reap_orphans.lua +31 -3
  36. data/lib/sidekiq_unique_jobs/lua/shared/_common.lua +1 -1
  37. data/lib/sidekiq_unique_jobs/lua/shared/_find_digest_in_process_set.lua +1 -1
  38. data/lib/sidekiq_unique_jobs/lua/unlock.lua +17 -5
  39. data/lib/sidekiq_unique_jobs/middleware/client.rb +3 -1
  40. data/lib/sidekiq_unique_jobs/middleware/server.rb +2 -0
  41. data/lib/sidekiq_unique_jobs/middleware.rb +4 -4
  42. data/lib/sidekiq_unique_jobs/on_conflict/reject.rb +4 -4
  43. data/lib/sidekiq_unique_jobs/on_conflict/reschedule.rb +3 -3
  44. data/lib/sidekiq_unique_jobs/options_with_fallback.rb +6 -8
  45. data/lib/sidekiq_unique_jobs/orphans/lua_reaper.rb +1 -1
  46. data/lib/sidekiq_unique_jobs/orphans/manager.rb +40 -12
  47. data/lib/sidekiq_unique_jobs/orphans/ruby_reaper.rb +99 -14
  48. data/lib/sidekiq_unique_jobs/redis/string.rb +3 -1
  49. data/lib/sidekiq_unique_jobs/reflectable.rb +11 -2
  50. data/lib/sidekiq_unique_jobs/reflections.rb +12 -1
  51. data/lib/sidekiq_unique_jobs/script/caller.rb +7 -7
  52. data/lib/sidekiq_unique_jobs/server.rb +13 -1
  53. data/lib/sidekiq_unique_jobs/sidekiq_unique_ext.rb +1 -1
  54. data/lib/sidekiq_unique_jobs/sidekiq_unique_jobs.rb +13 -3
  55. data/lib/sidekiq_unique_jobs/sidekiq_worker_methods.rb +40 -21
  56. data/lib/sidekiq_unique_jobs/testing.rb +53 -21
  57. data/lib/sidekiq_unique_jobs/timer_task.rb +266 -45
  58. data/lib/sidekiq_unique_jobs/timing.rb +1 -1
  59. data/lib/sidekiq_unique_jobs/upgrade_locks.rb +6 -6
  60. data/lib/sidekiq_unique_jobs/version.rb +1 -1
  61. data/lib/sidekiq_unique_jobs/web/helpers.rb +11 -1
  62. data/lib/sidekiq_unique_jobs/web/views/lock.erb +5 -3
  63. data/lib/sidekiq_unique_jobs/web.rb +22 -3
  64. data/lib/sidekiq_unique_jobs.rb +2 -0
  65. data/lib/tasks/changelog.rake +16 -16
  66. metadata +26 -13
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: db3824ece0f3732be10713c17098f95b761b2e8ae5b710842344393645cea160
4
- data.tar.gz: ec03e4ecf107710aa4ff3007313fde1cc558b7f2f4d577cc3ec357c0580eb869
3
+ metadata.gz: 8fec809cda45f395d8eed686c8b2ac69d59c332d04dc1a4968d66aa03b521c04
4
+ data.tar.gz: da85dc0ef4b155d3baec392d11230f7f59955ee617c1459cfaba4455e1cd4a17
5
5
  SHA512:
6
- metadata.gz: c03ff9a5fb1f30cf61181b97286f896203c7b32ed591f0b920bac9d8461402e2a7cdcf32b0a1c92488b15c01ba56631a1aec48c0e4ff73a76bf0187a3b22ef91
7
- data.tar.gz: 35447b5dc1536ba8e8f4e310e15356238cd6e78e7f76d239ef11ea90c05d5a0d397fa8f1092649cc3eeea317676a0d9fe67a09c650be0238b615e3e232b58b9b
6
+ metadata.gz: 6df2fccd762fe81f71ce9fa20d3e413ff2b4147bac2d10f2bcfbe479d420b895fe888050f8f3f980bfeee693448c052d12e9c78b5ecd524c96277e29d4277685
7
+ data.tar.gz: c60f92fdd29a4f37f5903776108f11e0c3ea1e04c5d2462700d5ee01ad57d1cdd5b20fa725ed662f95c873663ce04d932f6c90267b0cea631095285f57324de3
data/CHANGELOG.md CHANGED
@@ -1,5 +1,316 @@
1
1
  # Changelog
2
2
 
3
+ ## [Unreleased](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/HEAD)
4
+
5
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.28...HEAD)
6
+
7
+ **Fixed bugs:**
8
+
9
+ - fix\(digests\): ensure consistent digests [\#743](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/743) ([mhenrixon](https://github.com/mhenrixon))
10
+
11
+ **Merged pull requests:**
12
+
13
+ - fix\(after\_unlock\): regression from \#707 [\#737](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/737) ([adamcreekroad](https://github.com/adamcreekroad))
14
+
15
+ ## [v7.1.28](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.28) (2022-11-28)
16
+
17
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.27...v7.1.28)
18
+
19
+ **Fixed bugs:**
20
+
21
+ - Unique Jobs Not Running with Version 7.1.26 [\#730](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/730)
22
+
23
+ **Closed issues:**
24
+
25
+ - Error "undefined method `redis\_info' for Sidekiq:Module" on upgrade [\#740](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/740)
26
+ - spammed by `Nothing to delete; exiting` during spec [\#733](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/733)
27
+
28
+ **Merged pull requests:**
29
+
30
+ - sentence correction [\#744](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/744) ([SupriyaMedankar](https://github.com/SupriyaMedankar))
31
+
32
+ ## [v7.1.27](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.27) (2022-07-30)
33
+
34
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.26...v7.1.27)
35
+
36
+ **Implemented enhancements:**
37
+
38
+ - Feat\(logging\): Allow disabling logging [\#729](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/729) ([mhenrixon](https://github.com/mhenrixon))
39
+
40
+ **Fixed bugs:**
41
+
42
+ - Fix\(namespace\): Prevent self-conflict when redis-namespace is present [\#732](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/732) ([mhenrixon](https://github.com/mhenrixon))
43
+
44
+ **Closed issues:**
45
+
46
+ - Disable logging in Rails testing [\#727](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/727)
47
+ - Memory bloat / dangling keys / reaper not cleaning orphans [\#637](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/637)
48
+
49
+ ## [v7.1.26](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.26) (2022-07-28)
50
+
51
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.25...v7.1.26)
52
+
53
+ **Implemented enhancements:**
54
+
55
+ - Fix\(until\_expired\): Fix test and implementation [\#725](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/725) ([mhenrixon](https://github.com/mhenrixon))
56
+
57
+ **Fixed bugs:**
58
+
59
+ - Fix\(until\_and\_while\_executing\): Improve timeouts slightly [\#728](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/728) ([mhenrixon](https://github.com/mhenrixon))
60
+ - Fix\(unlock\): Delete primed keys on last entry [\#726](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/726) ([mhenrixon](https://github.com/mhenrixon))
61
+
62
+ **Merged pull requests:**
63
+
64
+ - Ensure batch delete removes expiring locks [\#724](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/724) ([francesmcmullin](https://github.com/francesmcmullin))
65
+ - Chore: Update dependencies [\#722](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/722) ([mhenrixon](https://github.com/mhenrixon))
66
+ - Move until\_expired digests to separate zset [\#721](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/721) ([francesmcmullin](https://github.com/francesmcmullin))
67
+ - Avoid skipping ranges when looping through queues [\#720](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/720) ([francesmcmullin](https://github.com/francesmcmullin))
68
+ - Bump actions/checkout from 2 to 3 [\#718](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/718) ([dependabot[bot]](https://github.com/apps/dependabot))
69
+ - Add Dependabot for GitHub Actions [\#717](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/717) ([petergoldstein](https://github.com/petergoldstein))
70
+ - Fix Sidekiq::Worker.clear\_all override not being applied [\#714](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/714) ([dsander](https://github.com/dsander))
71
+
72
+ ## [v7.1.25](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.25) (2022-06-13)
73
+
74
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.24...v7.1.25)
75
+
76
+ **Fixed bugs:**
77
+
78
+ - Fix: Include the correct middleware [\#716](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/716) ([mhenrixon](https://github.com/mhenrixon))
79
+
80
+ ## [v7.1.24](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.24) (2022-06-09)
81
+
82
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.23...v7.1.24)
83
+
84
+ **Implemented enhancements:**
85
+
86
+ - Chore: Sidekiq 6.5 compatibility [\#715](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/715) ([mhenrixon](https://github.com/mhenrixon))
87
+
88
+ **Merged pull requests:**
89
+
90
+ - Use sidekiq/testing `Worker.clear` API in sidekiq\_unique\_jobs/testing [\#713](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/713) ([dsander](https://github.com/dsander))
91
+
92
+ ## [v7.1.23](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.23) (2022-05-23)
93
+
94
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.22...v7.1.23)
95
+
96
+ **Fixed bugs:**
97
+
98
+ - fix: raise on error [\#712](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/712) ([mhenrixon](https://github.com/mhenrixon))
99
+
100
+ ## [v7.1.22](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.22) (2022-05-04)
101
+
102
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.21...v7.1.22)
103
+
104
+ **Fixed bugs:**
105
+
106
+ - Failed jobs waiting to be retried are not considered when fetching uniqueness [\#394](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/394)
107
+ - fix\(locksmith\): execute to yield without arguments [\#710](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/710) ([mhenrixon](https://github.com/mhenrixon))
108
+ - fix: re:lock until\_executing on worker failure [\#709](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/709) ([mhenrixon](https://github.com/mhenrixon))
109
+
110
+ **Closed issues:**
111
+
112
+ - Reviwing: Failed jobs waiting to be retried are not considered when fetching uniqueness [\#708](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/708)
113
+
114
+ ## [v7.1.21](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.21) (2022-04-23)
115
+
116
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.20...v7.1.21)
117
+
118
+ **Implemented enhancements:**
119
+
120
+ - Prepare for Sidekiq v7 [\#707](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/707) ([mhenrixon](https://github.com/mhenrixon))
121
+
122
+ **Closed issues:**
123
+
124
+ - DEPRECATION WARNING: default\_worker\_options is deprecated and will be removed from Sidekiq 7.0 \(use default\_job\_options instead\) \(called from notify\_agents at /Users/hackeron/Development/Tether/timeline/app/models/user.rb:303\) [\#705](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/705)
125
+
126
+ ## [v7.1.20](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.20) (2022-04-22)
127
+
128
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.19...v7.1.20)
129
+
130
+ **Implemented enhancements:**
131
+
132
+ - Manually handle timeouts [\#706](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/706) ([mhenrixon](https://github.com/mhenrixon))
133
+
134
+ **Merged pull requests:**
135
+
136
+ - improve README wrt. middleware config [\#704](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/704) ([slhck](https://github.com/slhck))
137
+
138
+ ## [v7.1.19](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.19) (2022-04-09)
139
+
140
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.18...v7.1.19)
141
+
142
+ **Fixed bugs:**
143
+
144
+ - concurrent-ruby 1.1.10 spikes volume of jobs [\#701](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/701)
145
+ - Reimplement the entire TimerTask as it was [\#702](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/702) ([mhenrixon](https://github.com/mhenrixon))
146
+
147
+ ## [v7.1.18](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.18) (2022-04-05)
148
+
149
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.17...v7.1.18)
150
+
151
+ **Implemented enhancements:**
152
+
153
+ - Make sure we reflect on execution failure [\#700](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/700) ([mhenrixon](https://github.com/mhenrixon))
154
+
155
+ ## [v7.1.17](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.17) (2022-04-05)
156
+
157
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.16...v7.1.17)
158
+
159
+ ## [v7.1.16](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.16) (2022-04-02)
160
+
161
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.15...v7.1.16)
162
+
163
+ **Implemented enhancements:**
164
+
165
+ - Abort Ruby Reaper when sidekiq queues are full [\#690](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/690) ([francesmcmullin](https://github.com/francesmcmullin))
166
+ - Quote '3.0' to ensure CI uses Ruby 3.0.x for the 3.0 entry [\#689](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/689) ([petergoldstein](https://github.com/petergoldstein))
167
+
168
+ **Fixed bugs:**
169
+
170
+ - Hotfix: Ensure consistent lock args [\#699](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/699) ([mhenrixon](https://github.com/mhenrixon))
171
+ - Expire older changelog entries first [\#698](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/698) ([mhenrixon](https://github.com/mhenrixon))
172
+ - Fix drift [\#688](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/688) ([mhenrixon](https://github.com/mhenrixon))
173
+
174
+ **Closed issues:**
175
+
176
+ - concurrent-ruby has dropped support for TimerTask timeouts [\#697](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/697)
177
+ - Most recent changelogs are removed first [\#696](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/696)
178
+ - Improve README slightly [\#694](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/694)
179
+ - locksmith.rb:327: NoMethodError: undefined method `+' for nil:NilClass [\#686](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/686)
180
+ - lock\_timeout cannot be nil [\#675](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/675)
181
+ - Skip reaping when queues are too large [\#670](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/670)
182
+
183
+ **Merged pull requests:**
184
+
185
+ - Improve readme [\#695](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/695) ([fwolfst](https://github.com/fwolfst))
186
+ - Add funding\_uri to gemspec [\#693](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/693) ([fwolfst](https://github.com/fwolfst))
187
+ - Fix worker validator [\#685](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/685) ([victorfgs](https://github.com/victorfgs))
188
+
189
+ ## [v7.1.15](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.15) (2022-02-10)
190
+
191
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.14...v7.1.15)
192
+
193
+ **Merged pull requests:**
194
+
195
+ - Fixing reschedule when using a non default queue [\#679](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/679) ([bigzed](https://github.com/bigzed))
196
+
197
+ ## [v7.1.14](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.14) (2022-02-04)
198
+
199
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.13...v7.1.14)
200
+
201
+ **Implemented enhancements:**
202
+
203
+ - Fix the remaining deprecation warnings [\#681](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/681) ([mhenrixon](https://github.com/mhenrixon))
204
+
205
+ ## [v7.1.13](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.13) (2022-02-03)
206
+
207
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.12...v7.1.13)
208
+
209
+ **Implemented enhancements:**
210
+
211
+ - Prepare for redis 5.0.0 [\#680](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/680) ([mhenrixon](https://github.com/mhenrixon))
212
+
213
+ **Fixed bugs:**
214
+
215
+ - Fix homepage url [\#667](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/667) ([dal-ioki](https://github.com/dal-ioki))
216
+
217
+ **Closed issues:**
218
+
219
+ - Job finished, but lock is not cleared [\#677](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/677)
220
+ - sidekiq\_options lock conflicts with sidekiq-lock gem lock option [\#669](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/669)
221
+ - Slow evalsha causing timeouts [\#668](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/668)
222
+ - Inconsistent documentation for config validation [\#647](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/647)
223
+
224
+ **Merged pull requests:**
225
+
226
+ - Bump bundler and friends [\#674](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/674) ([mhenrixon](https://github.com/mhenrixon))
227
+ - readme: fix minitest assertion. [\#672](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/672) ([crondaemon](https://github.com/crondaemon))
228
+ - Pass `item` in `after_unlock` callback [\#665](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/665) ([piloos](https://github.com/piloos))
229
+
230
+ ## [v7.1.12](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.12) (2021-12-01)
231
+
232
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.11...v7.1.12)
233
+
234
+ **Implemented enhancements:**
235
+
236
+ - Improve Ruby Reaper performance under heavy load [\#663](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/663)
237
+ - Improve reaper performance under heavy load [\#666](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/666) ([mhenrixon](https://github.com/mhenrixon))
238
+
239
+ ## [v7.1.11](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.11) (2021-11-30)
240
+
241
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.10...v7.1.11)
242
+
243
+ **Fixed bugs:**
244
+
245
+ - Fix ruby reaper edge case [\#661](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/661) ([mhenrixon](https://github.com/mhenrixon))
246
+
247
+ **Closed issues:**
248
+
249
+ - Question: Wait instead of cancelling if it is executing? [\#655](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/655)
250
+ - Expired Locks remain in zset of digests \[using "until\_expired" lock\] [\#653](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/653)
251
+ - \[Q&A\] Performance & Dead Locks [\#652](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/652)
252
+ - Sidekiq 6.3.0 includes Job module that clashes with sidekiq\_unique\_ext.rb class Job [\#651](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/651)
253
+
254
+ ## [v7.1.10](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.10) (2021-10-18)
255
+
256
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.8...v7.1.10)
257
+
258
+ **Fixed bugs:**
259
+
260
+ - "IndexError: string not matched" when job is replaced on client [\#635](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/635)
261
+
262
+ **Merged pull requests:**
263
+
264
+ - Update URL for Sidekiq's Enterprise unique jobs [\#648](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/648) ([jsantos](https://github.com/jsantos))
265
+
266
+ ## [v7.1.8](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.8) (2021-10-08)
267
+
268
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.0.13...v7.1.8)
269
+
270
+ **Fixed bugs:**
271
+
272
+ - undefined method `delete' for class `Sidekiq::Job' [\#634](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/634)
273
+ - INFO keys not persisted when job is enqueued [\#602](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/602)
274
+ - lock\_info set to true but no lock info showing up in web ui [\#589](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/589)
275
+ - Prevent too eager cleanup of lock info [\#645](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/645) ([mhenrixon](https://github.com/mhenrixon))
276
+
277
+ **Closed issues:**
278
+
279
+ - Compatibility with unreleased Sidekiq 6.3.0 [\#636](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/636)
280
+
281
+ **Merged pull requests:**
282
+
283
+ - Update docs [\#644](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/644) ([andypple](https://github.com/andypple))
284
+
285
+ ## [v7.0.13](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.13) (2021-09-27)
286
+
287
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.7...v7.0.13)
288
+
289
+ ## [v7.1.7](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.7) (2021-09-27)
290
+
291
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.6...v7.1.7)
292
+
293
+ **Implemented enhancements:**
294
+
295
+ - Styles [\#642](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/642) ([mhenrixon](https://github.com/mhenrixon))
296
+
297
+ **Fixed bugs:**
298
+
299
+ - OnConflict::Replace: yield when lock was achieved [\#640](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/640) ([mhenrixon](https://github.com/mhenrixon))
300
+
301
+ ## [v7.1.6](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.6) (2021-09-21)
302
+
303
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.5...v7.1.6)
304
+
305
+ **Closed issues:**
306
+
307
+ - until\_and\_while\_executing is not running the job at all in sidekiq-unique-jobs 7.1.4 [\#626](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/626)
308
+
309
+ **Merged pull requests:**
310
+
311
+ - Necessary upgrades for Sidekiq v6.2.2 [\#639](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/639) ([mhenrixon](https://github.com/mhenrixon))
312
+ - Tese to these in README.md [\#633](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/633) ([carrickr](https://github.com/carrickr))
313
+
3
314
  ## [v7.1.5](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.5) (2021-07-28)
4
315
 
5
316
  [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.4...v7.1.5)
@@ -61,7 +372,7 @@
61
372
  **Implemented enhancements:**
62
373
 
63
374
  - Reflections [\#611](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/611) ([mhenrixon](https://github.com/mhenrixon))
64
- - Start new orphan reaper process if orphan reaper is not running [\#604](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/604) ([Assa1121](https://github.com/Assa1121))
375
+ - Start new orphan reaper process if orphan reaper is not running [\#604](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/604) ([AlexFlint73](https://github.com/AlexFlint73))
65
376
 
66
377
  **Fixed bugs:**
67
378
 
@@ -112,7 +423,6 @@
112
423
 
113
424
  **Fixed bugs:**
114
425
 
115
- - lock\_info set to true but no lock info showing up in web ui [\#589](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/589)
116
426
  - Fix recording lock\_info [\#599](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/599) ([mhenrixon](https://github.com/mhenrixon))
117
427
 
118
428
  ## [v7.0.8](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.8) (2021-04-14)
@@ -208,7 +518,6 @@
208
518
  - Tasks run once, and then there is no launch [\#464](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/464)
209
519
  - Jobs executing and immediately returning [\#418](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/418)
210
520
  - until\_and\_while\_executing + sidekiq retry mechanism [\#395](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/395)
211
- - Failed jobs waiting to be retried are not considered when fetching uniqueness [\#394](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/394)
212
521
  - Fix that :PRIMED keys are seemingly not removed [\#574](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/574) ([mhenrixon](https://github.com/mhenrixon))
213
522
 
214
523
  **Closed issues:**
data/README.md CHANGED
@@ -31,7 +31,7 @@ Want to show me some ❤️ for the hard work I do on this gem? You can use the
31
31
  - [raise](#raise)
32
32
  - [reject](#reject)
33
33
  - [replace](#replace)
34
- - [Reschedule](#reschedule)
34
+ - [reschedule](#reschedule)
35
35
  - [Custom Strategies](#custom-strategies)
36
36
  - [3 Cleanup Dead Locks](#3-cleanup-dead-locks)
37
37
  - [Debugging](#debugging)
@@ -90,12 +90,13 @@ Want to show me some ❤️ for the hard work I do on this gem? You can use the
90
90
 
91
91
  This gem adds unique constraints to sidekiq jobs. The uniqueness is achieved by creating a set of keys in redis based off of `queue`, `class`, `args` (in the sidekiq job hash).
92
92
 
93
- By default, only one lock for a given hash can be acquired. What happens when a lock can't be acquired is governed by a chosen [Conflict Strategy](#conflict-strategy) strategy. Unless a conflict strategy is chosen
93
+ By default, only one lock for a given hash can be acquired. What happens when a lock can't be acquired is governed by a chosen [Conflict Strategy](#conflict-strategy) strategy. Unless a conflict strategy is chosen (?)
94
94
 
95
- This is the documentation for the master branch. You can find the documentation for each release by navigating to its tag.
95
+ This is the documentation for the `main` branch. You can find the documentation for each release by navigating to its tag.
96
96
 
97
97
  Here are links to some of the old versions
98
98
 
99
+ - [v7.0.12](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.0.12)
99
100
  - [v6.0.25](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v6.0.25)
100
101
  - [v5.0.10](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v5.0.10)
101
102
  - [v4.0.18](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v4.0.18)
@@ -122,9 +123,11 @@ Before v7, the middleware was configured automatically. Since some people report
122
123
 
123
124
  *NOTE* if you want to use the reaper you also need to configure the server middleware.
124
125
 
125
- [A full example](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/myapp/config/initializers/sidekiq.rb#L12)
126
+ The following shows how to modify your `config/initializers/sidekiq.rb` file to use the middleware. [Here is a full example.](https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/myapp/config/initializers/sidekiq.rb#L12)
126
127
 
127
128
  ```ruby
129
+ require "sidekiq-unique-jobs"
130
+
128
131
  Sidekiq.configure_server do |config|
129
132
  config.redis = { url: ENV["REDIS_URL"], driver: :hiredis }
130
133
 
@@ -150,7 +153,7 @@ end
150
153
 
151
154
  ### Your first worker
152
155
 
153
- The most likely to be used worker is `:until_executed`. This type of lock creates a lock from when `UntilExecutedWorker.perform_async` is called until right after `UntilExecutedWorker.new.perform` has been called.
156
+ The lock type most likely to be is `:until_executed`. This type of lock creates a lock from when `UntilExecutedWorker.perform_async` is called until right after `UntilExecutedWorker.new.perform` has been called.
154
157
 
155
158
  ```ruby
156
159
  # frozen_string_literal: true
@@ -158,8 +161,7 @@ The most likely to be used worker is `:until_executed`. This type of lock create
158
161
  class UntilExecutedWorker
159
162
  include Sidekiq::Worker
160
163
 
161
- sidekiq_options queue: :until_executed,
162
- lock: :until_executed
164
+ sidekiq_options lock: :until_executed
163
165
 
164
166
  def perform
165
167
  logger.info("cowboy")
@@ -345,9 +347,9 @@ Please not that if you try to override a default lock, an `ArgumentError` will b
345
347
 
346
348
  ## Conflict Strategy
347
349
 
348
- Decides how we handle conflict. We can either reject the job to the dead queue or reschedule it. Both are useful for jobs that absolutely need to run and have been configured to use the lock `WhileExecuting` that is used only by the sidekiq server process.
350
+ Decides how we handle conflict. We can either `reject` the job to the dead queue or `reschedule` it. Both are useful for jobs that absolutely need to run and have been configured to use the lock `WhileExecuting` that is used only by the sidekiq server process.
349
351
 
350
- The last one is log which can be be used with the lock `UntilExecuted` and `UntilExpired`. Now we write a log entry saying the job could not be pushed because it is a duplicate of another job with the same arguments.
352
+ Furthermore, `log` can be be used with the lock `UntilExecuted` and `UntilExpired`. Now we write a log entry saying the job could not be pushed because it is a duplicate of another job with the same arguments.
351
353
 
352
354
  It is possible for locks to have different conflict strategy for the client and server. This is useful for `:until_and_while_executing`.
353
355
 
@@ -362,7 +364,7 @@ sidekiq_options lock: :until_and_while_executing,
362
364
  sidekiq_options on_conflict: :log
363
365
  ```
364
366
 
365
- This strategy is intended to be used with `UntilExecuted` and `UntilExpired`. It will log a line about that this is job is a duplicate of another.
367
+ This strategy is intended to be used with `UntilExecuted` and `UntilExpired`. It will log a line that this job is a duplicate of another.
366
368
 
367
369
  ### raise
368
370
 
@@ -393,7 +395,7 @@ queue and retry the lock again.
393
395
  This is slightly dangerous and should probably only be used for jobs that are
394
396
  always scheduled in the future. Currently only attempting to retry one time.
395
397
 
396
- ### Reschedule
398
+ ### reschedule
397
399
 
398
400
  ```ruby
399
401
  sidekiq_options on_conflict: :reschedule
@@ -567,7 +569,7 @@ The reason this happens is that the server couldn't find a valid sidekiq worker
567
569
 
568
570
  ### Validating Worker Configuration
569
571
 
570
- Since v7 it is possible to perform some simple validation against your workers sidekiq_options. What it does is scan for some issues that are known to cause problems in production.
572
+ Since v7 it is possible to perform some simple validation against your workers `sidekiq_options`. What it does is scan for some issues that are known to cause problems in production.
571
573
 
572
574
  Let's take a _bad_ worker:
573
575
 
@@ -598,16 +600,17 @@ Expected BadWorker to have valid sidekiq options but found the following problem
598
600
  If you are not using RSpec (a lot of people prefer minitest or test unit) you can do something like:
599
601
 
600
602
  ```ruby
601
- assert SidekiqUniqueJobs.validate_worker!(BadWorker.get_sidekiq_options)
603
+ assert_raise(InvalidWorker){ SidekiqUniqueJobs.validate_worker!(BadWorker.get_sidekiq_options) }
602
604
  ```
603
605
 
604
606
  ### Uniqueness
605
607
 
606
- This has been probably the most confusing part of this gem. People get really confused with how unreliable the unique jobs have been. I there for decided to do what Mike is doing for sidekiq enterprise. Read the section about unique jobs: [Enterprise unique jobs][]
608
+ This has been probably the most confusing part of this gem. People get really confused with how unreliable the unique jobs have been. I there for decided to do what Mike is doing for sidekiq enterprise. Read the section about unique jobs: [Enterprise unique jobs][](?)
607
609
 
608
610
  ```ruby
609
611
  SidekiqUniqueJobs.configure do |config|
610
612
  config.enabled = !Rails.env.test?
613
+ config.logger_enabled = !Rails.env.test?
611
614
  end
612
615
  ```
613
616
 
@@ -734,6 +737,7 @@ Configure SidekiqUniqueJobs in an initializer or the sidekiq initializer on appl
734
737
  ```ruby
735
738
  SidekiqUniqueJobs.configure do |config|
736
739
  config.logger = Sidekiq.logger # default, change at your own discretion
740
+ config.logger_enabled = true # default, disable for test environments
737
741
  config.debug_lua = false # Turn on when debugging
738
742
  config.lock_info = false # Turn on when debugging
739
743
  config.lock_ttl = 600 # Expire locks after 10 minutes
@@ -1042,7 +1046,7 @@ There is a [![Join the chat at https://gitter.im/mhenrixon/sidekiq-unique-jobs](
1042
1046
 
1043
1047
  You can find a list of contributors over on [Contributors][]
1044
1048
 
1045
- [Enterprise unique jobs]: https://www.dailydrip.com/topics/sidekiq/drips/sidekiq-enterprise-unique-jobs
1049
+ [Enterprise unique jobs]: https://github.com/mperham/sidekiq/wiki/Ent-Unique-Jobs
1046
1050
  [Contributors]: https://github.com/mhenrixon/sidekiq-unique-jobs/graphs/contributors
1047
1051
  [v4.0.18]: https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v4.0.18
1048
1052
  [v5.0.10]: https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v5.0.10.
data/lib/sidekiq_unique_jobs/batch_delete.rb CHANGED
@@ -87,10 +87,11 @@ module SidekiqUniqueJobs
87
87
  #
88
88
  def batch_delete(conn)
89
89
  digests.each_slice(BATCH_SIZE) do |chunk|
90
- conn.pipelined do
90
+ conn.pipelined do |pipeline|
91
91
  chunk.each do |digest|
92
- del_digest(conn, digest)
93
- conn.zrem(SidekiqUniqueJobs::DIGESTS, digest)
92
+ del_digest(pipeline, digest)
93
+ pipeline.zrem(SidekiqUniqueJobs::DIGESTS, digest)
94
+ pipeline.zrem(SidekiqUniqueJobs::EXPIRING_DIGESTS, digest)
94
95
  @count += 1
95
96
  end
96
97
  end
@@ -99,19 +100,19 @@ module SidekiqUniqueJobs
99
100
  @count
100
101
  end
101
102
 
102
- def del_digest(conn, digest)
103
+ def del_digest(pipeline, digest)
103
104
  removable_keys = keys_for_digest(digest)
104
105
 
105
106
  if VersionCheck.satisfied?(redis_version, ">= 4.0.0")
106
- conn.unlink(*removable_keys)
107
+ pipeline.unlink(*removable_keys)
107
108
  else
108
- conn.del(*removable_keys)
109
+ pipeline.del(*removable_keys)
109
110
  end
110
111
  end
111
112
 
112
113
  def keys_for_digest(digest)
113
114
  [digest, "#{digest}:RUN"].each_with_object([]) do |key, digest_keys|
114
- digest_keys.concat([key])
115
+ digest_keys.push(key)
115
116
  digest_keys.concat(SUFFIXES.map { |suffix| "#{key}:#{suffix}" })
116
117
  end
117
118
  end
data/lib/sidekiq_unique_jobs/changelog.rb CHANGED
@@ -62,9 +62,9 @@ module SidekiqUniqueJobs
62
62
  #
63
63
  def page(cursor: 0, pattern: "*", page_size: 100)
64
64
  redis do |conn|
65
- total_size, result = conn.multi do
66
- conn.zcard(key)
67
- conn.zscan(key, cursor, match: pattern, count: page_size)
65
+ total_size, result = conn.multi do |pipeline|
66
+ pipeline.zcard(key)
67
+ pipeline.zscan(key, cursor, match: pattern, count: page_size)
68
68
  end
69
69
 
70
70
  [
data/lib/sidekiq_unique_jobs/cli.rb CHANGED
@@ -20,9 +20,11 @@ module SidekiqUniqueJobs
20
20
  option :count, aliases: :c, type: :numeric, default: 1000, desc: "The max number of digests to return"
21
21
  # :nodoc:
22
22
  def list(pattern = "*")
23
- entries = digests.entries(pattern: pattern, count: options[:count])
24
- say "Found #{entries.size} digests matching '#{pattern}':"
25
- print_in_columns(entries.sort) if entries.any?
23
+ max_count = options[:count]
24
+ say "Searching for regular digests"
25
+ list_entries(digests.entries(pattern: pattern, count: max_count), pattern)
26
+ say "Searching for expiring digests"
27
+ list_entries(expiring_digests.entries(pattern: pattern, count: max_count), pattern)
26
28
  end
27
29
 
28
30
  desc "del PATTERN", "deletes unique digests from redis by pattern"
@@ -32,11 +34,9 @@ module SidekiqUniqueJobs
32
34
  def del(pattern)
33
35
  max_count = options[:count]
34
36
  if options[:dry_run]
35
- result = digests.entries(pattern: pattern, count: max_count)
36
- say "Would delete #{result.size} digests matching '#{pattern}'"
37
+ count_entries_for_del(max_count, pattern)
37
38
  else
38
- deleted_count = digests.delete_by_pattern(pattern, count: max_count)
39
- say "Deleted #{deleted_count} digests matching '#{pattern}'"
39
+ del_entries(max_count, pattern)
40
40
  end
41
41
  end
42
42
 
@@ -51,12 +51,17 @@ module SidekiqUniqueJobs
51
51
  console_class.start
52
52
  end
53
53
 
54
- no_commands do
54
+ no_commands do # rubocop:disable Metrics/BlockLength
55
55
  # :nodoc:
56
56
  def digests
57
57
  @digests ||= SidekiqUniqueJobs::Digests.new
58
58
  end
59
59
 
60
+ # :nodoc:
61
+ def expiring_digests
62
+ @expiring_digests ||= SidekiqUniqueJobs::ExpiringDigests.new
63
+ end
64
+
60
65
  # :nodoc:
61
66
  def console_class
62
67
  require "pry"
@@ -65,6 +70,26 @@ module SidekiqUniqueJobs
65
70
  require "irb"
66
71
  IRB
67
72
  end
73
+
74
+ # :nodoc:
75
+ def list_entries(entries, pattern)
76
+ say "Found #{entries.size} digests matching '#{pattern}':"
77
+ print_in_columns(entries.sort) if entries.any?
78
+ end
79
+
80
+ # :nodoc:
81
+ def count_entries_for_del(max_count, pattern)
82
+ count = digests.entries(pattern: pattern, count: max_count).size +
83
+ expiring_digests.entries(pattern: pattern, count: max_count).size
84
+ say "Would delete #{count} digests matching '#{pattern}'"
85
+ end
86
+
87
+ # :nodoc:
88
+ def del_entries(max_count, pattern)
89
+ deleted_count = digests.delete_by_pattern(pattern, count: max_count).to_i +
90
+ expiring_digests.delete_by_pattern(pattern, count: max_count).to_i
91
+ say "Deleted #{deleted_count} digests matching '#{pattern}'"
92
+ end
68
93
  end
69
94
  end
70
95
  end