sidekiq-unique-jobs 7.1.28 → 8.0.0

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of sidekiq-unique-jobs might be problematic. Click here for more details.

Files changed (30) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +23 -2
  3. data/lib/sidekiq_unique_jobs/batch_delete.rb +2 -6
  4. data/lib/sidekiq_unique_jobs/changelog.rb +4 -14
  5. data/lib/sidekiq_unique_jobs/connection.rb +4 -7
  6. data/lib/sidekiq_unique_jobs/core_ext.rb +1 -1
  7. data/lib/sidekiq_unique_jobs/digests.rb +2 -12
  8. data/lib/sidekiq_unique_jobs/lock_digest.rb +1 -1
  9. data/lib/sidekiq_unique_jobs/locksmith.rb +8 -9
  10. data/lib/sidekiq_unique_jobs/lua/delete.lua +2 -5
  11. data/lib/sidekiq_unique_jobs/lua/delete_by_digest.lua +2 -5
  12. data/lib/sidekiq_unique_jobs/lua/reap_orphans.lua +2 -5
  13. data/lib/sidekiq_unique_jobs/lua/shared/_delete_from_sorted_set.lua +1 -0
  14. data/lib/sidekiq_unique_jobs/lua/unlock.lua +6 -9
  15. data/lib/sidekiq_unique_jobs/lua/upgrade.lua +0 -2
  16. data/lib/sidekiq_unique_jobs/middleware.rb +1 -1
  17. data/lib/sidekiq_unique_jobs/on_conflict/reject.rb +0 -43
  18. data/lib/sidekiq_unique_jobs/orphans/ruby_reaper.rb +4 -8
  19. data/lib/sidekiq_unique_jobs/redis/sorted_set.rb +9 -2
  20. data/lib/sidekiq_unique_jobs/reflections.rb +1 -1
  21. data/lib/sidekiq_unique_jobs/script/caller.rb +14 -8
  22. data/lib/sidekiq_unique_jobs/server.rb +0 -1
  23. data/lib/sidekiq_unique_jobs/sidekiq_unique_jobs.rb +1 -1
  24. data/lib/sidekiq_unique_jobs/upgrade_locks.rb +7 -10
  25. data/lib/sidekiq_unique_jobs/version.rb +1 -1
  26. data/lib/sidekiq_unique_jobs/web/helpers.rb +2 -2
  27. data/lib/sidekiq_unique_jobs/web/views/changelogs.erb +44 -38
  28. data/lib/sidekiq_unique_jobs/web/views/locks.erb +42 -37
  29. data/lib/sidekiq_unique_jobs/web.rb +6 -7
  30. metadata +11 -37
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 1891ac7a87d1169e9e78f9d1466d1f91971def15a48be2087e6781d6d3f37d5c
4
- data.tar.gz: e17387346df0d54d972f0335ba06ce8fd67bd4727c447e70549ae8bcb2863b51
3
+ metadata.gz: 2101cca32c043a044df29b2bcd2d8ba58054aa6162d70bdf0f6b54e1237ad3fd
4
+ data.tar.gz: eed69a742091cc23d4362f258105be1175abe25f54058a4a83178f99820bef69
5
5
  SHA512:
6
- metadata.gz: 885dd66133d98b1129423b8693051dba98c5dcc1becd7a58ab6a7175185122928cb7be829732b096443d3590f3362c2bfb7d852858e6045c89349b0aa74600ec
7
- data.tar.gz: 16102f3d498afb409453407bf0f8fc9bd92ef2a93555cf2cccd91e4a09ba4b3e640d1384becd627339eed445d5ee2ad225f58a8ffb9be25032f137d04b9bc8ef
6
+ metadata.gz: 60691cad84cc2ed74ff2edf8d6028488dfcff10b5be4c360e91873785e824f1ddef6ea0a56ce3339bc1a96f745990eb52956a85c4e0d2fde1b9893b1de66d145
7
+ data.tar.gz: 5b5ca3b64971118236c984fea50de9cf010b9e848a7a2460d4b5c000e1e288c672775c149109f29d3b1efaef1485ccb0aedb5404da23a8f500b3e82a1af37563
data/CHANGELOG.md CHANGED
@@ -1,13 +1,34 @@
1
1
  # Changelog
2
2
 
3
- ## [Unreleased](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/HEAD)
3
+ ## [v7.1.29](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.29) (2022-12-03)
4
4
 
5
- [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.27...HEAD)
5
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.28...v7.1.29)
6
+
7
+ **Fixed bugs:**
8
+
9
+ - fix\(digests\): ensure consistent digests [\#743](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/743) ([mhenrixon](https://github.com/mhenrixon))
10
+
11
+ **Merged pull requests:**
12
+
13
+ - fix\(after\_unlock\): regression from \#707 [\#737](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/737) ([adamcreekroad](https://github.com/adamcreekroad))
14
+
15
+ ## [v7.1.28](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.28) (2022-11-28)
16
+
17
+ [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.27...v7.1.28)
6
18
 
7
19
  **Fixed bugs:**
8
20
 
9
21
  - Unique Jobs Not Running with Version 7.1.26 [\#730](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/730)
10
22
 
23
+ **Closed issues:**
24
+
25
+ - Error "undefined method `redis\_info' for Sidekiq:Module" on upgrade [\#740](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/740)
26
+ - spammed by `Nothing to delete; exiting` during spec [\#733](https://github.com/mhenrixon/sidekiq-unique-jobs/issues/733)
27
+
28
+ **Merged pull requests:**
29
+
30
+ - sentence correction [\#744](https://github.com/mhenrixon/sidekiq-unique-jobs/pull/744) ([SupriyaMedankar](https://github.com/SupriyaMedankar))
31
+
11
32
  ## [v7.1.27](https://github.com/mhenrixon/sidekiq-unique-jobs/tree/v7.1.27) (2022-07-30)
12
33
 
13
34
  [Full Changelog](https://github.com/mhenrixon/sidekiq-unique-jobs/compare/v7.1.26...v7.1.27)
@@ -103,16 +103,12 @@ module SidekiqUniqueJobs
103
103
  def del_digest(pipeline, digest)
104
104
  removable_keys = keys_for_digest(digest)
105
105
 
106
- if VersionCheck.satisfied?(redis_version, ">= 4.0.0")
107
- pipeline.unlink(*removable_keys)
108
- else
109
- pipeline.del(*removable_keys)
110
- end
106
+ pipeline.unlink(*removable_keys)
111
107
  end
112
108
 
113
109
  def keys_for_digest(digest)
114
110
  [digest, "#{digest}:RUN"].each_with_object([]) do |key, digest_keys|
115
- digest_keys.concat([key])
111
+ digest_keys.push(key)
116
112
  digest_keys.concat(SUFFIXES.map { |suffix| "#{key}:#{suffix}" })
117
113
  end
118
114
  end
@@ -7,13 +7,6 @@ module SidekiqUniqueJobs
7
7
  # @author Mikael Henriksson <mikael@mhenrixon.com>
8
8
  #
9
9
  class Changelog < Redis::SortedSet
10
- #
11
- # @return [Integer] the number of matches to return by default
12
- DEFAULT_COUNT = 1_000
13
- #
14
- # @return [String] the default pattern to use for matching
15
- SCAN_PATTERN = "*"
16
-
17
10
  def initialize
18
11
  super(CHANGELOGS)
19
12
  end
@@ -42,12 +35,8 @@ module SidekiqUniqueJobs
42
35
  # @return [Array<Hash>] an array of entries
43
36
  #
44
37
  def entries(pattern: SCAN_PATTERN, count: DEFAULT_COUNT)
45
- options = {}
46
- options[:match] = pattern
47
- options[:count] = count
48
-
49
38
  redis do |conn|
50
- conn.zscan_each(key, **options).to_a.map { |entry| load_json(entry[0]) }
39
+ conn.zscan(key, match: pattern, count: count).to_a.map { |entry| load_json(entry[0]) }
51
40
  end
52
41
  end
53
42
 
@@ -67,10 +56,11 @@ module SidekiqUniqueJobs
67
56
  pipeline.zscan(key, cursor, match: pattern, count: page_size)
68
57
  end
69
58
 
59
+ # NOTE: When debugging, check the last item in the returned array.
70
60
  [
71
61
  total_size.to_i,
72
- result[0].to_i, # next_cursor
73
- result[1].map { |entry| load_json(entry[0]) }, # entries
62
+ result[0].to_i, # next_cursor
63
+ result[1].map { |entry| load_json(entry) }.select { |entry| entry.is_a?(Hash) },
74
64
  ]
75
65
  end
76
66
  end
@@ -10,13 +10,10 @@ module SidekiqUniqueJobs
10
10
  end
11
11
 
12
12
  # Creates a connection to redis
13
- # @return [Sidekiq::RedisConnection, ConnectionPool] a connection to redis
14
- def redis(r_pool = nil, &block)
15
- r_pool ||= defined?(redis_pool) ? redis_pool : r_pool
16
- if r_pool
17
- r_pool.with(&block)
18
- else
19
- Sidekiq.redis(&block)
13
+ # @return [Sidekiq::RedisConnection] a connection to redis
14
+ def redis(_r_pool = nil, &block)
15
+ Sidekiq.redis do |conn|
16
+ conn.with(&block)
20
17
  end
21
18
  end
22
19
  end
@@ -98,7 +98,7 @@ class Hash
98
98
  def _deep_transform_keys_in_object(object, &block)
99
99
  case object
100
100
  when Hash
101
- object.each_with_object({}) do |(key, value), result|
101
+ object.each_with_object(self.class.new) do |(key, value), result|
102
102
  result[yield(key)] = _deep_transform_keys_in_object(value, &block)
103
103
  end
104
104
  when Array
@@ -7,13 +7,6 @@ module SidekiqUniqueJobs
7
7
  # @author Mikael Henriksson <mikael@mhenrixon.com>
8
8
  #
9
9
  class Digests < Redis::SortedSet
10
- #
11
- # @return [Integer] the number of matches to return by default
12
- DEFAULT_COUNT = 1_000
13
- #
14
- # @return [String] the default pattern to use for matching
15
- SCAN_PATTERN = "*"
16
-
17
10
  def initialize(digests_key = DIGESTS)
18
11
  super(digests_key)
19
12
  end
@@ -77,11 +70,7 @@ module SidekiqUniqueJobs
77
70
  # @return [Array<String>] an array of digests matching the given pattern
78
71
  #
79
72
  def entries(pattern: SCAN_PATTERN, count: DEFAULT_COUNT)
80
- options = {}
81
- options[:match] = pattern
82
- options[:count] = count
83
-
84
- redis { |conn| conn.zscan_each(key, **options).to_a }.to_h
73
+ redis { |conn| conn.zscan(key, match: pattern, count: count).to_a }.to_h
85
74
  end
86
75
 
87
76
  #
@@ -100,6 +89,7 @@ module SidekiqUniqueJobs
100
89
  pipeline.zscan(key, cursor, match: pattern, count: page_size)
101
90
  end
102
91
 
92
+ # NOTE: When debugging, check the last item in the returned array.
103
93
  [
104
94
  total_size.to_i,
105
95
  digests[0].to_i, # next_cursor
@@ -51,7 +51,7 @@ module SidekiqUniqueJobs
51
51
  # Creates a namespaced unique digest based on the {#digestable_hash} and the {#lock_prefix}
52
52
  # @return [String] a unique digest
53
53
  def create_digest
54
- digest = OpenSSL::Digest::MD5.hexdigest(dump_json(digestable_hash))
54
+ digest = OpenSSL::Digest::MD5.hexdigest(dump_json(digestable_hash.sort))
55
55
  "#{lock_prefix}:#{digest}"
56
56
  end
57
57
 
@@ -240,7 +240,7 @@ module SidekiqUniqueJobs
240
240
  # @return [nil] when lock was not possible
241
241
  # @return [Object] whatever the block returns when lock was acquired
242
242
  #
243
- def primed_async(conn, wait = nil, &block)
243
+ def primed_async(conn, wait = nil, &block) # rubocop:disable Metrics/MethodLength
244
244
  timeout = (wait || config.timeout).to_i
245
245
  timeout = 1 if timeout.zero?
246
246
 
@@ -248,8 +248,12 @@ module SidekiqUniqueJobs
248
248
  concurrent_timeout = add_drift(timeout)
249
249
 
250
250
  reflect(:debug, :timeouts, item,
251
- timeouts: { brpoplpush_timeout: brpoplpush_timeout, concurrent_timeout: concurrent_timeout })
251
+ timeouts: {
252
+ brpoplpush_timeout: brpoplpush_timeout,
253
+ concurrent_timeout: concurrent_timeout,
254
+ })
252
255
 
256
+ # NOTE: When debugging, change .value to .value!
253
257
  primed_jid = Concurrent::Promises
254
258
  .future(conn) { |red_con| pop_queued(red_con, timeout) }
255
259
  .value
@@ -300,13 +304,8 @@ module SidekiqUniqueJobs
300
304
  def brpoplpush(conn, wait)
301
305
  # passing timeout 0 to brpoplpush causes it to block indefinitely
302
306
  raise InvalidArgument, "wait must be an integer" unless wait.is_a?(Integer)
303
- return conn.brpoplpush(key.queued, key.primed, wait) if conn.class.to_s == "Redis::Namespace"
304
307
 
305
- if VersionCheck.satisfied?(redis_version, ">= 6.2.0") && conn.respond_to?(:blmove)
306
- conn.blmove(key.queued, key.primed, "RIGHT", "LEFT", timeout: wait)
307
- else
308
- conn.brpoplpush(key.queued, key.primed, timeout: wait)
309
- end
308
+ conn.blmove(key.queued, key.primed, "RIGHT", "LEFT", wait)
310
309
  end
311
310
 
312
311
  #
@@ -356,7 +355,7 @@ module SidekiqUniqueJobs
356
355
  # @return [true, false]
357
356
  #
358
357
  def taken?(conn)
359
- conn.hexists(key.locked, job_id)
358
+ conn.hexists(key.locked, job_id) != 0
360
359
  end
361
360
 
362
361
  def argv
@@ -33,15 +33,12 @@ log_debug("BEGIN delete", digest)
33
33
 
34
34
  local redis_version = toversion(redisversion)
35
35
  local count = 0
36
- local del_cmd = "DEL"
37
36
 
38
37
  log_debug("ZREM", digests, digest)
39
38
  count = count + redis.call("ZREM", digests, digest)
40
39
 
41
- if redis_version["major"] >= 4 then del_cmd = "UNLINK"; end
42
-
43
- log_debug(del_cmd, digest, queued, primed, locked, info)
44
- count = count + redis.call(del_cmd, digest, queued, primed, locked, info)
40
+ log_debug("UNLINK", digest, queued, primed, locked, info)
41
+ count = count + redis.call("UNLINK", digest, queued, primed, locked, info)
45
42
 
46
43
 
47
44
  log("Deleted (" .. count .. ") keys")
@@ -25,14 +25,11 @@ local redisversion = tostring(ARGV[5])
25
25
  -------- BEGIN delete_by_digest.lua --------
26
26
  local counter = 0
27
27
  local redis_version = toversion(redisversion)
28
- local del_cmd = "DEL"
29
28
 
30
29
  log_debug("BEGIN delete_by_digest:", digest)
31
30
 
32
- if redis_version["major"] >= 4 then del_cmd = "UNLINK"; end
33
-
34
- log_debug(del_cmd, digest, queued, primed, locked, run_digest, run_queued, run_primed, run_locked)
35
- counter = redis.call(del_cmd, digest, queued, primed, locked, run_digest, run_queued, run_primed, run_locked)
31
+ log_debug("UNLINK", digest, queued, primed, locked, run_digest, run_queued, run_primed, run_locked)
32
+ counter = redis.call("UNLINK", digest, queued, primed, locked, run_digest, run_queued, run_primed, run_locked)
36
33
 
37
34
  log_debug("ZREM", digests, digest)
38
35
  redis.call("ZREM", digests, digest)
@@ -37,9 +37,6 @@ local total = redis.call("ZCARD", digests_set)
37
37
  local index = 0
38
38
  local del_count = 0
39
39
  local redis_ver = toversion(redisversion)
40
- local del_cmd = "DEL"
41
-
42
- if tonumber(redis_ver["major"]) >= 4 then del_cmd = "UNLINK"; end
43
40
 
44
41
  repeat
45
42
  log_debug("Interating through:", digests_set, "for orphaned locks")
@@ -81,7 +78,7 @@ repeat
81
78
  local run_locked = digest .. ":RUN:LOCKED"
82
79
  local run_info = digest .. ":RUN:INFO"
83
80
 
84
- redis.call(del_cmd, digest, queued, primed, locked, info, run_digest, run_queued, run_primed, run_locked, run_info)
81
+ redis.call("UNLINK", digest, queued, primed, locked, info, run_digest, run_queued, run_primed, run_locked, run_info)
85
82
 
86
83
  redis.call("ZREM", digests_set, digest)
87
84
  del_count = del_count + 1
@@ -108,7 +105,7 @@ if del_count < reaper_count then
108
105
  local run_locked = digest .. ":RUN:LOCKED"
109
106
  local run_info = digest .. ":RUN:INFO"
110
107
 
111
- redis.call(del_cmd, digest, queued, primed, locked, info, run_digest, run_queued, run_primed, run_locked, run_info)
108
+ redis.call("UNLINK", digest, queued, primed, locked, info, run_digest, run_queued, run_primed, run_locked, run_info)
112
109
 
113
110
  redis.call("ZREM", expiring_digests_set, digest)
114
111
  del_count = del_count + 1
@@ -3,6 +3,7 @@ local function delete_from_sorted_set(name, digest)
3
3
  local total = redis.call("zcard", name)
4
4
  local index = 0
5
5
  local result
6
+
6
7
  while (index < total) do
7
8
  local items = redis.call("ZRANGE", name, index, index + per -1)
8
9
  for _, item in pairs(items) do
@@ -66,13 +66,10 @@ log_debug("LREM", primed, -1, job_id)
66
66
  redis.call("LREM", primed, -1, job_id)
67
67
 
68
68
  local redis_version = toversion(redisversion)
69
- local del_cmd = "DEL"
70
-
71
- if tonumber(redis_version["major"]) >= 4 then del_cmd = "UNLINK"; end
72
69
 
73
70
  if lock_type ~= "until_expired" then
74
- log_debug(del_cmd, digest, info)
75
- redis.call(del_cmd, digest, info)
71
+ log_debug("UNLINK", digest, info)
72
+ redis.call("UNLINK", digest, info)
76
73
 
77
74
  log_debug("HDEL", locked, job_id)
78
75
  redis.call("HDEL", locked, job_id)
@@ -81,13 +78,13 @@ end
81
78
  local locked_count = redis.call("HLEN", locked)
82
79
 
83
80
  if locked_count and locked_count < 1 then
84
- log_debug(del_cmd, locked)
85
- redis.call(del_cmd, locked)
81
+ log_debug("UNLINK", locked)
82
+ redis.call("UNLINK", locked)
86
83
  end
87
84
 
88
85
  if redis.call("LLEN", primed) == 0 then
89
- log_debug(del_cmd, primed)
90
- redis.call(del_cmd, primed)
86
+ log_debug("UNLINK", primed)
87
+ redis.call("UNLINK", primed)
91
88
  end
92
89
 
93
90
  if limit and limit <= 1 and locked_count and locked_count <= 1 then
@@ -22,9 +22,7 @@ local new_version = redis.call("GET", live_version)
22
22
  local old_version = redis.call("GET", dead_version)
23
23
  local redis_version = toversion(redisversion)
24
24
  local upgraded = 0
25
- local del_cmd = "DEL"
26
25
 
27
- if redis_version["major"] >= 4 then del_cmd = "UNLINK"; end
28
26
  -------- BEGIN delete.lua --------
29
27
 
30
28
  log_debug("BEGIN upgrading from: ", old_version, "to:", new_version)
@@ -32,7 +32,7 @@ module SidekiqUniqueJobs
32
32
  @item = item
33
33
  @queue = queue
34
34
  @redis_pool = redis_pool
35
- self.job_class = item[CLASS]
35
+ self.job_class = worker_class
36
36
  return yield if unique_disabled?
37
37
 
38
38
  SidekiqUniqueJobs::Job.prepare(item) unless item[LOCK_DIGEST]
@@ -12,33 +12,6 @@ module SidekiqUniqueJobs
12
12
  def call
13
13
  log_info { "Adding dead #{item[CLASS]} job #{item[JID]}" }
14
14
 
15
- if deadset_kill?
16
- deadset_kill
17
- else
18
- push_to_deadset
19
- end
20
- end
21
-
22
- #
23
- # Sidekiq version compatibility check
24
- # @api private
25
- #
26
- #
27
- # @return [true, false] depending on if Sidekiq::Deadset responds to kill
28
- #
29
- def deadset_kill?
30
- deadset.respond_to?(:kill)
31
- end
32
-
33
- #
34
- # Use Sidekiqs built in Sidekiq::DeadSet#kill
35
- # to get rid of the job
36
- # @api private
37
- #
38
- #
39
- # @return [void]
40
- #
41
- def deadset_kill
42
15
  if kill_with_options?
43
16
  kill_job_with_options
44
17
  else
@@ -88,22 +61,6 @@ module SidekiqUniqueJobs
88
61
  @deadset ||= Sidekiq::DeadSet.new
89
62
  end
90
63
 
91
- #
92
- # Used for compatibility with older Sidekiq versions
93
- #
94
- #
95
- # @return [void]
96
- #
97
- def push_to_deadset
98
- redis do |conn|
99
- conn.multi do |pipeline|
100
- pipeline.zadd("dead", now_f, payload)
101
- pipeline.zremrangebyscore("dead", "-inf", now_f - Sidekiq::DeadSet.timeout)
102
- pipeline.zremrangebyrank("dead", 0, -Sidekiq::DeadSet.max_jobs)
103
- end
104
- end
105
- end
106
-
107
64
  #
108
65
  # The Sidekiq job hash as JSON
109
66
  #
@@ -79,11 +79,7 @@ module SidekiqUniqueJobs
79
79
  def expired_digests
80
80
  max_score = (start_time - reaper_timeout).to_f
81
81
 
82
- if VersionCheck.satisfied?(redis_version, ">= 6.2.0") && VersionCheck.satisfied?(::Redis::VERSION, ">= 4.6.0")
83
- conn.zrange(EXPIRING_DIGESTS, 0, max_score, byscore: true)
84
- else
85
- conn.zrangebyscore(EXPIRING_DIGESTS, 0, max_score)
86
- end
82
+ conn.zrange(EXPIRING_DIGESTS, 0, max_score, byscore: true)
87
83
  end
88
84
 
89
85
  #
@@ -184,7 +180,7 @@ module SidekiqUniqueJobs
184
180
 
185
181
  def active?(digest) # rubocop:disable Metrics/MethodLength, Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity
186
182
  Sidekiq.redis do |conn|
187
- procs = conn.sscan_each("processes").to_a
183
+ procs = conn.sscan("processes").to_a
188
184
  return false if procs.empty?
189
185
 
190
186
  procs.sort.each do |key|
@@ -235,7 +231,7 @@ module SidekiqUniqueJobs
235
231
  # @yield queues one at a time
236
232
  #
237
233
  def queues(conn, &block)
238
- conn.sscan_each("queues", &block)
234
+ conn.sscan("queues").each(&block)
239
235
  end
240
236
 
241
237
  def entries(conn, queue, &block) # rubocop:disable Metrics/MethodLength
@@ -290,7 +286,7 @@ module SidekiqUniqueJobs
290
286
  # @return [false] when missing
291
287
  #
292
288
  def in_sorted_set?(key, digest)
293
- conn.zscan_each(key, match: "*#{digest}*", count: 1).to_a.any?
289
+ conn.zscan(key, match: "*#{digest}*", count: 1).to_a.any?
294
290
  end
295
291
  end
296
292
  # rubocop:enable Metrics/ClassLength
@@ -8,6 +8,13 @@ module SidekiqUniqueJobs
8
8
  # @author Mikael Henriksson <mikael@mhenrixon.com>
9
9
  #
10
10
  class SortedSet < Entity
11
+ #
12
+ # @return [Integer] the number of matches to return by default
13
+ DEFAULT_COUNT = 1_000
14
+ #
15
+ # @return [String] the default pattern to use for matching
16
+ SCAN_PATTERN = "*"
17
+
11
18
  #
12
19
  # Return entries for this sorted set
13
20
  #
@@ -17,7 +24,7 @@ module SidekiqUniqueJobs
17
24
  # @return [Hash] when given with_scores: true
18
25
  #
19
26
  def entries(with_scores: true)
20
- entrys = redis { |conn| conn.zrange(key, 0, -1, with_scores: with_scores) }
27
+ entrys = redis { |conn| conn.zrange(key, 0, -1, withscores: with_scores) }
21
28
  return entrys unless with_scores
22
29
 
23
30
  entrys.each_with_object({}) { |pair, hash| hash[pair[0]] = pair[1] }
@@ -33,7 +40,7 @@ module SidekiqUniqueJobs
33
40
  def add(values)
34
41
  redis do |conn|
35
42
  if values.is_a?(Array)
36
- conn.zadd(key, values)
43
+ conn.zadd(key, *values)
37
44
  else
38
45
  conn.zadd(key, now_f, values)
39
46
  end
@@ -50,7 +50,7 @@ module SidekiqUniqueJobs
50
50
  #
51
51
  # @return [void] <description>
52
52
  #
53
- def dispatch(reflection, *args)
53
+ def dispatch(reflection, *args) # rubocop:disable Metrics/MethodLength
54
54
  if (block = @reflections[reflection])
55
55
  block.call(*args)
56
56
 
@@ -54,14 +54,9 @@ module SidekiqUniqueJobs
54
54
  # Only used to reduce a little bit of duplication
55
55
  # @see call_script
56
56
  def do_call(file_name, conn, keys, argv)
57
- argv = argv.dup.concat([
58
- now_f,
59
- debug_lua,
60
- max_history,
61
- file_name,
62
- redis_version,
63
- ])
64
- Script.execute(file_name, conn, keys: keys, argv: argv)
57
+ argv = argv.dup.push(now_f, debug_lua, max_history, file_name, redis_version)
58
+
59
+ Script.execute(file_name, conn, keys: keys, argv: normalize_argv(argv))
65
60
  end
66
61
 
67
62
  #
@@ -122,6 +117,17 @@ module SidekiqUniqueJobs
122
117
  def redis_version
123
118
  SidekiqUniqueJobs.config.redis_version
124
119
  end
120
+
121
+ def normalize_argv(argv)
122
+ argv.each_with_index do |item, index|
123
+ case item
124
+ when FalseClass, NilClass
125
+ argv[index] = 0
126
+ when TrueClass
127
+ argv[index] = 1
128
+ end
129
+ end
130
+ end
125
131
  end
126
132
  end
127
133
  end
@@ -33,7 +33,6 @@ module SidekiqUniqueJobs
33
33
  #
34
34
  def self.start
35
35
  SidekiqUniqueJobs::UpdateVersion.call
36
- SidekiqUniqueJobs::UpgradeLocks.call
37
36
  SidekiqUniqueJobs::Orphans::Manager.start
38
37
  SidekiqUniqueJobs::Orphans::ReaperResurrector.start
39
38
  end
@@ -198,7 +198,7 @@ module SidekiqUniqueJobs # rubocop:disable Metrics/ModuleLength
198
198
  # @return [String] a string like `5.0.2`
199
199
  #
200
200
  def fetch_redis_version
201
- Sidekiq.redis_info["redis_version"]
201
+ Sidekiq.default_configuration.redis_info["redis_version"]
202
202
  end
203
203
 
204
204
  #
@@ -6,7 +6,7 @@ module SidekiqUniqueJobs
6
6
  #
7
7
  # @author Mikael Henriksson <mikael@mhenrixon.com>
8
8
  #
9
- class UpgradeLocks # rubocop:disable Metrics/ClassLength
9
+ class UpgradeLocks
10
10
  #
11
11
  # @return [Integer] the number of keys to batch upgrade
12
12
  BATCH_SIZE = 100
@@ -56,9 +56,9 @@ module SidekiqUniqueJobs
56
56
 
57
57
  log_info("Start - Upgrading Locks")
58
58
 
59
- upgrade_v6_locks
60
- delete_unused_v6_keys
61
- delete_supporting_v6_keys
59
+ # upgrade_v6_locks
60
+ # delete_unused_v6_keys
61
+ # delete_supporting_v6_keys
62
62
 
63
63
  conn.hset(upgraded_key, version, now_f)
64
64
  log_info("Done - Upgrading Locks")
@@ -75,10 +75,11 @@ module SidekiqUniqueJobs
75
75
 
76
76
  def upgrade_v6_locks
77
77
  log_info("Start - Converting v6 locks to v7")
78
- conn.scan_each(match: "*:GRABBED", count: BATCH_SIZE) do |grabbed_key|
78
+ conn.scan(match: "*:GRABBED", count: BATCH_SIZE).each do |grabbed_key|
79
79
  upgrade_v6_lock(grabbed_key)
80
80
  @count += 1
81
81
  end
82
+
82
83
  log_info("Done - Converting v6 locks to v7")
83
84
  end
84
85
 
@@ -115,11 +116,7 @@ module SidekiqUniqueJobs
115
116
  return if keys.empty?
116
117
 
117
118
  conn.pipelined do |pipeline|
118
- if VersionCheck.satisfied?(redis_version, ">= 4.0.0")
119
- pipeline.unlink(*keys)
120
- else
121
- pipeline.del(*keys)
122
- end
119
+ pipeline.unlink(*keys)
123
120
  end
124
121
  end
125
122
 
@@ -3,5 +3,5 @@
3
3
  module SidekiqUniqueJobs
4
4
  #
5
5
  # @return [String] the current SidekiqUniqueJobs version
6
- VERSION = "7.1.28"
6
+ VERSION = "8.0.0"
7
7
  end
@@ -80,11 +80,11 @@ module SidekiqUniqueJobs
80
80
  #
81
81
  def cparams(options)
82
82
  stringified_options = options.transform_keys(&:to_s)
83
- params.merge(stringified_options).map do |key, value|
83
+ params.merge(stringified_options).filter_map do |key, value|
84
84
  next unless SAFE_CPARAMS.include?(key)
85
85
 
86
86
  "#{key}=#{CGI.escape(value.to_s)}"
87
- end.compact.join("&")
87
+ end.join("&")
88
88
  end
89
89
 
90
90
  #
@@ -1,54 +1,60 @@
1
1
  <header class="row">
2
2
  <div class="col-sm-5">
3
3
  <h3>
4
- <%= t('Changelog Entries') %>
4
+ <%= t('Changelog Entries') %>
5
5
  </h3>
6
6
  </div>
7
7
  <form action="<%= root_path %>changelogs" class="form form-inline" method="get">
8
8
  <%= csrf_tag %>
9
+
9
10
  <input name="filter" class="form-control" type="text" value="<%= @filter %>" />
11
+
10
12
  <button class="btn btn-default" type="submit">
11
- <%= t('Filter') %>
13
+ <%= t('Filter') %>
12
14
  </button>
13
15
  </form>
16
+
14
17
  <% if @changelogs.any? && @total_size > @count.to_i %>
15
- <div class="col-sm-4">
16
- <%= erb unique_template(:_paging), locals: { url: "#{root_path}changelogs" } %>
17
- </div>
18
+ <div class="col-sm-4">
19
+ <%= erb unique_template(:_paging), locals: { url: "#{root_path}changelogs" } %>
20
+ </div>
18
21
  <% end %>
19
22
  </header>
23
+
20
24
  <% if @changelogs.any? %>
21
- <div class="table_container">
22
- <form action="<%= root_path %>changelogs/delete_all" method="get">
23
- <input class="btn btn-danger btn-xs" type="submit" name="delete_all" value="<%= t('DeleteAll') %>" data-confirm="<%= t('AreYouSure') %>" />
24
- </form>
25
- <br/>
26
- <table class="table table-striped table-bordered table-hover">
27
- <thead>
28
- <tr>
29
- <th><%= t('Time') %></th>
30
- <th><%= t('Digest') %></th>
31
- <th><%= t('Script') %></th>
32
- <th><%= t('JID') %></th>
33
- <th><%= t('Prev JID') %></th>
34
- <th><%= t('Message') %></th>
35
- </tr>
36
- </thead>
37
- <tbody>
38
- <% @changelogs.each do |changelog| %>
39
- <tr class="changelog-row">
40
- <td><%= safe_relative_time(changelog["time"]) %></td>
41
- <td><%= changelog["digest"] %></td>
42
- <td><%= changelog["script"] %></td>
43
- <td><%= changelog["job_id"] %></td>
44
- <td><%= changelog["prev_jid"] %></td>
45
- <td><%= changelog["message"] %></th>
46
- </tr>
47
- <% end %>
48
- </tbody>
49
- </table>
50
- <form action="<%= root_path %>changelogs/delete_all" method="get">
51
- <input class="btn btn-danger btn-xs" type="submit" name="delete_all" value="<%= t('DeleteAll') %>" data-confirm="<%= t('AreYouSure') %>" />
52
- </form>
53
- </div>
25
+ <div class="table_container">
26
+ <form action="<%= root_path %>changelogs/delete_all" method="get">
27
+ <input class="btn btn-danger btn-xs" type="submit" name="delete_all" value="<%= t('DeleteAll') %>" data-confirm="<%= t('AreYouSure') %>" />
28
+ </form>
29
+ <br/>
30
+
31
+ <table class="table table-striped table-bordered table-hover">
32
+ <thead>
33
+ <tr>
34
+ <th><%= t('Time') %></th>
35
+ <th><%= t('Digest') %></th>
36
+ <th><%= t('Script') %></th>
37
+ <th><%= t('JID') %></th>
38
+ <th><%= t('Prev JID') %></th>
39
+ <th><%= t('Message') %></th>
40
+ </tr>
41
+ </thead>
42
+ <tbody>
43
+ <% @changelogs.each do |changelog| %>
44
+ <tr class="changelog-row">
45
+ <td><%= "bogus" %></td>
46
+ <td><%= changelog["digest"] %></td>
47
+ <td><%= changelog["script"] %></td>
48
+ <td><%= changelog["job_id"] %></td>
49
+ <td><%= changelog["prev_jid"] %></td>
50
+ <td><%= changelog["message"] %></th>
51
+ </tr>
52
+ <% end %>
53
+ </tbody>
54
+ </table>
55
+
56
+ <form action="<%= root_path %>changelogs/delete_all" method="get">
57
+ <input class="btn btn-danger btn-xs" type="submit" name="delete_all" value="<%= t('DeleteAll') %>" data-confirm="<%= t('AreYouSure') %>" />
58
+ </form>
59
+ </div>
54
60
  <% end %>
@@ -7,48 +7,53 @@
7
7
  <form action="<%= root_path %>locks" class="form form-inline" method="get">
8
8
  <%= csrf_tag %>
9
9
  <input name="filter" class="form-control" type="text" value="<%= @filter %>" />
10
+
10
11
  <button class="btn btn-default" type="submit">
11
- <%= t('Filter') %>
12
+ <%= t('Filter') %>
12
13
  </button>
14
+
13
15
  </form>
16
+
14
17
  <% if @locks.any? && @total_size > @count %>
15
- <div class="col-sm-4">
16
- <%= erb unique_template(:_paging), locals: { url: "#{root_path}locks" } %>
17
- </div>
18
+ <div class="col-sm-4">
19
+ <%= erb unique_template(:_paging), locals: { url: "#{root_path}locks" } %>
20
+ </div>
18
21
  <% end %>
19
22
  </header>
23
+
20
24
  <% if @locks.any? %>
21
- <div class="table_container">
22
- <table class="table table-striped table-bordered table-hover">
23
- <thead>
24
- <tr>
25
- <th><%= t('Delete') %></th>
26
- <th><%= t('Digest') %></th>
27
- <th><%= t('Lock') %></th>
28
- <th><%= t('Locks') %></th>
29
- <th><%= t('Since') %></th>
30
- </tr>
31
- </thead>
32
- <% @locks.each do |lock| %>
33
- <tbody>
34
- <tr class="lock-row">
35
- <td>
36
- <form action="<%= root_path %>locks/<%= lock.key %>/delete" method="get">
37
- <%= csrf_tag %>
38
- <input name="lock" value="<%= h lock.key %>" type="hidden" />
39
- <input class="btn btn-danger btn-xs" type="submit" name="delete" value="<%= t('Delete') %>" data-confirm="<%= t('AreYouSure') %>" />
40
- </form>
41
- </td>
42
- <td><a href="<%= root_path %>locks/<%= lock.key %>"><%= lock.key %></a></td>
43
- <td><%= lock.info["lock"] %></td>
44
- <td><%= lock.locked.count %></td>
45
- <td><%= safe_relative_time(lock.created_at) %></td>
46
- </tr>
47
- </tbody>
48
- <% end %>
49
- </table>
50
- <form action="<%= root_path %>locks/delete_all" method="get">
51
- <input class="btn btn-danger btn-xs" type="submit" name="delete_all" value="<%= t('DeleteAll') %>" data-confirm="<%= t('AreYouSure') %>" />
52
- </form>
53
- </div>
25
+ <div class="table_container">
26
+ <table class="table table-striped table-bordered table-hover">
27
+ <thead>
28
+ <tr>
29
+ <th><%= t('Delete') %></th>
30
+ <th><%= t('Digest') %></th>
31
+ <th><%= t('Lock') %></th>
32
+ <th><%= t('Locks') %></th>
33
+ <th><%= t('Since') %></th>
34
+ </tr>
35
+ </thead>
36
+ <% @locks.each do |lock| %>
37
+ <tbody>
38
+ <tr class="lock-row">
39
+ <td>
40
+ <form action="<%= root_path %>locks/<%= lock.key %>/delete" method="get">
41
+ <%= csrf_tag %>
42
+ <input name="lock" value="<%= h lock.key %>" type="hidden" />
43
+ <input class="btn btn-danger btn-xs" type="submit" name="delete" value="<%= t('Delete') %>" data-confirm="<%= t('AreYouSure') %>" />
44
+ </form>
45
+ </td>
46
+ <td><a href="<%= root_path %>locks/<%= lock.key %>"><%= lock.key %></a></td>
47
+ <td><%= lock.info["lock"] %></td>
48
+ <td><%= lock.locked.count %></td>
49
+ <td><%= safe_relative_time(lock.created_at) %></td>
50
+ </tr>
51
+ </tbody>
52
+ <% end %>
53
+ </table>
54
+
55
+ <form action="<%= root_path %>locks/delete_all" method="get">
56
+ <input class="btn btn-danger btn-xs" type="submit" name="delete_all" value="<%= t('DeleteAll') %>" data-confirm="<%= t('AreYouSure') %>" />
57
+ </form>
58
+ </div>
54
59
  <% end %>
@@ -5,7 +5,6 @@ require_relative "web/helpers"
5
5
  module SidekiqUniqueJobs
6
6
  # Utility module to help manage unique keys in redis.
7
7
  # Useful for deleting keys that for whatever reason wasn't deleted
8
- #
9
8
  # @author Mikael Henriksson <mikael@mhenrixon.com>
10
9
  module Web
11
10
  def self.registered(app) # rubocop:disable Metrics/MethodLength, Metrics/AbcSize, Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity
@@ -17,8 +16,8 @@ module SidekiqUniqueJobs
17
16
  @filter = params[:filter] || "*"
18
17
  @filter = "*" if @filter == ""
19
18
  @count = (params[:count] || 100).to_i
20
- @current_cursor = params[:cursor]
21
- @prev_cursor = params[:prev_cursor]
19
+ @current_cursor = params[:cursor].to_i
20
+ @prev_cursor = params[:prev_cursor].to_i
22
21
  @total_size, @next_cursor, @changelogs = changelog.page(
23
22
  cursor: @current_cursor,
24
23
  pattern: @filter,
@@ -37,8 +36,8 @@ module SidekiqUniqueJobs
37
36
  @filter = params[:filter] || "*"
38
37
  @filter = "*" if @filter == ""
39
38
  @count = (params[:count] || 100).to_i
40
- @current_cursor = params[:cursor]
41
- @prev_cursor = params[:prev_cursor]
39
+ @current_cursor = params[:cursor].to_i
40
+ @prev_cursor = params[:prev_cursor].to_i
42
41
 
43
42
  @total_size, @next_cursor, @locks = digests.page(
44
43
  cursor: @current_cursor,
@@ -53,8 +52,8 @@ module SidekiqUniqueJobs
53
52
  @filter = params[:filter] || "*"
54
53
  @filter = "*" if @filter == ""
55
54
  @count = (params[:count] || 100).to_i
56
- @current_cursor = params[:cursor]
57
- @prev_cursor = params[:prev_cursor]
55
+ @current_cursor = params[:cursor].to_i
56
+ @prev_cursor = params[:prev_cursor].to_i
58
57
 
59
58
  @total_size, @next_cursor, @locks = expiring_digests.page(
60
59
  cursor: @current_cursor,
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: sidekiq-unique-jobs
3
3
  version: !ruby/object:Gem::Version
4
- version: 7.1.28
4
+ version: 8.0.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Mikael Henriksson
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2022-12-03 00:00:00.000000000 Z
11
+ date: 2023-01-30 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: brpoplpush-redis_script
@@ -56,27 +56,27 @@ dependencies:
56
56
  requirements:
57
57
  - - ">="
58
58
  - !ruby/object:Gem::Version
59
- version: '5.0'
59
+ version: 7.0.0
60
60
  - - "<"
61
61
  - !ruby/object:Gem::Version
62
- version: '8.0'
62
+ version: 8.0.0
63
63
  type: :runtime
64
64
  prerelease: false
65
65
  version_requirements: !ruby/object:Gem::Requirement
66
66
  requirements:
67
67
  - - ">="
68
68
  - !ruby/object:Gem::Version
69
- version: '5.0'
69
+ version: 7.0.0
70
70
  - - "<"
71
71
  - !ruby/object:Gem::Version
72
- version: '8.0'
72
+ version: 8.0.0
73
73
  - !ruby/object:Gem::Dependency
74
74
  name: thor
75
75
  requirement: !ruby/object:Gem::Requirement
76
76
  requirements:
77
77
  - - ">="
78
78
  - !ruby/object:Gem::Version
79
- version: '0.20'
79
+ version: '1.0'
80
80
  - - "<"
81
81
  - !ruby/object:Gem::Version
82
82
  version: '3.0'
@@ -86,7 +86,7 @@ dependencies:
86
86
  requirements:
87
87
  - - ">="
88
88
  - !ruby/object:Gem::Version
89
- version: '0.20'
89
+ version: '1.0'
90
90
  - - "<"
91
91
  - !ruby/object:Gem::Version
92
92
  version: '3.0'
@@ -218,33 +218,7 @@ licenses:
218
218
  - MIT
219
219
  metadata:
220
220
  rubygems_mfa_required: 'true'
221
- post_install_message: |
222
- IMPORTANT!
223
-
224
- Automatic configuration of the sidekiq middleware is no longer done.
225
- Please see: https://github.com/mhenrixon/sidekiq-unique-jobs/blob/master/README.md#add-the-middleware
226
-
227
- This version deprecated the following sidekiq_options
228
-
229
- - sidekiq_options lock_args: :method_name
230
-
231
- It is now configured with:
232
-
233
- - sidekiq_options lock_args_method: :method_name
234
-
235
- This is also true for `Sidekiq.default_worker_options`
236
-
237
- We also deprecated the global configuration options:
238
- - default_lock_ttl
239
- - default_lock_ttl=
240
- - default_lock_timeout
241
- - default_lock_timeout=
242
-
243
- The new methods to use are:
244
- - lock_ttl
245
- - lock_ttl=
246
- - lock_timeout
247
- - lock_timeout=
221
+ post_install_message:
248
222
  rdoc_options: []
249
223
  require_paths:
250
224
  - lib
@@ -252,14 +226,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
252
226
  requirements:
253
227
  - - ">="
254
228
  - !ruby/object:Gem::Version
255
- version: '2.5'
229
+ version: '2.7'
256
230
  required_rubygems_version: !ruby/object:Gem::Requirement
257
231
  requirements:
258
232
  - - ">="
259
233
  - !ruby/object:Gem::Version
260
234
  version: '0'
261
235
  requirements: []
262
- rubygems_version: 3.3.26
236
+ rubygems_version: 3.4.2
263
237
  signing_key:
264
238
  specification_version: 4
265
239
  summary: Sidekiq middleware that prevents duplicates jobs