sidekiq 7.1.2 → 7.2.0

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of sidekiq might be problematic. Click here for more details.

Files changed (41) hide show
  1. checksums.yaml +4 -4
  2. data/Changes.md +69 -0
  3. data/README.md +2 -2
  4. data/lib/sidekiq/api.rb +3 -3
  5. data/lib/sidekiq/client.rb +6 -3
  6. data/lib/sidekiq/config.rb +13 -4
  7. data/lib/sidekiq/deploy.rb +1 -1
  8. data/lib/sidekiq/job_retry.rb +19 -3
  9. data/lib/sidekiq/job_util.rb +2 -0
  10. data/lib/sidekiq/metrics/query.rb +3 -1
  11. data/lib/sidekiq/metrics/shared.rb +1 -1
  12. data/lib/sidekiq/paginator.rb +2 -2
  13. data/lib/sidekiq/processor.rb +27 -26
  14. data/lib/sidekiq/rails.rb +10 -15
  15. data/lib/sidekiq/redis_client_adapter.rb +17 -2
  16. data/lib/sidekiq/redis_connection.rb +1 -0
  17. data/lib/sidekiq/scheduled.rb +1 -1
  18. data/lib/sidekiq/testing.rb +25 -6
  19. data/lib/sidekiq/version.rb +1 -1
  20. data/lib/sidekiq/web/action.rb +3 -3
  21. data/lib/sidekiq/web/application.rb +72 -6
  22. data/lib/sidekiq/web/csrf_protection.rb +1 -1
  23. data/lib/sidekiq/web/helpers.rb +31 -23
  24. data/lib/sidekiq/web.rb +13 -1
  25. data/web/assets/javascripts/application.js +16 -0
  26. data/web/assets/javascripts/dashboard-charts.js +17 -1
  27. data/web/assets/javascripts/dashboard.js +7 -9
  28. data/web/assets/javascripts/metrics.js +34 -0
  29. data/web/assets/stylesheets/application.css +9 -0
  30. data/web/locales/en.yml +2 -0
  31. data/web/locales/pt-br.yml +20 -0
  32. data/web/views/_job_info.erb +1 -1
  33. data/web/views/_metrics_period_select.erb +1 -1
  34. data/web/views/_summary.erb +7 -7
  35. data/web/views/busy.erb +3 -3
  36. data/web/views/dashboard.erb +23 -33
  37. data/web/views/filtering.erb +7 -0
  38. data/web/views/metrics.erb +36 -27
  39. data/web/views/metrics_for_job.erb +26 -35
  40. data/web/views/queues.erb +6 -2
  41. metadata +4 -3
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 587dc2a304b24daba86943907fe6b0285ecd302119e900e68403fcf0dedff163
4
- data.tar.gz: 713489d61003baf22b07879e959048bede62a5b661cf37a6c43540cd3cd59eac
3
+ metadata.gz: 30f824346db9b0ebf8ee13c6ac0101494e5fd6d05b4ed2ef3ad97c5b17ccbc10
4
+ data.tar.gz: bed22f02925116256550bbc34ef9decd70bdbca356d5d61abedd4d14a7dcac45
5
5
  SHA512:
6
- metadata.gz: 026b2dd393d8e9774f1afa6fe4df9cfca70520e84a0d566dde668e535809492476004f205668d16bbe2d5f0111fb4d51a4ddddd60bfd6a1c90d906bcb50b5c92
7
- data.tar.gz: 5b6769ef293543a4d7003e870c5ed5b1d9b7d44a941c4fc446afd16531fc1eaae2f684359fb7c093eb1761c517ff31c106f247c17a8e2808bb629adf47061a62
6
+ metadata.gz: 347e82cf6f215a1e4bd09c3f12d382be79d96bb83ccd1d09f928db19b936c2dd72c0f2e3a8988160086da7928a7911d84eddd005428c7b6a337763c4b60a3492
7
+ data.tar.gz: ef0a03f45d4d35e832f36b7a09ae0694838cd0cf3582dc20d53956bb5a61ba07811d410600b43a9bdf777e68dd2549910a125d23885afca8388b8513bb7ac8d1
data/Changes.md CHANGED
@@ -2,6 +2,70 @@
2
2
 
3
3
  [Sidekiq Changes](https://github.com/sidekiq/sidekiq/blob/main/Changes.md) | [Sidekiq Pro Changes](https://github.com/sidekiq/sidekiq/blob/main/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/sidekiq/sidekiq/blob/main/Ent-Changes.md)
4
4
 
5
+ 7.2.0
6
+ ----------
7
+
8
+ - `sidekiq_retries_exhausted` can return `:discard` to avoid the deadset
9
+ and all death handlers [#6091]
10
+ - Metrics filtering by job class in Web UI [#5974]
11
+ - Better readability and formatting for numbers within the Web UI [#6080]
12
+ - Add explicit error if user code tries to nest test modes [#6078]
13
+ ```ruby
14
+ Sidekiq::Testing.inline! # global setting
15
+ Sidekiq::Testing.fake! do # override within block
16
+ # ok
17
+ Sidekiq::Testing.inline! do # can't override the override
18
+ # not ok, nested
19
+ end
20
+ end
21
+ ```
22
+ - **SECURITY** Forbid inline JavaScript execution in Web UI [#6074]
23
+ - Adjust redis-client adapter to avoid `method_missing` [#6083]
24
+ This can result in app code breaking if your app's Redis API usage was
25
+ depending on Sidekiq's adapter to correct invalid redis-client API usage.
26
+ One example:
27
+ ```ruby
28
+ # bad, not redis-client native
29
+ # Unsupported command argument type: TrueClass (TypeError)
30
+ Sidekiq.redis { |c| c.set("key", "value", nx: true, ex: 15) }
31
+ # good
32
+ Sidekiq.redis { |c| c.set("key", "value", "nx", "ex", 15) }
33
+ ```
34
+
35
+ 7.1.6
36
+ ----------
37
+
38
+ - The block forms of testing modes (inline, fake) are now thread-safe so you can have
39
+ a multithreaded test suite which uses different modes for different tests. [#6069]
40
+ - Fix breakage with non-Proc error handlers [#6065]
41
+
42
+ 7.1.5
43
+ ----------
44
+
45
+ - **FEATURE**: Job filtering within the Web UI. This feature has been open
46
+ sourced from Sidekiq Pro. [#6052]
47
+ - **API CHANGE** Error handlers now take three arguments `->(ex, context, config)`.
48
+ The previous calling convention will work until Sidekiq 8.0 but will print
49
+ out a deprecation warning. [#6051]
50
+ - Fix issue with the `batch_size` and `at` options in `S::Client.push_bulk` [#6040]
51
+ - Fix inline testing firing batch callbacks early [#6057]
52
+ - Use new log broadcast API in Rails 7.1 [#6054]
53
+ - Crash if user tries to use RESP2 `protocol: 2` [#6061]
54
+
55
+ 7.1.4
56
+ ----------
57
+
58
+ - Fix empty `retry_for` logic [#6035]
59
+
60
+ 7.1.3
61
+ ----------
62
+
63
+ - Add `sidekiq_options retry_for: 48.hours` to allow time-based retry windows [#6029]
64
+ - Support sidekiq_retry_in and sidekiq_retries_exhausted_block in ActiveJobs (#5994)
65
+ - Lowercase all Rack headers for Rack 3.0 [#5951]
66
+ - Validate Sidekiq::Web page refresh delay to avoid potential DoS,
67
+ CVE-2023-26141, thanks for reporting Keegan!
68
+
5
69
  7.1.2
6
70
  ----------
7
71
 
@@ -116,6 +180,11 @@ end
116
180
  - Job Execution metrics!!!
117
181
  - See `docs/7.0-Upgrade.md` for release notes
118
182
 
183
+ 6.5.{10,11,12}
184
+ ----------
185
+
186
+ - Fixes for Rails 7.1 [#6067, #6070]
187
+
119
188
  6.5.9
120
189
  ----------
121
190
 
data/README.md CHANGED
@@ -83,7 +83,7 @@ You can purchase at https://sidekiq.org; email support@contribsys.com for help.
83
83
  Useful resources:
84
84
 
85
85
  * Product documentation is in the [wiki](https://github.com/sidekiq/sidekiq/wiki).
86
- * Occasional announcements are made to the [@sidekiq](https://twitter.com/sidekiq) Twitter account.
86
+ * Occasional announcements are made to the [@sidekiq](https://ruby.social/@sidekiq) Mastodon account.
87
87
  * The [Sidekiq tag](https://stackoverflow.com/questions/tagged/sidekiq) on Stack Overflow has lots of useful Q & A.
88
88
 
89
89
  Every Friday morning is Sidekiq office hour: I video chat and answer questions.
@@ -103,4 +103,4 @@ The license for Sidekiq Pro and Sidekiq Enterprise can be found in [COMM-LICENSE
103
103
  Author
104
104
  -----------------
105
105
 
106
- Mike Perham, [@getajobmike](https://twitter.com/getajobmike) / [@sidekiq](https://twitter.com/sidekiq), [https://www.mikeperham.com](https://www.mikeperham.com) / [https://www.contribsys.com](https://www.contribsys.com)
106
+ Mike Perham, [@getajobmike](https://ruby.social/@getajobmike) / [@sidekiq](https://ruby.social/@sidekiq), [https://www.mikeperham.com](https://www.mikeperham.com) / [https://www.contribsys.com](https://www.contribsys.com)
data/lib/sidekiq/api.rb CHANGED
@@ -679,7 +679,7 @@ module Sidekiq
679
679
  range_start = page * page_size + offset_size
680
680
  range_end = range_start + page_size - 1
681
681
  elements = Sidekiq.redis { |conn|
682
- conn.zrange name, range_start, range_end, withscores: true
682
+ conn.zrange name, range_start, range_end, "withscores"
683
683
  }
684
684
  break if elements.empty?
685
685
  page -= 1
@@ -706,7 +706,7 @@ module Sidekiq
706
706
  end
707
707
 
708
708
  elements = Sidekiq.redis { |conn|
709
- conn.zrange(name, begin_score, end_score, "BYSCORE", withscores: true)
709
+ conn.zrange(name, begin_score, end_score, "BYSCORE", "withscores")
710
710
  }
711
711
 
712
712
  elements.each_with_object([]) do |element, result|
@@ -881,7 +881,7 @@ module Sidekiq
881
881
  # @api private
882
882
  def cleanup
883
883
  # dont run cleanup more than once per minute
884
- return 0 unless Sidekiq.redis { |conn| conn.set("process_cleanup", "1", nx: true, ex: 60) }
884
+ return 0 unless Sidekiq.redis { |conn| conn.set("process_cleanup", "1", "NX", "EX", "60") }
885
885
 
886
886
  count = 0
887
887
  Sidekiq.redis do |conn|
@@ -66,6 +66,7 @@ module Sidekiq
66
66
  # args - an array of simple arguments to the perform method, must be JSON-serializable
67
67
  # at - timestamp to schedule the job (optional), must be Numeric (e.g. Time.now.to_f)
68
68
  # retry - whether to retry this job if it fails, default true or an integer number of retries
69
+ # retry_for - relative amount of time to retry this job if it fails, default nil
69
70
  # backtrace - whether to save any error backtrace, default false
70
71
  #
71
72
  # If class is set to the class name, the jobs' options will be based on Sidekiq's default
@@ -73,7 +74,7 @@ module Sidekiq
73
74
  #
74
75
  # Any options valid for a job class's sidekiq_options are also available here.
75
76
  #
76
- # All options must be strings, not symbols. NB: because we are serializing to JSON, all
77
+ # All keys must be strings, not symbols. NB: because we are serializing to JSON, all
77
78
  # symbols in 'args' will be converted to strings. Note that +backtrace: true+ can take quite a bit of
78
79
  # space in Redis; a large volume of failing jobs can start Redis swapping if you aren't careful.
79
80
  #
@@ -110,7 +111,7 @@ module Sidekiq
110
111
  # prevented a job push.
111
112
  #
112
113
  # Example (pushing jobs in batches):
113
- # push_bulk('class' => 'MyJob', 'args' => (1..100_000).to_a, batch_size: 1_000)
114
+ # push_bulk('class' => MyJob, 'args' => (1..100_000).to_a, batch_size: 1_000)
114
115
  #
115
116
  def push_bulk(items)
116
117
  batch_size = items.delete(:batch_size) || items.delete("batch_size") || 1_000
@@ -123,19 +124,21 @@ module Sidekiq
123
124
  raise ArgumentError, "Explicitly passing 'jid' when pushing more than one job is not supported" if jid && args.size > 1
124
125
 
125
126
  normed = normalize_item(items)
127
+ slice_index = 0
126
128
  result = args.each_slice(batch_size).flat_map do |slice|
127
129
  raise ArgumentError, "Bulk arguments must be an Array of Arrays: [[1], [2]]" unless slice.is_a?(Array) && slice.all?(Array)
128
130
  break [] if slice.empty? # no jobs to push
129
131
 
130
132
  payloads = slice.map.with_index { |job_args, index|
131
133
  copy = normed.merge("args" => job_args, "jid" => SecureRandom.hex(12))
132
- copy["at"] = (at.is_a?(Array) ? at[index] : at) if at
134
+ copy["at"] = (at.is_a?(Array) ? at[slice_index + index] : at) if at
133
135
  result = middleware.invoke(items["class"], copy, copy["queue"], @redis_pool) do
134
136
  verify_json(copy)
135
137
  copy
136
138
  end
137
139
  result || nil
138
140
  }
141
+ slice_index += batch_size
139
142
 
140
143
  to_push = payloads.compact
141
144
  raw_push(to_push) unless to_push.empty?
@@ -34,8 +34,7 @@ module Sidekiq
34
34
  backtrace_cleaner: ->(backtrace) { backtrace }
35
35
  }
36
36
 
37
- ERROR_HANDLER = ->(ex, ctx) {
38
- cfg = ctx[:_config] || Sidekiq.default_configuration
37
+ ERROR_HANDLER = ->(ex, ctx, cfg = Sidekiq.default_configuration) {
39
38
  l = cfg.logger
40
39
  l.warn(Sidekiq.dump_json(ctx)) unless ctx.empty?
41
40
  l.warn("#{ex.class.name}: #{ex.message}")
@@ -259,14 +258,24 @@ module Sidekiq
259
258
  @logger = logger
260
259
  end
261
260
 
261
+ private def parameter_size(handler)
262
+ target = handler.is_a?(Proc) ? handler : handler.method(:call)
263
+ target.parameters.size
264
+ end
265
+
262
266
  # INTERNAL USE ONLY
263
267
  def handle_exception(ex, ctx = {})
264
268
  if @options[:error_handlers].size == 0
265
269
  p ["!!!!!", ex]
266
270
  end
267
- ctx[:_config] = self
268
271
  @options[:error_handlers].each do |handler|
269
- handler.call(ex, ctx)
272
+ if parameter_size(handler) == 2
273
+ # TODO Remove in 8.0
274
+ logger.info { "DEPRECATION: Sidekiq exception handlers now take three arguments, see #{handler}" }
275
+ handler.call(ex, {_config: self}.merge(ctx))
276
+ else
277
+ handler.call(ex, ctx, self)
278
+ end
270
279
  rescue Exception => e
271
280
  l = logger
272
281
  l.error "!!! ERROR HANDLER THREW AN ERROR !!!"
@@ -44,7 +44,7 @@ module Sidekiq
44
44
 
45
45
  @pool.with do |c|
46
46
  # only allow one deploy mark for a given label for the next minute
47
- lock = c.set("deploylock-#{label}", stamp, nx: true, ex: 60)
47
+ lock = c.set("deploylock-#{label}", stamp, "nx", "ex", "60")
48
48
  if lock
49
49
  c.multi do |pipe|
50
50
  pipe.hsetnx(key, stamp, label)
@@ -170,9 +170,11 @@ module Sidekiq
170
170
  msg["error_backtrace"] = compress_backtrace(lines)
171
171
  end
172
172
 
173
- # Goodbye dear message, you (re)tried your best I'm sure.
174
173
  return retries_exhausted(jobinst, msg, exception) if count >= max_retry_attempts
175
174
 
175
+ rf = msg["retry_for"]
176
+ return retries_exhausted(jobinst, msg, exception) if rf && ((msg["failed_at"] + rf) < Time.now.to_f)
177
+
176
178
  strategy, delay = delay_for(jobinst, count, exception, msg)
177
179
  case strategy
178
180
  when :discard
@@ -197,7 +199,14 @@ module Sidekiq
197
199
  # sidekiq_retry_in can return two different things:
198
200
  # 1. When to retry next, as an integer of seconds
199
201
  # 2. A symbol which re-routes the job elsewhere, e.g. :discard, :kill, :default
200
- jobinst&.sidekiq_retry_in_block&.call(count, exception, msg)
202
+ block = jobinst&.sidekiq_retry_in_block
203
+
204
+ # the sidekiq_retry_in_block can be defined in a wrapped class (ActiveJob for instance)
205
+ unless msg["wrapped"].nil?
206
+ wrapped = Object.const_get(msg["wrapped"])
207
+ block = wrapped.respond_to?(:sidekiq_retry_in_block) ? wrapped.sidekiq_retry_in_block : nil
208
+ end
209
+ block&.call(count, exception, msg)
201
210
  rescue Exception => e
202
211
  handle_exception(e, {context: "Failure scheduling retry using the defined `sidekiq_retry_in` in #{jobinst.class.name}, falling back to default"})
203
212
  nil
@@ -217,13 +226,20 @@ module Sidekiq
217
226
  end
218
227
 
219
228
  def retries_exhausted(jobinst, msg, exception)
220
- begin
229
+ rv = begin
221
230
  block = jobinst&.sidekiq_retries_exhausted_block
231
+
232
+ # the sidekiq_retries_exhausted_block can be defined in a wrapped class (ActiveJob for instance)
233
+ unless msg["wrapped"].nil?
234
+ wrapped = Object.const_get(msg["wrapped"])
235
+ block = wrapped.respond_to?(:sidekiq_retries_exhausted_block) ? wrapped.sidekiq_retries_exhausted_block : nil
236
+ end
222
237
  block&.call(msg, exception)
223
238
  rescue => e
224
239
  handle_exception(e, {context: "Error calling retries_exhausted", job: msg})
225
240
  end
226
241
 
242
+ return if rv == :discard # poof!
227
243
  send_to_morgue(msg) unless msg["dead"] == false
228
244
 
229
245
  @capsule.config.death_handlers.each do |handler|
@@ -13,6 +13,7 @@ module Sidekiq
13
13
  raise(ArgumentError, "Job class must be either a Class or String representation of the class name: `#{item}`") unless item["class"].is_a?(Class) || item["class"].is_a?(String)
14
14
  raise(ArgumentError, "Job 'at' must be a Numeric timestamp: `#{item}`") if item.key?("at") && !item["at"].is_a?(Numeric)
15
15
  raise(ArgumentError, "Job tags must be an Array: `#{item}`") if item["tags"] && !item["tags"].is_a?(Array)
16
+ raise(ArgumentError, "retry_for must be a relative amount of time, e.g. 48.hours `#{item}`") if item["retry_for"] && item["retry_for"] > 1_000_000_000
16
17
  end
17
18
 
18
19
  def verify_json(item)
@@ -54,6 +55,7 @@ module Sidekiq
54
55
  item["jid"] ||= SecureRandom.hex(12)
55
56
  item["class"] = item["class"].to_s
56
57
  item["queue"] = item["queue"].to_s
58
+ item["retry_for"] = item["retry_for"].to_i if item["retry_for"]
57
59
  item["created_at"] ||= Time.now.to_f
58
60
  item
59
61
  end
@@ -20,7 +20,8 @@ module Sidekiq
20
20
  end
21
21
 
22
22
  # Get metric data for all jobs from the last hour
23
- def top_jobs(minutes: 60)
23
+ # +class_filter+: return only results for classes matching filter
24
+ def top_jobs(class_filter: nil, minutes: 60)
24
25
  result = Result.new
25
26
 
26
27
  time = @time
@@ -39,6 +40,7 @@ module Sidekiq
39
40
  redis_results.each do |hash|
40
41
  hash.each do |k, v|
41
42
  kls, metric = k.split("|")
43
+ next if class_filter && !class_filter.match?(kls)
42
44
  result.job_results[kls].add_metric metric, time, v.to_i
43
45
  end
44
46
  time -= 60
@@ -73,7 +73,7 @@ module Sidekiq
73
73
  def fetch(conn, now = Time.now)
74
74
  window = now.utc.strftime("%d-%H:%-M")
75
75
  key = "#{@klass}-#{window}"
76
- conn.bitfield(key, *FETCH)
76
+ conn.bitfield_ro(key, *FETCH)
77
77
  end
78
78
 
79
79
  def persist(conn, now = Time.now)
@@ -19,9 +19,9 @@ module Sidekiq
19
19
  total_size, items = conn.multi { |transaction|
20
20
  transaction.zcard(key)
21
21
  if rev
22
- transaction.zrange(key, starting, ending, "REV", withscores: true)
22
+ transaction.zrange(key, starting, ending, "REV", "withscores")
23
23
  else
24
- transaction.zrange(key, starting, ending, withscores: true)
24
+ transaction.zrange(key, starting, ending, "withscores")
25
25
  end
26
26
  }
27
27
  [current_page, total_size, items]
@@ -148,6 +148,8 @@ module Sidekiq
148
148
 
149
149
  IGNORE_SHUTDOWN_INTERRUPTS = {Sidekiq::Shutdown => :never}
150
150
  private_constant :IGNORE_SHUTDOWN_INTERRUPTS
151
+ ALLOW_SHUTDOWN_INTERRUPTS = {Sidekiq::Shutdown => :immediate}
152
+ private_constant :ALLOW_SHUTDOWN_INTERRUPTS
151
153
 
152
154
  def process(uow)
153
155
  jobstr = uow.job
@@ -171,36 +173,35 @@ module Sidekiq
171
173
  end
172
174
 
173
175
  ack = false
174
- begin
175
- dispatch(job_hash, queue, jobstr) do |inst|
176
- config.server_middleware.invoke(inst, job_hash, queue) do
177
- execute_job(inst, job_hash["args"])
176
+ Thread.handle_interrupt(IGNORE_SHUTDOWN_INTERRUPTS) do
177
+ Thread.handle_interrupt(ALLOW_SHUTDOWN_INTERRUPTS) do
178
+ dispatch(job_hash, queue, jobstr) do |inst|
179
+ config.server_middleware.invoke(inst, job_hash, queue) do
180
+ execute_job(inst, job_hash["args"])
181
+ end
178
182
  end
183
+ ack = true
184
+ rescue Sidekiq::Shutdown
185
+ # Had to force kill this job because it didn't finish
186
+ # within the timeout. Don't acknowledge the work since
187
+ # we didn't properly finish it.
188
+ rescue Sidekiq::JobRetry::Handled => h
189
+ # this is the common case: job raised error and Sidekiq::JobRetry::Handled
190
+ # signals that we created a retry successfully. We can acknowlege the job.
191
+ ack = true
192
+ e = h.cause || h
193
+ handle_exception(e, {context: "Job raised exception", job: job_hash})
194
+ raise e
195
+ rescue Exception => ex
196
+ # Unexpected error! This is very bad and indicates an exception that got past
197
+ # the retry subsystem (e.g. network partition). We won't acknowledge the job
198
+ # so it can be rescued when using Sidekiq Pro.
199
+ handle_exception(ex, {context: "Internal exception!", job: job_hash, jobstr: jobstr})
200
+ raise ex
179
201
  end
180
- ack = true
181
- rescue Sidekiq::Shutdown
182
- # Had to force kill this job because it didn't finish
183
- # within the timeout. Don't acknowledge the work since
184
- # we didn't properly finish it.
185
- rescue Sidekiq::JobRetry::Handled => h
186
- # this is the common case: job raised error and Sidekiq::JobRetry::Handled
187
- # signals that we created a retry successfully. We can acknowlege the job.
188
- ack = true
189
- e = h.cause || h
190
- handle_exception(e, {context: "Job raised exception", job: job_hash})
191
- raise e
192
- rescue Exception => ex
193
- # Unexpected error! This is very bad and indicates an exception that got past
194
- # the retry subsystem (e.g. network partition). We won't acknowledge the job
195
- # so it can be rescued when using Sidekiq Pro.
196
- handle_exception(ex, {context: "Internal exception!", job: job_hash, jobstr: jobstr})
197
- raise ex
198
202
  ensure
199
203
  if ack
200
- # We don't want a shutdown signal to interrupt job acknowledgment.
201
- Thread.handle_interrupt(IGNORE_SHUTDOWN_INTERRUPTS) do
202
- uow.acknowledge
203
- end
204
+ uow.acknowledge
204
205
  end
205
206
  end
206
207
  end
data/lib/sidekiq/rails.rb CHANGED
@@ -20,10 +20,6 @@ module Sidekiq
20
20
  def inspect
21
21
  "#<Sidekiq::Rails::Reloader @app=#{@app.class.name}>"
22
22
  end
23
-
24
- def to_json(*)
25
- Sidekiq.dump_json(inspect)
26
- end
27
23
  end
28
24
 
29
25
  # By including the Options module, we allow AJs to directly control sidekiq features
@@ -43,17 +39,6 @@ module Sidekiq
43
39
  end
44
40
  end
45
41
 
46
- initializer "sidekiq.rails_logger" do
47
- Sidekiq.configure_server do |config|
48
- # This is the integration code necessary so that if a job uses `Rails.logger.info "Hello"`,
49
- # it will appear in the Sidekiq console with all of the job context. See #5021 and
50
- # https://github.com/rails/rails/blob/b5f2b550f69a99336482739000c58e4e04e033aa/railties/lib/rails/commands/server/server_command.rb#L82-L84
51
- unless ::Rails.logger == config.logger || ::ActiveSupport::Logger.logger_outputs_to?(::Rails.logger, $stdout)
52
- ::Rails.logger.extend(::ActiveSupport::Logger.broadcast(config.logger))
53
- end
54
- end
55
- end
56
-
57
42
  initializer "sidekiq.backtrace_cleaner" do
58
43
  Sidekiq.configure_server do |config|
59
44
  config[:backtrace_cleaner] = ->(backtrace) { ::Rails.backtrace_cleaner.clean(backtrace) }
@@ -67,6 +52,16 @@ module Sidekiq
67
52
  config.after_initialize do
68
53
  Sidekiq.configure_server do |config|
69
54
  config[:reloader] = Sidekiq::Rails::Reloader.new
55
+
56
+ # This is the integration code necessary so that if a job uses `Rails.logger.info "Hello"`,
57
+ # it will appear in the Sidekiq console with all of the job context.
58
+ unless ::Rails.logger == config.logger || ::ActiveSupport::Logger.logger_outputs_to?(::Rails.logger, $stdout)
59
+ if ::Rails::VERSION::STRING < "7.1"
60
+ ::Rails.logger.extend(::ActiveSupport::Logger.broadcast(config.logger))
61
+ else
62
+ ::Rails.logger.broadcast_to(config.logger)
63
+ end
64
+ end
70
65
  end
71
66
  end
72
67
  end
@@ -21,6 +21,22 @@ module Sidekiq
21
21
  @client.call("EVALSHA", sha, keys.size, *keys, *argv)
22
22
  end
23
23
 
24
+ # this is the set of Redis commands used by Sidekiq. Not guaranteed
25
+ # to be comprehensive, we use this as a performance enhancement to
26
+ # avoid calling method_missing on most commands
27
+ USED_COMMANDS = %w[bitfield bitfield_ro del exists expire flushdb
28
+ get hdel hget hgetall hincrby hlen hmget hset hsetnx incr incrby
29
+ lindex llen lmove lpop lpush lrange lrem mget mset ping pttl
30
+ publish rpop rpush sadd scard script set sismember smembers
31
+ srem ttl type unlink zadd zcard zincrby zrange zrem
32
+ zremrangebyrank zremrangebyscore]
33
+
34
+ USED_COMMANDS.each do |name|
35
+ define_method(name) do |*args|
36
+ @client.call(name, *args)
37
+ end
38
+ end
39
+
24
40
  private
25
41
 
26
42
  # this allows us to use methods like `conn.hmset(...)` instead of having to use
@@ -63,8 +79,7 @@ module Sidekiq
63
79
  opts = options.dup
64
80
 
65
81
  if opts[:namespace]
66
- raise ArgumentError, "Your Redis configuration uses the namespace '#{opts[:namespace]}' but this feature isn't supported by redis-client. " \
67
- "Either use the redis adapter or remove the namespace."
82
+ raise ArgumentError, "Your Redis configuration uses the namespace '#{opts[:namespace]}' but this feature is no longer supported in Sidekiq 7+. See https://github.com/sidekiq/sidekiq/blob/main/docs/7.0-Upgrade.md#redis-namespace."
68
83
  end
69
84
 
70
85
  opts.delete(:size)
@@ -14,6 +14,7 @@ module Sidekiq
14
14
  logger = symbolized_options.delete(:logger)
15
15
  logger&.info { "Sidekiq #{Sidekiq::VERSION} connecting to Redis with options #{scrub(symbolized_options)}" }
16
16
 
17
+ raise "Sidekiq 7+ does not support Redis protocol 2" if symbolized_options[:protocol] == 2
17
18
  size = symbolized_options.delete(:size) || 5
18
19
  pool_timeout = symbolized_options.delete(:pool_timeout) || 1
19
20
  pool_name = symbolized_options.delete(:pool_name)
@@ -193,7 +193,7 @@ module Sidekiq
193
193
  # should never depend on sidekiq/api.
194
194
  def cleanup
195
195
  # dont run cleanup more than once per minute
196
- return 0 unless redis { |conn| conn.set("process_cleanup", "1", nx: true, ex: 60) }
196
+ return 0 unless redis { |conn| conn.set("process_cleanup", "1", "NX", "EX", "60") }
197
197
 
198
198
  count = 0
199
199
  redis do |conn|
@@ -5,23 +5,42 @@ require "sidekiq"
5
5
 
6
6
  module Sidekiq
7
7
  class Testing
8
+ class TestModeAlreadySetError < RuntimeError; end
8
9
  class << self
9
- attr_accessor :__test_mode
10
+ attr_accessor :__global_test_mode
10
11
 
12
+ # Calling without a block sets the global test mode, affecting
13
+ # all threads. Calling with a block only affects the current Thread.
11
14
  def __set_test_mode(mode)
12
15
  if block_given?
13
- current_mode = __test_mode
16
+ # Reentrant testing modes will lead to a rat's nest of code which is
17
+ # hard to reason about. You can set the testing mode once globally and
18
+ # you can override that global setting once per-thread.
19
+ raise TestModeAlreadySetError, "Nesting test modes is not supported" if __local_test_mode
20
+
21
+ self.__local_test_mode = mode
14
22
  begin
15
- self.__test_mode = mode
16
23
  yield
17
24
  ensure
18
- self.__test_mode = current_mode
25
+ self.__local_test_mode = nil
19
26
  end
20
27
  else
21
- self.__test_mode = mode
28
+ self.__global_test_mode = mode
22
29
  end
23
30
  end
24
31
 
32
+ def __test_mode
33
+ __local_test_mode || __global_test_mode
34
+ end
35
+
36
+ def __local_test_mode
37
+ Thread.current[:__sidekiq_test_mode]
38
+ end
39
+
40
+ def __local_test_mode=(value)
41
+ Thread.current[:__sidekiq_test_mode] = value
42
+ end
43
+
25
44
  def disable!(&block)
26
45
  __set_test_mode(:disable, &block)
27
46
  end
@@ -64,7 +83,7 @@ module Sidekiq
64
83
  class EmptyQueueError < RuntimeError; end
65
84
 
66
85
  module TestingClient
67
- def raw_push(payloads)
86
+ def atomic_push(conn, payloads)
68
87
  if Sidekiq::Testing.fake?
69
88
  payloads.each do |job|
70
89
  job = Sidekiq.load_json(Sidekiq.dump_json(job))
@@ -1,6 +1,6 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Sidekiq
4
- VERSION = "7.1.2"
4
+ VERSION = "7.2.0"
5
5
  MAJOR = 7
6
6
  end
@@ -15,11 +15,11 @@ module Sidekiq
15
15
  end
16
16
 
17
17
  def halt(res)
18
- throw :halt, [res, {"content-type" => "text/plain"}, [res.to_s]]
18
+ throw :halt, [res, {Rack::CONTENT_TYPE => "text/plain"}, [res.to_s]]
19
19
  end
20
20
 
21
21
  def redirect(location)
22
- throw :halt, [302, {"location" => "#{request.base_url}#{location}"}, []]
22
+ throw :halt, [302, {Web::LOCATION => "#{request.base_url}#{location}"}, []]
23
23
  end
24
24
 
25
25
  def params
@@ -68,7 +68,7 @@ module Sidekiq
68
68
  end
69
69
 
70
70
  def json(payload)
71
- [200, {"content-type" => "application/json", "cache-control" => "private, no-store"}, [Sidekiq.dump_json(payload)]]
71
+ [200, {Rack::CONTENT_TYPE => "application/json", Rack::CACHE_CONTROL => "private, no-store"}, [Sidekiq.dump_json(payload)]]
72
72
  end
73
73
 
74
74
  def initialize(env, block)