sidekiq 6.4.2 → 6.5.0

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of sidekiq might be problematic. Click here for more details.

checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 70d899b76bdd764a2ec7e5f23b6d056494b58897daaf49db672daad3e0f60237
4
- data.tar.gz: daae0430ecaf8eb7f172c70e628581c20a17e1a7cb1a58170b9082fc3dde1ea2
3
+ metadata.gz: dedc78179d612e64435e57f0f6644c2645c06ffdc24151c9ad51df4035b51efa
4
+ data.tar.gz: 5987eba09190793cfccbff56066b54b5308a1fe74be7551cb5b9d6d03d256c3a
5
5
  SHA512:
6
- metadata.gz: 930a0feb7ff47473eb995f8ba695468498afecb311c25296c93042ecfa00cd2bf48e19efd5ba775b4895c065705a344043e0c0e5d7b4115c2dceb1ec7cf341f9
7
- data.tar.gz: 14ab645609e7c9f3fa954432097cd46d6e9e74e5bddc9ed1541697d13f3399add407ffb5af23c89c82651793c97e2c54b8fd4f845ca20287ce6bf033fd9caed2
6
+ metadata.gz: 9a26b5eb6e2248870f07ede054cac8330b52ad548d27b40d3950b6d985ef5ef8359e4dfc062757dc4b479036f0b2716bacfc821a20bd63413ea7c3f772bc6ead
7
+ data.tar.gz: 46a35f7f27693364b1b6077e602d3f69e476508654157f48fcd457a70bbcdb563f8515812ef37abae207604596e1d42f78132ac54471109f913e87c11a707cdb
data/Changes.md CHANGED
@@ -2,6 +2,26 @@
2
2
 
3
3
  [Sidekiq Changes](https://github.com/mperham/sidekiq/blob/main/Changes.md) | [Sidekiq Pro Changes](https://github.com/mperham/sidekiq/blob/main/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/mperham/sidekiq/blob/main/Ent-Changes.md)
4
4
 
5
+ 6.5.0
6
+ ---------
7
+
8
+ - Substantial refactoring of Sidekiq server internals, part of a larger effort
9
+ to reduce Sidekiq's internal usage of global methods and data, see [docs/component.md](docs/component.md),
10
+ [docs/global_to_local.md](docs/global_to_local.md) and [docs/middleware.md](docs/middleware.md).
11
+ - **Add beta support for the `redis-client` gem**. This will become the default Redis driver in Sidekiq 7.0. [#5298]
12
+ Read more: https://github.com/mperham/sidekiq/wiki/Using-redis-client
13
+ - **Add beta support for DB transaction-aware client** [#5291]
14
+ Add this line to your initializer and any jobs created during a transaction
15
+ will only be pushed to Redis **after the transaction commits**. You will need to add the
16
+ `after_commit_everywhere` gem to your Gemfile.
17
+ ```ruby
18
+ Sidekiq.transactional_push!
19
+ ```
20
+ This feature does not have a lot of production usage yet; please try it out and let us
21
+ know if you have any issues. It will be fully supported in Sidekiq 7.0 or removed if it
22
+ proves problematic.
23
+ - Fix regression with middleware arguments [#5312]
24
+
5
25
  6.4.2
6
26
  ---------
7
27
 
data/bin/sidekiqload CHANGED
@@ -11,6 +11,10 @@ Bundler.require(:default, :load_test)
11
11
  require_relative "../lib/sidekiq/cli"
12
12
  require_relative "../lib/sidekiq/launcher"
13
13
 
14
+ if ENV["SIDEKIQ_REDIS_CLIENT"]
15
+ Sidekiq::RedisConnection.adapter = :redis_client
16
+ end
17
+
14
18
  Sidekiq.configure_server do |config|
15
19
  config.options[:concurrency] = 10
16
20
  config.redis = {db: 13, port: 6380}
@@ -118,15 +122,23 @@ Monitoring = Thread.new do
118
122
  end
119
123
  end
120
124
 
125
+ def with_latency(latency, &block)
126
+ Sidekiq.logger.error "Simulating #{latency}ms of latency between Sidekiq and redis"
127
+ if latency > 0
128
+ Toxiproxy[:redis].downstream(:latency, latency: latency).apply(&block)
129
+ else
130
+ yield
131
+ end
132
+ end
133
+
121
134
  begin
122
135
  # RubyProf::exclude_threads = [ Monitoring ]
123
136
  # RubyProf.start
124
137
  events = Sidekiq.options[:lifecycle_events][:startup]
125
138
  events.each(&:call)
126
139
  events.clear
127
-
128
- Sidekiq.logger.error "Simulating 1ms of latency between Sidekiq and redis"
129
- Toxiproxy[:redis].downstream(:latency, latency: 1).apply do
140
+
141
+ with_latency(Integer(ENV.fetch("LATENCY", "1"))) do
130
142
  launcher = Sidekiq::Launcher.new(Sidekiq.options)
131
143
  launcher.run
132
144
 
data/lib/sidekiq/api.rb CHANGED
@@ -191,7 +191,7 @@ module Sidekiq
191
191
  stat_hash[dates[idx]] = value ? value.to_i : 0
192
192
  end
193
193
  end
194
- rescue Redis::CommandError
194
+ rescue RedisConnection.adapter::CommandError
195
195
  # mget will trigger a CROSSSLOT error when run against a Cluster
196
196
  # TODO Someone want to add Cluster support?
197
197
  end
@@ -294,6 +294,10 @@ module Sidekiq
294
294
  end
295
295
  end
296
296
  alias_method :💣, :clear
297
+
298
+ def as_json(options = nil) # :nodoc:
299
+ {name: name} # 5336
300
+ end
297
301
  end
298
302
 
299
303
  ##
@@ -442,7 +446,8 @@ module Sidekiq
442
446
  rescue => ex
443
447
  # #1761 in dev mode, it's possible to have jobs enqueued which haven't been loaded into
444
448
  # memory yet so the YAML can't be loaded.
445
- Sidekiq.logger.warn "Unable to load YAML: #{ex.message}" unless Sidekiq.options[:environment] == "development"
449
+ # TODO is this still necessary? Zeitwerk reloader should handle?
450
+ Sidekiq.logger.warn "Unable to load YAML: #{ex.message}" unless Sidekiq.config[:environment] == "development"
446
451
  default
447
452
  end
448
453
 
@@ -470,7 +475,7 @@ module Sidekiq
470
475
 
471
476
  def initialize(parent, score, item)
472
477
  super(item)
473
- @score = score
478
+ @score = Float(score)
474
479
  @parent = parent
475
480
  end
476
481
 
@@ -587,6 +592,10 @@ module Sidekiq
587
592
  end
588
593
  end
589
594
  alias_method :💣, :clear
595
+
596
+ def as_json(options = nil) # :nodoc:
597
+ {name: name} # 5336
598
+ end
590
599
  end
591
600
 
592
601
  class JobSet < SortedSet
@@ -606,7 +615,7 @@ module Sidekiq
606
615
  range_start = page * page_size + offset_size
607
616
  range_end = range_start + page_size - 1
608
617
  elements = Sidekiq.redis { |conn|
609
- conn.zrange name, range_start, range_end, with_scores: true
618
+ conn.zrange name, range_start, range_end, withscores: true
610
619
  }
611
620
  break if elements.empty?
612
621
  page -= 1
@@ -629,7 +638,7 @@ module Sidekiq
629
638
  end
630
639
 
631
640
  elements = Sidekiq.redis { |conn|
632
- conn.zrangebyscore(name, begin_score, end_score, with_scores: true)
641
+ conn.zrangebyscore(name, begin_score, end_score, withscores: true)
633
642
  }
634
643
 
635
644
  elements.each_with_object([]) do |element, result|
@@ -758,11 +767,11 @@ module Sidekiq
758
767
  end
759
768
 
760
769
  def self.max_jobs
761
- Sidekiq.options[:dead_max_jobs]
770
+ Sidekiq[:dead_max_jobs]
762
771
  end
763
772
 
764
773
  def self.timeout
765
- Sidekiq.options[:dead_timeout_in_seconds]
774
+ Sidekiq[:dead_timeout_in_seconds]
766
775
  end
767
776
  end
768
777
 
data/lib/sidekiq/cli.rb CHANGED
@@ -9,18 +9,23 @@ require "erb"
9
9
  require "fileutils"
10
10
 
11
11
  require "sidekiq"
12
+ require "sidekiq/component"
12
13
  require "sidekiq/launcher"
13
- require "sidekiq/util"
14
14
 
15
15
  module Sidekiq
16
16
  class CLI
17
- include Util
17
+ include Sidekiq::Component
18
18
  include Singleton unless $TESTING
19
19
 
20
20
  attr_accessor :launcher
21
21
  attr_accessor :environment
22
+ attr_accessor :config
22
23
 
23
24
  def parse(args = ARGV.dup)
25
+ @config = Sidekiq
26
+ @config[:error_handlers].clear
27
+ @config[:error_handlers] << @config.method(:default_error_handler)
28
+
24
29
  setup_options(args)
25
30
  initialize_logger
26
31
  validate!
@@ -36,7 +41,7 @@ module Sidekiq
36
41
  def run(boot_app: true)
37
42
  boot_application if boot_app
38
43
 
39
- if environment == "development" && $stdout.tty? && Sidekiq.log_formatter.is_a?(Sidekiq::Logger::Formatters::Pretty)
44
+ if environment == "development" && $stdout.tty? && @config.log_formatter.is_a?(Sidekiq::Logger::Formatters::Pretty)
40
45
  print_banner
41
46
  end
42
47
  logger.info "Booted Rails #{::Rails.version} application in #{environment} environment" if rails_app?
@@ -67,7 +72,7 @@ module Sidekiq
67
72
 
68
73
  # touch the connection pool so it is created before we
69
74
  # fire startup and start multithreading.
70
- info = Sidekiq.redis_info
75
+ info = @config.redis_info
71
76
  ver = info["redis_version"]
72
77
  raise "You are connecting to Redis v#{ver}, Sidekiq requires Redis v4.0.0 or greater" if ver < "4"
73
78
 
@@ -85,22 +90,22 @@ module Sidekiq
85
90
 
86
91
  # Since the user can pass us a connection pool explicitly in the initializer, we
87
92
  # need to verify the size is large enough or else Sidekiq's performance is dramatically slowed.
88
- cursize = Sidekiq.redis_pool.size
89
- needed = Sidekiq.options[:concurrency] + 2
93
+ cursize = @config.redis_pool.size
94
+ needed = @config[:concurrency] + 2
90
95
  raise "Your pool of #{cursize} Redis connections is too small, please increase the size to at least #{needed}" if cursize < needed
91
96
 
92
97
  # cache process identity
93
- Sidekiq.options[:identity] = identity
98
+ @config[:identity] = identity
94
99
 
95
100
  # Touch middleware so it isn't lazy loaded by multiple threads, #3043
96
- Sidekiq.server_middleware
101
+ @config.server_middleware
97
102
 
98
103
  # Before this point, the process is initializing with just the main thread.
99
104
  # Starting here the process will now have multiple threads running.
100
105
  fire_event(:startup, reverse: false, reraise: true)
101
106
 
102
- logger.debug { "Client Middleware: #{Sidekiq.client_middleware.map(&:klass).join(", ")}" }
103
- logger.debug { "Server Middleware: #{Sidekiq.server_middleware.map(&:klass).join(", ")}" }
107
+ logger.debug { "Client Middleware: #{@config.client_middleware.map(&:klass).join(", ")}" }
108
+ logger.debug { "Server Middleware: #{@config.server_middleware.map(&:klass).join(", ")}" }
104
109
 
105
110
  launch(self_read)
106
111
  end
@@ -110,7 +115,7 @@ module Sidekiq
110
115
  logger.info "Starting processing, hit Ctrl-C to stop"
111
116
  end
112
117
 
113
- @launcher = Sidekiq::Launcher.new(options)
118
+ @launcher = Sidekiq::Launcher.new(@config)
114
119
 
115
120
  begin
116
121
  launcher.run
@@ -173,25 +178,25 @@ module Sidekiq
173
178
  # Heroku sends TERM and then waits 30 seconds for process to exit.
174
179
  "TERM" => ->(cli) { raise Interrupt },
175
180
  "TSTP" => ->(cli) {
176
- Sidekiq.logger.info "Received TSTP, no longer accepting new work"
181
+ cli.logger.info "Received TSTP, no longer accepting new work"
177
182
  cli.launcher.quiet
178
183
  },
179
184
  "TTIN" => ->(cli) {
180
185
  Thread.list.each do |thread|
181
- Sidekiq.logger.warn "Thread TID-#{(thread.object_id ^ ::Process.pid).to_s(36)} #{thread.name}"
186
+ cli.logger.warn "Thread TID-#{(thread.object_id ^ ::Process.pid).to_s(36)} #{thread.name}"
182
187
  if thread.backtrace
183
- Sidekiq.logger.warn thread.backtrace.join("\n")
188
+ cli.logger.warn thread.backtrace.join("\n")
184
189
  else
185
- Sidekiq.logger.warn "<no backtrace available>"
190
+ cli.logger.warn "<no backtrace available>"
186
191
  end
187
192
  end
188
193
  }
189
194
  }
190
- UNHANDLED_SIGNAL_HANDLER = ->(cli) { Sidekiq.logger.info "No signal handler registered, ignoring" }
195
+ UNHANDLED_SIGNAL_HANDLER = ->(cli) { cli.logger.info "No signal handler registered, ignoring" }
191
196
  SIGNAL_HANDLERS.default = UNHANDLED_SIGNAL_HANDLER
192
197
 
193
198
  def handle_signal(sig)
194
- Sidekiq.logger.debug "Got #{sig} signal"
199
+ logger.debug "Got #{sig} signal"
195
200
  SIGNAL_HANDLERS[sig].call(self)
196
201
  end
197
202
 
@@ -237,7 +242,7 @@ module Sidekiq
237
242
  config_dir = if File.directory?(opts[:require].to_s)
238
243
  File.join(opts[:require], "config")
239
244
  else
240
- File.join(options[:require], "config")
245
+ File.join(@config[:require], "config")
241
246
  end
242
247
 
243
248
  %w[sidekiq.yml sidekiq.yml.erb].each do |config_file|
@@ -254,27 +259,23 @@ module Sidekiq
254
259
  opts[:concurrency] = Integer(ENV["RAILS_MAX_THREADS"]) if opts[:concurrency].nil? && ENV["RAILS_MAX_THREADS"]
255
260
 
256
261
  # merge with defaults
257
- options.merge!(opts)
258
- end
259
-
260
- def options
261
- Sidekiq.options
262
+ @config.merge!(opts)
262
263
  end
263
264
 
264
265
  def boot_application
265
266
  ENV["RACK_ENV"] = ENV["RAILS_ENV"] = environment
266
267
 
267
- if File.directory?(options[:require])
268
+ if File.directory?(@config[:require])
268
269
  require "rails"
269
270
  if ::Rails::VERSION::MAJOR < 5
270
271
  raise "Sidekiq no longer supports this version of Rails"
271
272
  else
272
273
  require "sidekiq/rails"
273
- require File.expand_path("#{options[:require]}/config/environment.rb")
274
+ require File.expand_path("#{@config[:require]}/config/environment.rb")
274
275
  end
275
- options[:tag] ||= default_tag
276
+ @config[:tag] ||= default_tag
276
277
  else
277
- require options[:require]
278
+ require @config[:require]
278
279
  end
279
280
  end
280
281
 
@@ -291,8 +292,8 @@ module Sidekiq
291
292
  end
292
293
 
293
294
  def validate!
294
- if !File.exist?(options[:require]) ||
295
- (File.directory?(options[:require]) && !File.exist?("#{options[:require]}/config/application.rb"))
295
+ if !File.exist?(@config[:require]) ||
296
+ (File.directory?(@config[:require]) && !File.exist?("#{@config[:require]}/config/application.rb"))
296
297
  logger.info "=================================================================="
297
298
  logger.info " Please point Sidekiq to a Rails application or a Ruby file "
298
299
  logger.info " to load your job classes with -r [DIR|FILE]."
@@ -302,7 +303,7 @@ module Sidekiq
302
303
  end
303
304
 
304
305
  [:concurrency, :timeout].each do |opt|
305
- raise ArgumentError, "#{opt}: #{options[opt]} is not a valid value" if options.key?(opt) && options[opt].to_i <= 0
306
+ raise ArgumentError, "#{opt}: #{@config[opt]} is not a valid value" if @config[opt].to_i <= 0
306
307
  end
307
308
  end
308
309
 
@@ -376,7 +377,7 @@ module Sidekiq
376
377
  end
377
378
 
378
379
  def initialize_logger
379
- Sidekiq.logger.level = ::Logger::DEBUG if options[:verbose]
380
+ @config.logger.level = ::Logger::DEBUG if @config[:verbose]
380
381
  end
381
382
 
382
383
  def parse_config(path)
@@ -71,7 +71,7 @@ module Sidekiq
71
71
  #
72
72
  def push(item)
73
73
  normed = normalize_item(item)
74
- payload = middleware.invoke(normed["class"], normed, normed["queue"], @redis_pool) do
74
+ payload = middleware.invoke(item["class"], normed, normed["queue"], @redis_pool) do
75
75
  normed
76
76
  end
77
77
  if payload
@@ -110,7 +110,7 @@ module Sidekiq
110
110
  payloads = args.map.with_index { |job_args, index|
111
111
  copy = normed.merge("args" => job_args, "jid" => SecureRandom.hex(12))
112
112
  copy["at"] = (at.is_a?(Array) ? at[index] : at) if at
113
- result = middleware.invoke(copy["class"], copy, copy["queue"], @redis_pool) do
113
+ result = middleware.invoke(items["class"], copy, copy["queue"], @redis_pool) do
114
114
  verify_json(copy)
115
115
  copy
116
116
  end
@@ -201,7 +201,7 @@ module Sidekiq
201
201
  conn.pipelined do |pipeline|
202
202
  atomic_push(pipeline, payloads)
203
203
  end
204
- rescue Redis::BaseError => ex
204
+ rescue RedisConnection.adapter::BaseError => ex
205
205
  # 2550 Failover can cause the server to become a replica, need
206
206
  # to disconnect and reopen the socket to get back to the primary.
207
207
  # 4495 Use the same logic if we have a "Not enough replicas" error from the primary
@@ -220,7 +220,7 @@ module Sidekiq
220
220
 
221
221
  def atomic_push(conn, payloads)
222
222
  if payloads.first.key?("at")
223
- conn.zadd("schedule", payloads.map { |hash|
223
+ conn.zadd("schedule", *payloads.map { |hash|
224
224
  at = hash.delete("at").to_s
225
225
  [at, Sidekiq.dump_json(hash)]
226
226
  })
@@ -0,0 +1,64 @@
1
+ module Sidekiq
2
+ ##
3
+ # Sidekiq::Component assumes a config instance is available at @config
4
+ module Component
5
+ attr_reader :config
6
+
7
+ def watchdog(last_words)
8
+ yield
9
+ rescue Exception => ex
10
+ handle_exception(ex, {context: last_words})
11
+ raise ex
12
+ end
13
+
14
+ def safe_thread(name, &block)
15
+ Thread.new do
16
+ Thread.current.name = name
17
+ watchdog(name, &block)
18
+ end
19
+ end
20
+
21
+ def logger
22
+ config.logger
23
+ end
24
+
25
+ def redis(&block)
26
+ config.redis(&block)
27
+ end
28
+
29
+ def tid
30
+ Thread.current["sidekiq_tid"] ||= (Thread.current.object_id ^ ::Process.pid).to_s(36)
31
+ end
32
+
33
+ def hostname
34
+ ENV["DYNO"] || Socket.gethostname
35
+ end
36
+
37
+ def process_nonce
38
+ @@process_nonce ||= SecureRandom.hex(6)
39
+ end
40
+
41
+ def identity
42
+ @@identity ||= "#{hostname}:#{::Process.pid}:#{process_nonce}"
43
+ end
44
+
45
+ def handle_exception(ex, ctx = {})
46
+ config.handle_exception(ex, ctx)
47
+ end
48
+
49
+ def fire_event(event, options = {})
50
+ reverse = options[:reverse]
51
+ reraise = options[:reraise]
52
+
53
+ arr = config[:lifecycle_events][event]
54
+ arr.reverse! if reverse
55
+ arr.each do |block|
56
+ block.call
57
+ rescue => ex
58
+ handle_exception(ex, {context: "Exception during Sidekiq lifecycle event.", event: event})
59
+ raise ex if reraise
60
+ end
61
+ arr.clear # once we've fired an event, we never fire it again
62
+ end
63
+ end
64
+ end
data/lib/sidekiq/fetch.rb CHANGED
@@ -1,14 +1,16 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  require "sidekiq"
4
+ require "sidekiq/component"
4
5
 
5
6
  module Sidekiq
6
7
  class BasicFetch
8
+ include Sidekiq::Component
7
9
  # We want the fetch operation to timeout every few seconds so the thread
8
10
  # can check if the process is shutting down.
9
11
  TIMEOUT = 2
10
12
 
11
- UnitOfWork = Struct.new(:queue, :job) {
13
+ UnitOfWork = Struct.new(:queue, :job, :config) {
12
14
  def acknowledge
13
15
  # nothing to do
14
16
  end
@@ -18,17 +20,17 @@ module Sidekiq
18
20
  end
19
21
 
20
22
  def requeue
21
- Sidekiq.redis do |conn|
23
+ config.redis do |conn|
22
24
  conn.rpush(queue, job)
23
25
  end
24
26
  end
25
27
  }
26
28
 
27
- def initialize(options)
28
- raise ArgumentError, "missing queue list" unless options[:queues]
29
- @options = options
30
- @strictly_ordered_queues = !!@options[:strict]
31
- @queues = @options[:queues].map { |q| "queue:#{q}" }
29
+ def initialize(config)
30
+ raise ArgumentError, "missing queue list" unless config[:queues]
31
+ @config = config
32
+ @strictly_ordered_queues = !!@config[:strict]
33
+ @queues = @config[:queues].map { |q| "queue:#{q}" }
32
34
  if @strictly_ordered_queues
33
35
  @queues.uniq!
34
36
  @queues << TIMEOUT
@@ -44,30 +46,30 @@ module Sidekiq
44
46
  return nil
45
47
  end
46
48
 
47
- work = Sidekiq.redis { |conn| conn.brpop(*qs) }
48
- UnitOfWork.new(*work) if work
49
+ queue, job = redis { |conn| conn.brpop(*qs) }
50
+ UnitOfWork.new(queue, job, config) if queue
49
51
  end
50
52
 
51
53
  def bulk_requeue(inprogress, options)
52
54
  return if inprogress.empty?
53
55
 
54
- Sidekiq.logger.debug { "Re-queueing terminated jobs" }
56
+ logger.debug { "Re-queueing terminated jobs" }
55
57
  jobs_to_requeue = {}
56
58
  inprogress.each do |unit_of_work|
57
59
  jobs_to_requeue[unit_of_work.queue] ||= []
58
60
  jobs_to_requeue[unit_of_work.queue] << unit_of_work.job
59
61
  end
60
62
 
61
- Sidekiq.redis do |conn|
63
+ redis do |conn|
62
64
  conn.pipelined do |pipeline|
63
65
  jobs_to_requeue.each do |queue, jobs|
64
66
  pipeline.rpush(queue, jobs)
65
67
  end
66
68
  end
67
69
  end
68
- Sidekiq.logger.info("Pushed #{inprogress.size} jobs back to Redis")
70
+ logger.info("Pushed #{inprogress.size} jobs back to Redis")
69
71
  rescue => ex
70
- Sidekiq.logger.warn("Failed to requeue #{inprogress.size} jobs: #{ex.message}")
72
+ logger.warn("Failed to requeue #{inprogress.size} jobs: #{ex.message}")
71
73
  end
72
74
 
73
75
  # Creating the Redis#brpop command takes into account any
@@ -66,12 +66,13 @@ module Sidekiq
66
66
 
67
67
  class Skip < Handled; end
68
68
 
69
- include Sidekiq::Util
69
+ include Sidekiq::Component
70
70
 
71
71
  DEFAULT_MAX_RETRY_ATTEMPTS = 25
72
72
 
73
- def initialize(options = {})
74
- @max_retries = Sidekiq.options.merge(options).fetch(:max_retries, DEFAULT_MAX_RETRY_ATTEMPTS)
73
+ def initialize(options)
74
+ @config = options
75
+ @max_retries = @config[:max_retries] || DEFAULT_MAX_RETRY_ATTEMPTS
75
76
  end
76
77
 
77
78
  # The global retry handler requires only the barest of data.
@@ -4,7 +4,8 @@ require "time"
4
4
  module Sidekiq
5
5
  module JobUtil
6
6
  # These functions encapsulate various job utilities.
7
- # They must be simple and free from side effects.
7
+
8
+ TRANSIENT_ATTRIBUTES = %w[]
8
9
 
9
10
  def validate(item)
10
11
  raise(ArgumentError, "Job must be a Hash with 'class' and 'args' keys: `#{item}`") unless item.is_a?(Hash) && item.key?("class") && item.key?("args")
@@ -16,13 +17,13 @@ module Sidekiq
16
17
 
17
18
  def verify_json(item)
18
19
  job_class = item["wrapped"] || item["class"]
19
- if Sidekiq.options[:on_complex_arguments] == :raise
20
+ if Sidekiq[:on_complex_arguments] == :raise
20
21
  msg = <<~EOM
21
22
  Job arguments to #{job_class} must be native JSON types, see https://github.com/mperham/sidekiq/wiki/Best-Practices.
22
23
  To disable this error, remove `Sidekiq.strict_args!` from your initializer.
23
24
  EOM
24
25
  raise(ArgumentError, msg) unless json_safe?(item)
25
- elsif Sidekiq.options[:on_complex_arguments] == :warn
26
+ elsif Sidekiq[:on_complex_arguments] == :warn
26
27
  Sidekiq.logger.warn <<~EOM unless json_safe?(item)
27
28
  Job arguments to #{job_class} do not serialize to JSON safely. This will raise an error in
28
29
  Sidekiq 7.0. See https://github.com/mperham/sidekiq/wiki/Best-Practices or raise an error today
@@ -42,6 +43,9 @@ module Sidekiq
42
43
 
43
44
  raise(ArgumentError, "Job must include a valid queue name") if item["queue"].nil? || item["queue"] == ""
44
45
 
46
+ # remove job attributes which aren't necessary to persist into Redis
47
+ TRANSIENT_ATTRIBUTES.each { |key| item.delete(key) }
48
+
45
49
  item["jid"] ||= SecureRandom.hex(12)
46
50
  item["class"] = item["class"].to_s
47
51
  item["queue"] = item["queue"].to_s