sidekiq-requeue_missing_class 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +7 -0
- data/LICENSE.txt +21 -0
- data/README.md +67 -0
- data/lib/sidekiq/requeue_missing_class/configuration.rb +30 -0
- data/lib/sidekiq/requeue_missing_class/middleware.rb +79 -0
- data/lib/sidekiq/requeue_missing_class/version.rb +7 -0
- data/lib/sidekiq/requeue_missing_class.rb +31 -0
- metadata +109 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: 5efea06725faf4b01f45b52a57344a9fd60c4d34f3145cc514c57f32e261afee
|
|
4
|
+
data.tar.gz: f89b8d91859416c7bf4253dd9d11e42e79eca3b50dd1ea8efbeabd345eb2f8f0
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: a048a9719bd5e5109fc0f74c3cd9d7b6147b7d3e1d8a59962f30c99639353f9cf442d261fd69e445ed71b8f46249473a02e77d2494b40b449775cc7a2d67b932
|
|
7
|
+
data.tar.gz: fac7f03b37efe1764fa911a7dacb6d0e142ff0dd951c0b92b9a3e11ecc9f5eefee17e3746d9bc27a2063c6790727fc5818468b91913a574ceb2973f5cc879f78
|
data/CHANGELOG.md
ADDED
data/LICENSE.txt
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
The MIT License (MIT)
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Dmitry Tsepelev
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in
|
|
13
|
+
all copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
|
21
|
+
THE SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
# sidekiq-requeue_missing_class  
|
|
2
|
+
|
|
3
|
+
`sidekiq-requeue_missing_class` helps you survive rolling deploys that introduce a **new** Sidekiq/ActiveJob class: instead of erroring, jobs whose class is not loadable yet are pushed back with a short delay until an updated worker picks them up.
|
|
4
|
+
|
|
5
|
+
## Professional Support
|
|
6
|
+
|
|
7
|
+
Need help with your Sidekiq setup, background jobs architecture, or Rails performance?
|
|
8
|
+
I'm available for consulting — [get in touch](https://dmitrytsepelev.dev/consulting).
|
|
9
|
+
|
|
10
|
+
## The problem
|
|
11
|
+
|
|
12
|
+
You add a new job and deploy. Mid-rollout, a freshly started pod (or your scheduler) enqueues `SendShinyNewThingJob` — but the **old** workers are still running and don't have that class yet. An old worker pops the job and blows up:
|
|
13
|
+
|
|
14
|
+
```
|
|
15
|
+
ActiveJob::UnknownJobClassError (Not::Loaded::Yet)
|
|
16
|
+
# or, for a native worker:
|
|
17
|
+
NameError: uninitialized constant SendShinyNewThingJob
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
"Just deploy code before you enqueue it" is the textbook answer, and you should do that where you can. But with rolling deploys, autoscaling and schedulers, the race window is real and annoying to close by discipline alone.
|
|
21
|
+
|
|
22
|
+
## How it works
|
|
23
|
+
|
|
24
|
+
The gem adds a tiny Sidekiq **server middleware** that, before running a job, checks whether the class meant to run it is actually loadable in *this* process. If it isn't, the job is pushed back with a short delay (instead of erroring), so an updated worker picks it up once the rollout finishes. After a configurable number of attempts it gives up and lets the job run — so a genuinely-missing class still surfaces the usual error instead of looping forever.
|
|
25
|
+
|
|
26
|
+
Works for both **ActiveJob-over-Sidekiq** and **native Sidekiq** jobs:
|
|
27
|
+
|
|
28
|
+
- ActiveJob jobs are wrapped, so the middleware looks at the wrapped `job_class`.
|
|
29
|
+
- Native Sidekiq jobs are checked by their worker class directly.
|
|
30
|
+
- "Loadable" means `Object.const_get(name)` succeeds (same autoloading path Sidekiq/Zeitwerk take), so classes that just need autoloading are *not* treated as missing.
|
|
31
|
+
|
|
32
|
+
## Getting started
|
|
33
|
+
|
|
34
|
+
Add the gem to your Gemfile:
|
|
35
|
+
|
|
36
|
+
```ruby
|
|
37
|
+
gem "sidekiq-requeue_missing_class"
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
and require it:
|
|
41
|
+
|
|
42
|
+
```ruby
|
|
43
|
+
require "sidekiq/requeue_missing_class"
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
That's it — the middleware registers itself on the server side. The check is a single constant lookup for jobs whose class is present, so the overhead for the happy path is negligible.
|
|
47
|
+
|
|
48
|
+
## Configuration
|
|
49
|
+
|
|
50
|
+
```ruby
|
|
51
|
+
Sidekiq::RequeueMissingClass.configure do |c|
|
|
52
|
+
c.delay = 30 # seconds before a re-queued job runs again (default: 30)
|
|
53
|
+
c.max_requeues = 10 # attempts before giving up (default: 10)
|
|
54
|
+
c.logger = Sidekiq.logger # or nil to stay quiet
|
|
55
|
+
end
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
Pick `delay × max_requeues` to comfortably outlast your slowest rollout.
|
|
59
|
+
|
|
60
|
+
## When you don't need it
|
|
61
|
+
|
|
62
|
+
- If your deploy strategy already guarantees code-before-enqueue ordering, you may not need this.
|
|
63
|
+
- It only helps with the **missing-class** race, not with incompatible argument changes — version your job arguments separately.
|
|
64
|
+
|
|
65
|
+
## License
|
|
66
|
+
|
|
67
|
+
The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
|
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Sidekiq
|
|
4
|
+
module RequeueMissingClass
|
|
5
|
+
# Tunables for the middleware. Configure via:
|
|
6
|
+
#
|
|
7
|
+
# Sidekiq::RequeueMissingClass.configure do |c|
|
|
8
|
+
# c.delay = 15
|
|
9
|
+
# c.max_requeues = 20
|
|
10
|
+
# end
|
|
11
|
+
class Configuration
|
|
12
|
+
# Seconds to wait before a re-queued job becomes available again.
|
|
13
|
+
attr_accessor :delay
|
|
14
|
+
|
|
15
|
+
# How many times a single job may be re-queued before we give up and let
|
|
16
|
+
# it run (so the usual "unknown class" error surfaces instead of looping
|
|
17
|
+
# forever when the class is genuinely gone).
|
|
18
|
+
attr_accessor :max_requeues
|
|
19
|
+
|
|
20
|
+
# Anything responding to #info, or nil to stay quiet.
|
|
21
|
+
attr_accessor :logger
|
|
22
|
+
|
|
23
|
+
def initialize
|
|
24
|
+
@delay = 30
|
|
25
|
+
@max_requeues = 10
|
|
26
|
+
@logger = Sidekiq.logger
|
|
27
|
+
end
|
|
28
|
+
end
|
|
29
|
+
end
|
|
30
|
+
end
|
|
@@ -0,0 +1,79 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Sidekiq
|
|
4
|
+
module RequeueMissingClass
|
|
5
|
+
# Server middleware that re-queues a job when the class meant to run it is
|
|
6
|
+
# not defined in the current worker process.
|
|
7
|
+
#
|
|
8
|
+
# During a rolling deploy that introduces a new job class, a freshly started
|
|
9
|
+
# pod (or a scheduler) can enqueue a job for a class that the still-running
|
|
10
|
+
# OLD workers do not have yet. An old worker that pops it would otherwise
|
|
11
|
+
# blow up (ActiveJob::UnknownJobClassError / NameError). Instead we push the
|
|
12
|
+
# job back with a short delay so an updated worker — which has the class —
|
|
13
|
+
# picks it up once the rollout finishes.
|
|
14
|
+
class Middleware
|
|
15
|
+
# ActiveJob wraps the real job; the wrapper class is always present, so we
|
|
16
|
+
# have to look one level deeper at the wrapped job_class.
|
|
17
|
+
ACTIVE_JOB_WRAPPERS = [
|
|
18
|
+
"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper",
|
|
19
|
+
"Sidekiq::ActiveJob::Wrapper"
|
|
20
|
+
].freeze
|
|
21
|
+
|
|
22
|
+
COUNTER_KEY = "requeue_missing_class_count"
|
|
23
|
+
|
|
24
|
+
def call(_worker, job, queue)
|
|
25
|
+
target = target_class_name(job)
|
|
26
|
+
return yield if target.nil? || loadable?(target)
|
|
27
|
+
|
|
28
|
+
attempts = job.fetch(COUNTER_KEY, 0)
|
|
29
|
+
return yield if attempts >= config.max_requeues
|
|
30
|
+
|
|
31
|
+
requeue(job, queue, target, attempts)
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
private
|
|
35
|
+
|
|
36
|
+
def requeue(job, queue, target, attempts)
|
|
37
|
+
Sidekiq::Client.push(
|
|
38
|
+
job.merge(
|
|
39
|
+
"queue" => queue,
|
|
40
|
+
"at" => (Time.now + config.delay).to_f,
|
|
41
|
+
COUNTER_KEY => attempts + 1
|
|
42
|
+
)
|
|
43
|
+
)
|
|
44
|
+
|
|
45
|
+
log_requeue(target, attempts)
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
def log_requeue(target, attempts)
|
|
49
|
+
config.logger&.info(
|
|
50
|
+
"[RequeueMissingClass] #{target} not loadable in this process; " \
|
|
51
|
+
"re-queued with #{config.delay.to_i}s delay (#{attempts + 1}/#{config.max_requeues})"
|
|
52
|
+
)
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
# The class that will actually run this job: for ActiveJob it's the wrapped
|
|
56
|
+
# job_class, for a native Sidekiq job it's the worker class itself.
|
|
57
|
+
def target_class_name(job)
|
|
58
|
+
if ACTIVE_JOB_WRAPPERS.include?(job["class"])
|
|
59
|
+
job.dig("args", 0, "job_class")
|
|
60
|
+
else
|
|
61
|
+
job["class"]
|
|
62
|
+
end
|
|
63
|
+
end
|
|
64
|
+
|
|
65
|
+
# Mirrors how Sidekiq/ActiveJob resolve the constant (autoloading through
|
|
66
|
+
# Zeitwerk): a missing class raises NameError.
|
|
67
|
+
def loadable?(class_name)
|
|
68
|
+
Object.const_get(class_name)
|
|
69
|
+
true
|
|
70
|
+
rescue NameError
|
|
71
|
+
false
|
|
72
|
+
end
|
|
73
|
+
|
|
74
|
+
def config
|
|
75
|
+
RequeueMissingClass.configuration
|
|
76
|
+
end
|
|
77
|
+
end
|
|
78
|
+
end
|
|
79
|
+
end
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "sidekiq"
|
|
4
|
+
|
|
5
|
+
require_relative "requeue_missing_class/version"
|
|
6
|
+
require_relative "requeue_missing_class/configuration"
|
|
7
|
+
require_relative "requeue_missing_class/middleware"
|
|
8
|
+
|
|
9
|
+
module Sidekiq
|
|
10
|
+
# Entry point for the gem: holds the singleton configuration and registers the
|
|
11
|
+
# server middleware. See RequeueMissingClass::Middleware for the actual logic.
|
|
12
|
+
module RequeueMissingClass
|
|
13
|
+
class << self
|
|
14
|
+
def configuration
|
|
15
|
+
@configuration ||= Configuration.new
|
|
16
|
+
end
|
|
17
|
+
|
|
18
|
+
def configure
|
|
19
|
+
yield(configuration)
|
|
20
|
+
end
|
|
21
|
+
end
|
|
22
|
+
end
|
|
23
|
+
end
|
|
24
|
+
|
|
25
|
+
# Register automatically on the server side. The middleware is cheap (a constant
|
|
26
|
+
# lookup) for jobs whose class is present, so it's safe to keep first in the chain.
|
|
27
|
+
Sidekiq.configure_server do |sidekiq_config|
|
|
28
|
+
sidekiq_config.server_middleware do |chain|
|
|
29
|
+
chain.add Sidekiq::RequeueMissingClass::Middleware
|
|
30
|
+
end
|
|
31
|
+
end
|
metadata
ADDED
|
@@ -0,0 +1,109 @@
|
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
|
2
|
+
name: sidekiq-requeue_missing_class
|
|
3
|
+
version: !ruby/object:Gem::Version
|
|
4
|
+
version: 0.1.0
|
|
5
|
+
platform: ruby
|
|
6
|
+
authors:
|
|
7
|
+
- Dmitry Tsepelev
|
|
8
|
+
bindir: bin
|
|
9
|
+
cert_chain: []
|
|
10
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
|
11
|
+
dependencies:
|
|
12
|
+
- !ruby/object:Gem::Dependency
|
|
13
|
+
name: sidekiq
|
|
14
|
+
requirement: !ruby/object:Gem::Requirement
|
|
15
|
+
requirements:
|
|
16
|
+
- - ">="
|
|
17
|
+
- !ruby/object:Gem::Version
|
|
18
|
+
version: '6.0'
|
|
19
|
+
type: :runtime
|
|
20
|
+
prerelease: false
|
|
21
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
22
|
+
requirements:
|
|
23
|
+
- - ">="
|
|
24
|
+
- !ruby/object:Gem::Version
|
|
25
|
+
version: '6.0'
|
|
26
|
+
- !ruby/object:Gem::Dependency
|
|
27
|
+
name: rake
|
|
28
|
+
requirement: !ruby/object:Gem::Requirement
|
|
29
|
+
requirements:
|
|
30
|
+
- - "~>"
|
|
31
|
+
- !ruby/object:Gem::Version
|
|
32
|
+
version: '13.0'
|
|
33
|
+
type: :development
|
|
34
|
+
prerelease: false
|
|
35
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
36
|
+
requirements:
|
|
37
|
+
- - "~>"
|
|
38
|
+
- !ruby/object:Gem::Version
|
|
39
|
+
version: '13.0'
|
|
40
|
+
- !ruby/object:Gem::Dependency
|
|
41
|
+
name: rspec
|
|
42
|
+
requirement: !ruby/object:Gem::Requirement
|
|
43
|
+
requirements:
|
|
44
|
+
- - "~>"
|
|
45
|
+
- !ruby/object:Gem::Version
|
|
46
|
+
version: '3.0'
|
|
47
|
+
type: :development
|
|
48
|
+
prerelease: false
|
|
49
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
50
|
+
requirements:
|
|
51
|
+
- - "~>"
|
|
52
|
+
- !ruby/object:Gem::Version
|
|
53
|
+
version: '3.0'
|
|
54
|
+
- !ruby/object:Gem::Dependency
|
|
55
|
+
name: rubocop
|
|
56
|
+
requirement: !ruby/object:Gem::Requirement
|
|
57
|
+
requirements:
|
|
58
|
+
- - ">="
|
|
59
|
+
- !ruby/object:Gem::Version
|
|
60
|
+
version: '0'
|
|
61
|
+
type: :development
|
|
62
|
+
prerelease: false
|
|
63
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
64
|
+
requirements:
|
|
65
|
+
- - ">="
|
|
66
|
+
- !ruby/object:Gem::Version
|
|
67
|
+
version: '0'
|
|
68
|
+
description: Sidekiq server middleware that re-queues a job when the class meant to
|
|
69
|
+
run it is not yet loaded in the current process — the race you hit during a rolling
|
|
70
|
+
deploy that adds a new job class.
|
|
71
|
+
email:
|
|
72
|
+
- dmitry.a.tsepelev@gmail.com
|
|
73
|
+
executables: []
|
|
74
|
+
extensions: []
|
|
75
|
+
extra_rdoc_files: []
|
|
76
|
+
files:
|
|
77
|
+
- CHANGELOG.md
|
|
78
|
+
- LICENSE.txt
|
|
79
|
+
- README.md
|
|
80
|
+
- lib/sidekiq/requeue_missing_class.rb
|
|
81
|
+
- lib/sidekiq/requeue_missing_class/configuration.rb
|
|
82
|
+
- lib/sidekiq/requeue_missing_class/middleware.rb
|
|
83
|
+
- lib/sidekiq/requeue_missing_class/version.rb
|
|
84
|
+
homepage: https://github.com/DmitryTsepelev/sidekiq-requeue_missing_class
|
|
85
|
+
licenses:
|
|
86
|
+
- MIT
|
|
87
|
+
metadata:
|
|
88
|
+
homepage_uri: https://github.com/DmitryTsepelev/sidekiq-requeue_missing_class
|
|
89
|
+
source_code_uri: https://github.com/DmitryTsepelev/sidekiq-requeue_missing_class
|
|
90
|
+
changelog_uri: https://github.com/DmitryTsepelev/sidekiq-requeue_missing_class/blob/master/CHANGELOG.md
|
|
91
|
+
rubygems_mfa_required: 'true'
|
|
92
|
+
rdoc_options: []
|
|
93
|
+
require_paths:
|
|
94
|
+
- lib
|
|
95
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
|
96
|
+
requirements:
|
|
97
|
+
- - ">="
|
|
98
|
+
- !ruby/object:Gem::Version
|
|
99
|
+
version: '3.0'
|
|
100
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
101
|
+
requirements:
|
|
102
|
+
- - ">="
|
|
103
|
+
- !ruby/object:Gem::Version
|
|
104
|
+
version: '0'
|
|
105
|
+
requirements: []
|
|
106
|
+
rubygems_version: 4.0.3
|
|
107
|
+
specification_version: 4
|
|
108
|
+
summary: Survive rolling deploys that introduce a new Sidekiq/ActiveJob class
|
|
109
|
+
test_files: []
|