sidekiq 3.4.1 → 7.3.0

data/Changes.md

@@ -1,3 +1,1117 @@
+# Sidekiq Changes
+
+[Sidekiq Changes](https://github.com/sidekiq/sidekiq/blob/main/Changes.md) | [Sidekiq Pro Changes](https://github.com/sidekiq/sidekiq/blob/main/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/sidekiq/sidekiq/blob/main/Ent-Changes.md)
+
+7.3.0
+----------
+
+- **NEW FEATURE** Add `Sidekiq::IterableJob`, iteration support for long-running jobs. [#6286, fatkodima]
+  Iterable jobs are interruptible and can restart quickly if
+  running during a deploy. You must ensure that `each_iteration`
+  doesn't take more than Sidekiq's `-t` timeout (default: 25 seconds). Iterable jobs must not implement `perform`.
+```ruby
+class ProcessArrayJob
+  include Sidekiq::IterableJob
+  def build_enumerator(*args, **kwargs)
+    array_enumerator(args, **kwargs)
+  end
+  def each_iteration(arg)
+    puts arg
+  end
+end
+ProcessArrayJob.perform_async(1, 2, 3)
+```
+See the [Iteration](//github.com/sidekiq/sidekiq/wiki/Iteration) wiki page and the RDoc in `Sidekiq::IterableJob`.
+This feature should be considered BETA until the next minor release.
+- **SECURITY** The Web UI no longer allows extensions to use `<script>`.
+  Adjust CSP to disallow inline scripts within the Web UI. Please see
+  `examples/webui-ext` for how to register Web UI extensions and use
+  dynamic CSS and JS. This will make Sidekiq immune to XSS attacks. [#6270]
+- Add config option, `:skip_default_job_logging` to disable Sidekiq's default
+  start/finish job logging. [#6200]
+- Allow `Sidekiq::Limiter.redis` to use Redis Cluster [#6288]
+- Retain CurrentAttributeѕ after inline execution [#6307]
+- Ignore non-existent CurrentAttributes attributes when restoring [#6341]
+- Raise default Redis {read,write,connect} timeouts from 1 to 3 seconds
+  to minimize ReadTimeoutErrors [#6162]
+- Add `logger` as a dependency since it will become bundled in Ruby 3.5 [#6320]
+- Ignore unsupported locales in the Web UI [#6313]
+
+7.2.4
+----------
+
+- Fix XSS in metrics filtering introduced in 7.2.0, CVE-2024-32887
+  Thanks to @UmerAdeemCheema for the security report.
+
+7.2.3
+----------
+
+- [Support Dragonfly.io](https://www.mikeperham.com/2024/02/01/supporting-dragonfly/) as an alternative Redis implementation
+- Fix error unpacking some compressed error backtraces [#6241]
+- Fix potential heartbeat data leak [#6227]
+- Add ability to find a currently running work by jid [#6212, fatkodima]
+
+7.2.2
+----------
+
+- Add `Process.warmup` call in Ruby 3.3+
+- Batch jobs now skip transactional push [#6160]
+
+7.2.1
+----------
+
+- Add `Sidekiq::Work` type which replaces the raw Hash as the third parameter in
+  `Sidekiq::WorkSet#each { |pid, tid, hash| ... }` [#6145]
+- **DEPRECATED**: direct access to the attributes within the `hash` block parameter above.
+  The `Sidekiq::Work` instance contains accessor methods to get at the same data, e.g.
+```ruby
+work["queue"] # Old
+work.queue # New
+```
+- Fix Ruby 3.3 warnings around `base64` gem [#6151, earlopain]
+
+7.2.0
+----------
+
+- `sidekiq_retries_exhausted` can return `:discard` to avoid the deadset
+  and all death handlers [#6091]
+- Metrics filtering by job class in Web UI [#5974]
+- Better readability and formatting for numbers within the Web UI [#6080]
+- Add explicit error if user code tries to nest test modes [#6078]
+```ruby
+Sidekiq::Testing.inline! # global setting
+Sidekiq::Testing.fake! do # override within block
+  # ok
+  Sidekiq::Testing.inline! do # can't override the override
+    # not ok, nested
+  end
+end
+```
+- **SECURITY** Forbid inline JavaScript execution in Web UI [#6074]
+- Adjust redis-client adapter to avoid `method_missing` [#6083]
+  This can result in app code breaking if your app's Redis API usage was
+  depending on Sidekiq's adapter to correct invalid redis-client API usage.
+  One example:
+```ruby
+# bad, not redis-client native
+# Unsupported command argument type: TrueClass (TypeError)
+Sidekiq.redis { |c| c.set("key", "value", nx: true, ex: 15) }
+# good
+Sidekiq.redis { |c| c.set("key", "value", "nx", "ex", 15) }
+```
+
+7.1.6
+----------
+
+- The block forms of testing modes (inline, fake) are now thread-safe so you can have
+  a multithreaded test suite which uses different modes for different tests. [#6069]
+- Fix breakage with non-Proc error handlers [#6065]
+
+7.1.5
+----------
+
+- **FEATURE**: Job filtering within the Web UI. This feature has been open
+  sourced from Sidekiq Pro. [#6052]
+- **API CHANGE** Error handlers now take three arguments `->(ex, context, config)`.
+  The previous calling convention will work until Sidekiq 8.0 but will print
+  out a deprecation warning. [#6051]
+- Fix issue with the `batch_size` and `at` options in `S::Client.push_bulk` [#6040]
+- Fix inline testing firing batch callbacks early [#6057]
+- Use new log broadcast API in Rails 7.1 [#6054]
+- Crash if user tries to use RESP2 `protocol: 2` [#6061]
+
+7.1.4
+----------
+
+- Fix empty `retry_for` logic [#6035]
+
+7.1.3
+----------
+
+- Add `sidekiq_options retry_for: 48.hours` to allow time-based retry windows [#6029]
+- Support sidekiq_retry_in and sidekiq_retries_exhausted_block in ActiveJobs (#5994)
+- Lowercase all Rack headers for Rack 3.0 [#5951]
+- Validate Sidekiq::Web page refresh delay to avoid potential DoS,
+  CVE-2023-26141, thanks for reporting Keegan!
+
+7.1.2
+----------
+
+- Mark Web UI assets as private so CDNs won't cache them [#5936]
+- Fix stackoverflow when using Oj and the JSON log formatter [#5920]
+- Remove spurious `enqueued_at` from scheduled ActiveJobs [#5937]
+
+7.1.1
+----------
+
+- Support multiple CurrentAttributes [#5904]
+- Speed up latency fetch with large queues on Redis <7 [#5910]
+- Allow a larger default client pool [#5886]
+- Ensure Sidekiq.options[:environment] == RAILS_ENV [#5932]
+
+7.1.0
+----------
+
+- Improve display of ActiveJob arguments in Web UI [#5825, cover]
+- Update `push_bulk` to push `batch_size` jobs at a time and allow laziness [#5827, fatkodima]
+  This allows Sidekiq::Client to push unlimited jobs as long as it has enough memory for the batch_size.
+- Update `perform_bulk` to use `push_bulk` internally.
+- Change return value of `push_bulk` to map 1-to-1 with arguments.
+  If you call `push_bulk(args: [[1], [2], [3]])`, you will now always get
+  an array of 3 values as the result: `["jid1", nil, "jid3"]` where nil means
+  that particular job did not push successfully (possibly due to middleware
+  stopping it). Previously nil values were removed so it was impossible to tell
+  which jobs pushed successfully and which did not.
+- Migrate away from all deprecated Redis commands [#5788]
+  Sidekiq will now print a warning if you use one of those deprecated commands.
+- Prefix all Sidekiq thread names [#5872]
+
+7.0.9
+----------
+
+- Restore confirmation dialogs in Web UI [#5881, shevaun]
+- Increase fetch timeout to minimize ReadTimeoutError [#5874]
+- Reverse histogram tooltip ordering [#5868]
+- Add Scottish Gaelic (gd) locale [#5867, GunChleoc]
+
+7.0.8
+----------
+
+- **SECURITY** Sanitize `period` input parameter on Metrics pages.
+  Specially crafted values can lead to XSS. This functionality
+  was introduced in 7.0.4. Thank you to spercex @ huntr.dev [#5694]
+- Add job hash as 3rd parameter to the `sidekiq_retry_in` block.
+
+7.0.7
+----------
+
+- Fix redis-client API usage which could result in stuck Redis
+connections [#5823]
+- Fix AS::Duration with `sidekiq_retry_in` [#5806]
+- Restore dumping config options on startup with `-v` [#5822]
+
+7.0.5,7.0.6
+----------
+
+- More context for debugging json unsafe errors [#5787]
+
+7.0.4
+----------
+
+- Performance and memory optimizations [#5768, fatkodima]
+- Add 1-8 hour period selector to Metrics pages [#5694]
+- Fix process display with `sidekiqmon` [#5733]
+
+7.0.3
+----------
+
+- Don't warn about memory policy on Redis Enterprise [#5712]
+- Don't allow Quiet/Stop on embedded Sidekiq instances [#5716]
+- Fix `size: X` for configuring the default Redis pool size [#5702]
+- Improve the display of queue weights on Busy page [#5642]
+- Freeze CurrentAttributes on a job once initially set [#5692]
+
+7.0.2
+----------
+
+- Improve compatibility with custom loggers [#5673]
+- Add queue weights on Busy page [#5640]
+- Add BID link on job_info page if job is part of a Batch [#5623]
+- Allow custom extensions to add rows/links within Job detail pages [#5624]
+```ruby
+Sidekiq::Web.custom_job_info_rows << AddAccountLink.new
+
+class AddAccountLink
+  include CGI::Util
+  def add_pair(job)
+    # yield a (name, value) pair
+    # You can include HTML tags and CSS, Sidekiq does not do any
+    # escaping so beware user data injection! Note how we use CGI's
+    # `h` escape helper.
+    aid = job["account_id"]
+    yield "Account", "<a href='/accounts/#{h aid}'>#{h aid}</a>" if aid
+  end
+end
+```
+
+7.0.1
+----------
+
+- Allow an embedding process to reuse its own heartbeat thread
+- Update zh-cn localization
+
+7.0.0
+----------
+
+- Embedded mode!
+- Capsules!!
+- Job Execution metrics!!!
+- See `docs/7.0-Upgrade.md` for release notes
+
+6.5.{10,11,12}
+----------
+
+- Fixes for Rails 7.1 [#6067, #6070]
+
+6.5.9
+----------
+
+- Ensure Sidekiq.options[:environment] == RAILS_ENV [#5932]
+
+6.5.8
+----------
+
+- Fail if using a bad version of scout_apm [#5616]
+- Add pagination to Busy page [#5556]
+- Speed up WorkSet#each [#5559]
+- Adjust CurrentAttributes to work with the String class name so we aren't referencing the Class within a Rails initializer [#5536]
+
+6.5.7
+----------
+
+- Updates for JA and ZH locales
+- Further optimizations for scheduled polling [#5513]
+
+6.5.6
+----------
+
+- Fix deprecation warnings with redis-rb 4.8.0 [#5484]
+- Lock redis-rb to < 5.0 as we are moving to redis-client in Sidekiq 7.0
+
+6.5.5
+----------
+
+- Fix require issue with job_retry.rb [#5462]
+- Improve Sidekiq::Web compatibility with Rack 3.x
+
+6.5.4
+----------
+
+- Fix invalid code on Ruby 2.5 [#5460]
+- Fix further metrics dependency issues [#5457]
+
+6.5.3
+----------
+
+- Don't require metrics code without explicit opt-in [#5456]
+
+6.5.2
+----------
+
+- [Job Metrics are under active development, help wanted!](https://github.com/sidekiq/sidekiq/wiki/Metrics#contributing) **BETA**
+- Add `Context` column on queue page which shows any CurrentAttributes [#5450]
+- `sidekiq_retry_in` may now return `:discard` or `:kill` to dynamically stop job retries [#5406]
+- Smarter sorting of processes in /busy Web UI [#5398]
+- Fix broken hamburger menu in mobile UI [#5428]
+- Require redis-rb 4.5.0. Note that Sidekiq will break if you use the
+  [`Redis.exists_returns_integer = false`](https://github.com/redis/redis-rb/blob/master/CHANGELOG.md#450) flag. [#5394]
+
+6.5.1
+----------
+
+- Fix `push_bulk` breakage [#5387]
+
+6.5.0
+---------
+
+- Substantial refactoring of Sidekiq server internals, part of a larger effort
+  to reduce Sidekiq's internal usage of global methods and data, see [docs/global_to_local.md](docs/global_to_local.md) and [docs/middleware.md](docs/middleware.md).
+- **Add beta support for the `redis-client` gem**. This will become the default Redis driver in Sidekiq 7.0. [#5298]
+  Read more: https://github.com/sidekiq/sidekiq/wiki/Using-redis-client
+- **Add beta support for DB transaction-aware client** [#5291]
+  Add this line to your initializer and any jobs created during a transaction
+  will only be pushed to Redis **after the transaction commits**. You will need to add the
+  `after_commit_everywhere` gem to your Gemfile.
+```ruby
+Sidekiq.transactional_push!
+```
+  This feature does not have a lot of production usage yet; please try it out and let us
+  know if you have any issues. It will be fully supported in Sidekiq 7.0 or removed if it
+  proves problematic.
+- Fix regression with middleware arguments [#5312]
+
+6.4.2
+---------
+
+- Strict argument checking now runs after client-side middleware [#5246]
+- Fix page events with live polling [#5184]
+- Many under-the-hood changes to remove all usage of the term "worker"
+  from the Sidekiq codebase and APIs. This mostly involved RDoc and local
+  variable names but a few constants and public APIs were changed. The old
+  APIs will be removed in Sidekiq 7.0.
+```
+Sidekiq::DEFAULT_WORKER_OPTIONS -> Sidekiq.default_job_options
+Sidekiq.default_worker_options -> Sidekiq.default_job_options
+Sidekiq::Queues["default"].jobs_by_worker(HardJob) -> Sidekiq::Queues["default"].jobs_by_class(HardJob)
+```
+
+6.4.1
+---------
+
+- Fix pipeline/multi deprecations in redis-rb 4.6
+- Fix sidekiq.yml YAML load errors on Ruby 3.1 [#5141]
+- Sharding support for `perform_bulk` [#5129]
+- Refactor job logger for SPEEEEEEED
+
+6.4.0
+---------
+
+- **SECURITY**: Validate input to avoid possible DoS in Web UI.
+- Add **strict argument checking** [#5071]
+  Sidekiq will now log a warning if JSON-unsafe arguments are passed to `perform_async`.
+  Add `Sidekiq.strict_args!(false)` to your initializer to disable this warning.
+  This warning will switch to an exception in Sidekiq 7.0.
+- Note that Delayed Extensions will be removed in Sidekiq 7.0 [#5076]
+- Add `perform_{inline,sync}` in Sidekiq::Job to run a job synchronously [#5061, hasan-ally]
+```ruby
+SomeJob.perform_async(args...)
+SomeJob.perform_sync(args...)
+SomeJob.perform_inline(args...)
+```
+  You can also dynamically redirect a job to run synchronously:
+```ruby
+SomeJob.set("sync": true).perform_async(args...) # will run via perform_inline
+```
+- Replace Sidekiq::Worker `app/workers` generator with Sidekiq::Job `app/sidekiq` generator [#5055]
+```
+bin/rails generate sidekiq:job ProcessOrderJob
+```
+- Fix job retries losing CurrentAttributes [#5090]
+- Tweak shutdown to give long-running threads time to cleanup [#5095]
+
+6.3.1
+---------
+
+- Fix keyword arguments error with CurrentAttributes on Ruby 3.0 [#5048]
+
+6.3.0
+---------
+
+- **BREAK**: The Web UI has been refactored to remove jQuery. Any UI extensions
+  which use jQuery will break.
+- **FEATURE**: Sidekiq.logger has been enhanced so any `Rails.logger`
+  output in jobs now shows up in the Sidekiq console. Remove any logger
+  hacks in your initializer and see if it Just Works™ now. [#5021]
+- **FEATURE**: Add `Sidekiq::Job` alias for `Sidekiq::Worker`, to better
+  reflect industry standard terminology. You can now do this:
+```ruby
+class MyJob
+  include Sidekiq::Job
+  sidekiq_options ...
+  def perform(args)
+  end
+end
+```
+- **FEATURE**: Support for serializing ActiveSupport::CurrentAttributes into each job. [#4982]
+```ruby
+# config/initializers/sidekiq.rb
+require "sidekiq/middleware/current_attributes"
+Sidekiq::CurrentAttributes.persist(Myapp::Current) # Your AS::CurrentAttributes singleton
+```
+- **FEATURE**: Add `Sidekiq::Worker.perform_bulk` for enqueuing jobs in bulk,
+  similar to `Sidekiq::Client.push_bulk` [#5042]
+```ruby
+MyJob.perform_bulk([[1], [2], [3]])
+```
+- Implement `queue_as`, `wait` and `wait_until` for ActiveJob compatibility [#5003]
+- Scheduler now uses Lua to reduce Redis load and network roundtrips [#5044]
+- Retry Redis operation if we get an `UNBLOCKED` Redis error [#4985]
+- Run existing signal traps, if any, before running Sidekiq's trap [#4991]
+- Fix fetch bug when using weighted queues which caused Sidekiq to stop
+  processing queues randomly [#5031]
+
+6.2.2
+---------
+
+- Reduce retry jitter, add jitter to `sidekiq_retry_in` values [#4957]
+- Minimize scheduler load on Redis at scale [#4882]
+- Improve logging of delay jobs [#4904, BuonOno]
+- Minor CSS improvements for buttons and tables, design PRs always welcome!
+- Tweak Web UI `Cache-Control` header [#4966]
+- Rename internal API class `Sidekiq::Job` to `Sidekiq::JobRecord` [#4955]
+
+6.2.1
+---------
+
+- Update RTT warning logic to handle transient RTT spikes [#4851]
+- Fix very low priority CVE on unescaped queue name [#4852]
+- Add note about sessions and Rails apps in API mode
+
+6.2.0
+---------
+
+- Store Redis RTT and log if poor [#4824]
+- Add process/thread stats to Busy page [#4806]
+- Improve Web UI on mobile devices [#4840]
+- **Refactor Web UI session usage** [#4804]
+  Numerous people have hit "Forbidden" errors and struggled with Sidekiq's
+  Web UI session requirement. If you have code in your initializer for
+  Web sessions, it's quite possible it will need to be removed. Here's
+  an overview:
+```
+Sidekiq::Web needs a valid Rack session for CSRF protection. If this is a Rails app,
+make sure you mount Sidekiq::Web *inside* your routes in `config/routes.rb` so
+Sidekiq can reuse the Rails session:
+
+  Rails.application.routes.draw do
+    mount Sidekiq::Web => "/sidekiq"
+    ....
+  end
+
+If this is a bare Rack app, use a session middleware before Sidekiq::Web:
+
+  # first, use IRB to create a shared secret key for sessions and commit it
+  require 'securerandom'; File.open(".session.key", "w") {|f| f.write(SecureRandom.hex(32)) }
+
+  # now, update your Rack app to include the secret with a session cookie middleware
+  use Rack::Session::Cookie, secret: File.read(".session.key"), same_site: true, max_age: 86400
+  run Sidekiq::Web
+
+If this is a Rails app in API mode, you need to enable sessions.
+
+  https://guides.rubyonrails.org/api_app.html#using-session-middlewares
+```
+
+6.1.3
+---------
+
+- Warn if Redis is configured to evict data under memory pressure [#4752]
+- Add process RSS on the Busy page [#4717]
+
+6.1.2
+---------
+
+- Improve readability in dark mode Web UI [#4674]
+- Fix Web UI crash with corrupt session [#4672]
+- Allow middleware to yield arguments [#4673, @eugeneius]
+- Migrate CI from CircleCI to GitHub Actions [#4677]
+
+6.1.1
+---------
+
+- Jobs are now sorted by age in the Busy Workers table. [#4641]
+- Fix "check all" JS logic in Web UI [#4619]
+
+6.1.0
+---------
+
+- Web UI - Dark Mode fixes [#4543, natematykiewicz]
+- Ensure `Rack::ContentLength` is loaded as middleware for correct Web UI responses [#4541]
+- Avoid exception dumping SSL store in Redis connection logging [#4532]
+- Better error messages in Sidekiq::Client [#4549]
+- Remove rack-protection, reimplement CSRF protection [#4588]
+- Require redis-rb 4.2 [#4591]
+- Update to jquery 1.12.4 [#4593]
+- Refactor internal fetch logic and API [#4602]
+
+6.0.7
+---------
+
+- Refactor systemd integration to work better with custom binaries [#4511]
+- Don't connect to Redis at process exit if not needed [#4502]
+- Remove Redis connection naming [#4479]
+- Fix Redis Sentinel password redaction [#4499]
+- Add Vietnamese locale (vi) [#4528]
+
+6.0.6
+---------
+
+- **Integrate with systemd's watchdog and notification features** [#4488]
+  Set `Type=notify` in [sidekiq.service](https://github.com/sidekiq/sidekiq/blob/4b8a8bd3ae42f6e48ae1fdaf95ed7d7af18ed8bb/examples/systemd/sidekiq.service#L30-L39). The integration works automatically.
+- Use `setTimeout` rather than `setInterval` to avoid thundering herd [#4480]
+- Fix edge case where a job can be pushed without a queue.
+- Flush job stats at exit [#4498]
+- Check RAILS_ENV before RACK_ENV [#4493]
+- Add Lithuanian locale [#4476]
+
+6.0.5
+---------
+
+- Fix broken Web UI response when using NewRelic and Rack 2.1.2+. [#4440]
+- Update APIs to use `UNLINK`, not `DEL`. [#4449]
+- Fix Ruby 2.7 warnings [#4412]
+- Add support for `APP_ENV` [[95fa5d9]](https://github.com/sidekiq/sidekiq/commit/95fa5d90192148026e52ca2902f1b83c70858ce8)
+
+6.0.4
+---------
+
+- Fix ActiveJob's `sidekiq_options` integration [#4404]
+- Sidekiq Pro users will now see a Pause button next to each queue in
+  the Web UI, allowing them to pause queues manually [#4374, shayonj]
+- Fix Sidekiq::Workers API unintentional change in 6.0.2 [#4387]
+
+
+6.0.3
+---------
+
+- Fix `Sidekiq::Client.push_bulk` API which was erroneously putting
+  invalid `at` values in the job payloads [#4321]
+
+6.0.2
+---------
+
+- Fix Sidekiq Enterprise's rolling restart functionality, broken by refactoring in 6.0.0. [#4334]
+- More internal refactoring and performance tuning [fatkodima]
+
+6.0.1
+---------
+
+- **Performance tuning**, Sidekiq should be 10-15% faster now [#4303, 4299,
+  4269, fatkodima]
+- **Dark Mode support in Web UI** (further design polish welcome!) [#4227, mperham,
+  fatkodima, silent-e]
+- **Job-specific log levels**, allowing you to turn on debugging for
+  problematic workers. [fatkodima, #4287]
+```ruby
+MyWorker.set(log_level: :debug).perform_async(...)
+```
+- **Ad-hoc job tags**. You can tag your jobs with, e.g, subdomain, tenant, country,
+  locale, application, version, user/client, "alpha/beta/pro/ent", types of jobs,
+  teams/people responsible for jobs, additional metadata, etc.
+  Tags are shown on different pages with job listings. Sidekiq Pro users
+  can filter based on them [fatkodima, #4280]
+```ruby
+class MyWorker
+  include Sidekiq::Worker
+  sidekiq_options tags: ['bank-ops', 'alpha']
+  ...
+end
+```
+- Fetch scheduled jobs in batches before pushing into specific queues.
+  This will decrease enqueueing time of scheduled jobs by a third. [fatkodima, #4273]
+```
+ScheduledSet with 10,000 jobs
+Before: 56.6 seconds
+After:  39.2 seconds
+```
+- Compress error backtraces before pushing into Redis, if you are
+  storing error backtraces, this will halve the size of your RetrySet
+  in Redis [fatkodima, #4272]
+```
+RetrySet with 100,000 jobs
+Before: 261 MB
+After:  129 MB
+```
+- Support display of ActiveJob 6.0 payloads in the Web UI [#4263]
+- Add `SortedSet#scan` for pattern based scanning. For large sets this API will be **MUCH** faster
+  than standard iteration using each. [fatkodima, #4262]
+```ruby
+  Sidekiq::DeadSet.new.scan("UnreliableApi") do |job|
+    job.retry
+  end
+```
+- Dramatically speed up SortedSet#find\_job(jid) by using Redis's ZSCAN
+  support, approx 10x faster. [fatkodima, #4259]
+```
+zscan   0.179366   0.047727   0.227093 (  1.161376)
+enum    8.522311   0.419826   8.942137 (  9.785079)
+```
+- Respect rails' generators `test_framework` option and gracefully handle extra `worker` suffix on generator [fatkodima, #4256]
+- Add ability to sort 'Enqueued' page on Web UI by position in the queue [fatkodima, #4248]
+- Support `Client.push_bulk` with different delays [fatkodima, #4243]
+```ruby
+Sidekiq::Client.push_bulk("class" => FooJob, "args" => [[1], [2]], "at" => [1.minute.from_now.to_f, 5.minutes.from_now.to_f])
+```
+- Easier way to test enqueuing specific ActionMailer and ActiveRecord delayed jobs. Instead of manually
+  parsing embedded class, you can now test by fetching jobs for specific classes. [fatkodima, #4292]
+```ruby
+assert_equal 1, Sidekiq::Extensions::DelayedMailer.jobs_for(FooMailer).size
+```
+- Add `sidekiqmon` to gemspec executables [#4242]
+- Gracefully handle `Sidekiq.logger = nil` [#4240]
+- Inject Sidekiq::LogContext module if user-supplied logger does not include it [#4239]
+
+6.0
+---------
+
+This release has major breaking changes.  Read and test carefully in production.
+
+- With Rails 6.0.2+, ActiveJobs can now use `sidekiq_options` directly to configure Sidekiq
+  features/internals like the retry subsystem. [#4213, pirj]
+```ruby
+class MyJob < ActiveJob::Base
+  queue_as :myqueue
+  sidekiq_options retry: 10, backtrace: 20
+  def perform(...)
+  end
+end
+```
+- Logging has been redesigned to allow for pluggable log formatters:
+```ruby
+Sidekiq.configure_server do |config|
+  config.log_formatter = Sidekiq::Logger::Formatters::JSON.new
+end
+```
+See the [Logging wiki page](https://github.com/sidekiq/sidekiq/wiki/Logging) for more details.
+- **BREAKING CHANGE** Validate proper usage of the `REDIS_PROVIDER`
+  variable.  This variable is meant to hold the name of the environment
+  variable which contains your Redis URL, so that you can switch Redis
+  providers quickly and easily with a single variable change.  It is not
+  meant to hold the actual Redis URL itself.  If you want to manually set
+  the Redis URL (not recommended as it implies you have no failover),
+  then you may set `REDIS_URL` directly. [#3969]
+- **BREAKING CHANGE** Increase default shutdown timeout from 8 seconds
+  to 25 seconds.  Both Heroku and ECS now use 30 second shutdown timeout
+  by default and we want Sidekiq to take advantage of this time.  If you
+  have deployment scripts which depend on the old default timeout, use `-t 8` to
+  get the old behavior. [#3968]
+- **BREAKING CHANGE** Remove the daemonization, logfile and pidfile
+  arguments to Sidekiq. Use a proper process supervisor (e.g. systemd or
+  foreman) to manage Sidekiq.  See the Deployment wiki page for links to
+  more resources.
+- Integrate the StandardRB code formatter to ensure consistent code
+  styling. [#4114, gearnode]
+
+5.2.10
+---------
+
+- Backport fix for CVE-2022-23837.
+- Migrate to `exists?` for redis-rb.
+- Lock redis-rb to <4.6 to avoid deprecations.
+
+5.2.9
+---------
+
+- Release Rack lock due to a cascade of CVEs. [#4566]
+  Pro-tip: don't lock Rack.
+
+5.2.8
+---------
+
+- Lock to Rack 2.0.x to prevent future incompatibilities
+- Fix invalid reference in `sidekiqctl`
+
+5.2.7
+---------
+
+- Fix stale `enqueued_at` when retrying [#4149]
+- Move build to [Circle CI](https://circleci.com/gh/mperham/sidekiq) [#4120]
+
+5.2.6
+---------
+
+- Fix edge case where a job failure during Redis outage could result in a lost job [#4141]
+- Better handling of malformed job arguments in payload [#4095]
+- Restore bootstap's dropdown css component [#4099, urkle]
+- Display human-friendly time diff for longer queue latencies [#4111, interlinked]
+- Allow `Sidekiq::Worker#set` to be chained
+
+5.2.5
+---------
+
+- Fix default usage of `config/sidekiq.yml` [#4077, Tensho]
+
+5.2.4
+---------
+
+- Add warnings for various deprecations and changes coming in Sidekiq 6.0.
+  See the 6-0 branch. [#4056]
+- Various improvements to the Sidekiq test suite and coverage [#4026, #4039, Tensho]
+
+5.2.3
+---------
+
+- Warning message on invalid REDIS\_PROVIDER [#3970]
+- Add `sidekiqctl status` command [#4003, dzunk]
+- Update elapsed time calculatons to use monotonic clock [#3999]
+- Fix a few issues with mobile Web UI styling [#3973, navied]
+- Jobs with `retry: false` now go through the global `death_handlers`,
+  meaning you can take action on failed ephemeral jobs. [#3980, Benjamin-Dobell]
+- Fix race condition in defining Workers. [#3997, mattbooks]
+
+5.2.2
+---------
+
+- Raise error for duplicate queue names in config to avoid unexpected fetch algorithm change [#3911]
+- Fix concurrency bug on JRuby [#3958, mattbooks]
+- Add "Kill All" button to the retries page [#3938]
+
+5.2.1
+-----------
+
+- Fix concurrent modification error during heartbeat [#3921]
+
+5.2.0
+-----------
+
+- **Decrease default concurrency from 25 to 10** [#3892]
+- Verify connection pool sizing upon startup [#3917]
+- Smoother scheduling for large Sidekiq clusters [#3889]
+- Switch Sidekiq::Testing impl from alias\_method to Module#prepend, for resiliency [#3852]
+- Update Sidekiq APIs to use SCAN for scalability [#3848, ffiller]
+- Remove concurrent-ruby gem dependency [#3830]
+- Optimize Web UI's bootstrap.css [#3914]
+
+5.1.3
+-----------
+
+- Fix version comparison so Ruby 2.2.10 works. [#3808, nateberkopec]
+
+5.1.2
+-----------
+
+- Add link to docs in Web UI footer
+- Fix crash on Ctrl-C in Windows [#3775, Bernica]
+- Remove `freeze` calls on String constants. This is superfluous with Ruby
+  2.3+ and `frozen_string_literal: true`. [#3759]
+- Fix use of AR middleware outside of Rails [#3787]
+- Sidekiq::Worker `sidekiq_retry_in` block can now return nil or 0 to use
+  the default backoff delay [#3796, dsalahutdinov]
+
+5.1.1
+-----------
+
+- Fix Web UI incompatibility with Redis 3.x gem [#3749]
+
+5.1.0
+-----------
+
+- **NEW** Global death handlers - called when your job exhausts all
+  retries and dies.  Now you can take action when a job fails permanently. [#3721]
+- **NEW** Enable ActiveRecord query cache within jobs by default [#3718, sobrinho]
+  This will prevent duplicate SELECTS; cache is cleared upon any UPDATE/INSERT/DELETE.
+  See the issue for how to bypass the cache or disable it completely.
+- Scheduler timing is now more accurate, 15 -> 5 seconds [#3734]
+- Exceptions during the :startup event will now kill the process [#3717]
+- Make `Sidekiq::Client.via` reentrant [#3715]
+- Fix use of Sidekiq logger outside of the server process [#3714]
+- Tweak `constantize` to better match Rails class lookup. [#3701, caffeinated-tech]
+
+5.0.5
+-----------
+
+- Update gemspec to allow newer versions of the Redis gem [#3617]
+- Refactor Worker.set so it can be memoized [#3602]
+- Fix display of Redis URL in web footer, broken in 5.0.3 [#3560]
+- Update `Sidekiq::Job#display_args` to avoid mutation [#3621]
+
+5.0.4
+-----------
+
+- Fix "slow startup" performance regression from 5.0.2. [#3525]
+- Allow users to disable ID generation since some redis providers disable the CLIENT command. [#3521]
+
+5.0.3
+-----------
+
+- Fix overriding `class_attribute` core extension from ActiveSupport with Sidekiq one [PikachuEXE, #3499]
+- Allow job logger to be overridden [AlfonsoUceda, #3502]
+- Set a default Redis client identifier for debugging [#3516]
+- Fix "Uninitialized constant" errors on startup with the delayed extensions [#3509]
+
+5.0.2
+-----------
+
+- fix broken release, thanks @nateberkopec
+
+5.0.1
+-----------
+
+- Fix incorrect server identity when daemonizing [jwilm, #3496]
+- Work around error running Web UI against Redis Cluster [#3492]
+- Remove core extensions, Sidekiq is now monkeypatch-free! [#3474]
+- Reimplement Web UI's HTTP\_ACCEPT\_LANGUAGE parsing because the spec is utterly
+  incomprehensible for various edge cases. [johanlunds, natematykiewicz, #3449]
+- Update `class_attribute` core extension to avoid warnings
+- Expose `job_hash_context` from `Sidekiq::Logging` to support log customization
+
+5.0.0
+-----------
+
+- **BREAKING CHANGE** Job dispatch was refactored for safer integration with
+  Rails 5.  The **Logging** and **RetryJobs** server middleware were removed and
+  functionality integrated directly into Sidekiq::Processor.  These aren't
+  commonly used public APIs so this shouldn't impact most users.
+```
+Sidekiq::Middleware::Server::RetryJobs -> Sidekiq::JobRetry
+Sidekiq::Middleware::Server::Logging -> Sidekiq::JobLogger
+```
+- Quieting Sidekiq is now done via the TSTP signal, the USR1 signal is deprecated.
+- The `delay` extension APIs are no longer available by default, you
+  must opt into them.
+- The Web UI is now BiDi and can render RTL languages like Arabic, Farsi and Hebrew.
+- Rails 3.2 and Ruby 2.0 and 2.1 are no longer supported.
+- The `SomeWorker.set(options)` API was re-written to avoid thread-local state. [#2152]
+- Sidekiq Enterprise's encrypted jobs now display "[encrypted data]" in the Web UI instead
+  of random hex bytes.
+- Please see the [5.0 Upgrade notes](docs/5.0-Upgrade.md) for more detail.
+
+4.2.10
+-----------
+
+- Scheduled jobs can now be moved directly to the Dead queue via API [#3390]
+- Fix edge case leading to job duplication when using Sidekiq Pro's
+  reliability feature [#3388]
+- Fix error class name display on retry page [#3348]
+- More robust latency calculation [#3340]
+
+4.2.9
+-----------
+
+- Rollback [#3303] which broke Heroku Redis users [#3311]
+- Add support for TSTP signal, for Sidekiq 5.0 forward compatibility. [#3302]
+
+4.2.8
+-----------
+
+- Fix rare edge case with Redis driver that can create duplicate jobs [#3303]
+- Fix Rails 5 loading issue [#3275]
+- Restore missing tooltips to timestamps in Web UI [#3310]
+- Work on **Sidekiq 5.0** is now active! [#3301]
+
+4.2.7
+-----------
+
+- Add new integration testing to verify code loading and job execution
+  in development and production modes with Rails 4 and 5 [#3241]
+- Fix delayed extensions in development mode [#3227, DarthSim]
+- Use Worker's `retry` default if job payload does not have a retry
+  attribute [#3234, mlarraz]
+
+4.2.6
+-----------
+
+- Run Rails Executor when in production [#3221, eugeneius]
+
+4.2.5
+-----------
+
+- Re-enable eager loading of all code when running non-development Rails 5. [#3203]
+- Better root URL handling for zany web servers [#3207]
+
+4.2.4
+-----------
+
+- Log errors coming from the Rails 5 reloader. [#3212, eugeneius]
+- Clone job data so middleware changes don't appear in Busy tab
+
+4.2.3
+-----------
+
+- Disable use of Rails 5's Reloader API in non-development modes, it has proven
+  to be unstable under load [#3154]
+- Allow disabling of Sidekiq::Web's cookie session to handle the
+  case where the app provides a session already [#3180, inkstak]
+```ruby
+Sidekiq::Web.set :sessions, false
+```
+- Fix Web UI sharding support broken in 4.2.2. [#3169]
+- Fix timestamps not updating during UI polling [#3193, shaneog]
+- Relax rack-protection version to >= 1.5.0
+- Provide consistent interface to exception handlers, changing the structure of the context hash. [#3161]
+
+4.2.2
+-----------
+
+- Fix ever-increasing cookie size with nginx [#3146, cconstantine]
+- Fix so Web UI works without trailing slash [#3158, timdorr]
+
+4.2.1
+-----------
+
+- Ensure browser does not cache JSON/AJAX responses. [#3136]
+- Support old Sinatra syntax for setting config [#3139]
+
+4.2.0
+-----------
+
+- Enable development-mode code reloading.  **With Rails 5.0+, you don't need
+  to restart Sidekiq to pick up your Sidekiq::Worker changes anymore!** [#2457]
+- **Remove Sinatra dependency**.  Sidekiq's Web UI now uses Rack directly.
+  Thank you to Sidekiq's newest committer, **badosu**, for writing the code
+  and doing a lot of testing to ensure compatibility with many different
+  3rd party plugins.  If your Web UI works with 4.1.4 but fails with
+  4.2.0, please open an issue. [#3075]
+- Allow tuning of concurrency with the `RAILS_MAX_THREADS` env var. [#2985]
+  This is the same var used by Puma so you can tune all of your systems
+  the same way:
+```sh
+web: RAILS_MAX_THREADS=5 bundle exec puma ...
+worker: RAILS_MAX_THREADS=10 bundle exec sidekiq ...
+```
+Using `-c` or `config/sidekiq.yml` overrides this setting.  I recommend
+adjusting your `config/database.yml` to use it too so connections are
+auto-scaled:
+```yaml
+  pool: <%= ENV['RAILS_MAX_THREADS'] || 5 %>
+```
+
+4.1.4
+-----------
+
+- Unlock Sinatra so a Rails 5.0 compatible version may be used [#3048]
+- Fix race condition on startup with JRuby [#3043]
+
+
+4.1.3
+-----------
+
+- Please note the Redis 3.3.0 gem has a [memory leak](https://github.com/redis/redis-rb/issues/612),
+  Redis 3.2.2 is recommended until that issue is fixed.
+- Sinatra 1.4.x is now a required dependency, avoiding cryptic errors
+  and old bugs due to people not upgrading Sinatra for years. [#3042]
+- Fixed race condition in heartbeat which could rarely lead to lingering
+  processes on the Busy tab. [#2982]
+```ruby
+# To clean up lingering processes, modify this as necessary to connect to your Redis.
+# After 60 seconds, lingering processes should disappear from the Busy page.
+
+require 'redis'
+r = Redis.new(url: "redis://localhost:6379/0")
+# uncomment if you need a namespace
+#require 'redis-namespace'
+#r = Redis::Namespace.new("foo", r)
+r.smembers("processes").each do |pro|
+  r.expire(pro, 60)
+  r.expire("#{pro}:workers", 60)
+end
+```
+
+
+4.1.2
+-----------
+
+- Fix Redis data leak with worker data when a busy Sidekiq process
+  crashes.  You can find and expire leaked data in Redis with this
+script:
+```bash
+$ redis-cli keys  "*:workers" | while read LINE ; do TTL=`redis-cli expire "$LINE" 60`; echo "$LINE"; done;
+```
+  Please note that `keys` can be dangerous to run on a large, busy Redis.  Caveat runner.
+- Freeze all string literals with Ruby 2.3. [#2741]
+- Client middleware can now stop bulk job push. [#2887]
+
+4.1.1
+-----------
+
+- Much better behavior when Redis disappears and comes back. [#2866]
+- Update FR locale [dbachet]
+- Don't fill logfile in case of Redis downtime [#2860]
+- Allow definition of a global retries_exhausted handler. [#2807]
+```ruby
+Sidekiq.configure_server do |config|
+  config.default_retries_exhausted = -> (job, ex) do
+    Sidekiq.logger.info "#{job['class']} job is now dead"
+  end
+end
+```
+
+4.1.0
+-----------
+
+- Tag quiet processes in the Web UI [#2757, jcarlson]
+- Pass last exception to sidekiq\_retries\_exhausted block [#2787, Nowaker]
+```ruby
+class MyWorker
+  include Sidekiq::Worker
+  sidekiq_retries_exhausted do |job, exception|
+  end
+end
+```
+- Add native support for ActiveJob's `set(options)` method allowing
+you to override worker options dynamically.  This should make it
+even easier to switch between ActiveJob and Sidekiq's native APIs [#2780]
+```ruby
+class MyWorker
+  include Sidekiq::Worker
+  sidekiq_options queue: 'default', retry: true
+
+  def perform(*args)
+    # do something
+  end
+end
+
+MyWorker.set(queue: 'high', retry: false).perform_async(1)
+```
+
+4.0.2
+-----------
+
+- Better Japanese translations
+- Remove `json` gem dependency from gemspec. [#2743]
+- There's a new testing API based off the `Sidekiq::Queues` namespace. All
+  assertions made against the Worker class still work as expected.
+  [#2676, brandonhilkert]
+```ruby
+assert_equal 0, Sidekiq::Queues["default"].size
+HardWorker.perform_async("log")
+assert_equal 1, Sidekiq::Queues["default"].size
+assert_equal "log", Sidekiq::Queues["default"].first['args'][0]
+Sidekiq::Queues.clear_all
+```
+
+4.0.1
+-----------
+
+- Yank new queue-based testing API [#2663]
+- Fix invalid constant reference in heartbeat
+
+4.0.0
+-----------
+
+- Sidekiq's internals have been completely overhauled for performance
+  and to remove dependencies.  This has resulted in major speedups, as
+  [detailed on my blog](http://www.mikeperham.com/2015/10/14/optimizing-sidekiq/).
+- See the [4.0 upgrade notes](docs/4.0-Upgrade.md) for more detail.
+
+3.5.4
+-----------
+
+- Ensure exception message is a string [#2707]
+- Revert racy Process.kill usage in sidekiqctl
+
+3.5.3
+-----------
+
+- Adjust shutdown event to run in parallel with the rest of system shutdown. [#2635]
+
+3.5.2
+-----------
+
+- **Sidekiq 3 is now in maintenance mode**, only major bugs will be fixed.
+- The exception triggering a retry is now passed into `sidekiq_retry_in`,
+  allowing you to retry more frequently for certain types of errors.
+  [#2619, kreynolds]
+```ruby
+  sidekiq_retry_in do |count, ex|
+    case ex
+    when RuntimeError
+      5 * count
+    else
+      10 * count
+    end
+  end
+```
+
+3.5.1
+-----------
+
+- **FIX MEMORY LEAK** Under rare conditions, threads may leak [#2598, gazay]
+- Add Ukrainian locale [#2561, elrakita]
+- Disconnect and retry Redis operations if we see a READONLY error [#2550]
+- Add server middleware testing harness; see [wiki](https://github.com/sidekiq/sidekiq/wiki/Testing#testing-server-middleware) [#2534, ryansch]
+
+3.5.0
+-----------
+
+- Polished new banner! [#2522, firedev]
+- Upgrade to Celluloid 0.17. [#2420, digitalextremist]
+- Activate sessions in Sinatra for CSRF protection, requires Rails
+  monkeypatch due to rails/rails#15843. [#2460, jc00ke]
+
+3.4.2
+-----------
+
+- Don't allow `Sidekiq::Worker` in ActiveJob::Base classes. [#2424]
+- Safer display of job data in Web UI [#2405]
+- Fix CSRF vulnerability in Web UI, thanks to Egor Homakov for
+  reporting. [#2422] If you are running the Web UI as a standalone Rack app,
+  ensure you have a [session middleware
+configured](https://github.com/sidekiq/sidekiq/wiki/Monitoring#standalone):
+```ruby
+use Rack::Session::Cookie, :secret => "some unique secret string here"
+```
+
 3.4.1
 -----------
 
@@ -205,7 +1319,7 @@ sidekiq_options :dead => false, :retry => 5
 3.0.0
 -----------
 
-Please see [3.0-Upgrade.md](3.0-Upgrade.md) for more comprehensive upgrade notes.
+Please see [3.0-Upgrade.md](docs/3.0-Upgrade.md) for more comprehensive upgrade notes.
 
 - **Dead Job Queue** - jobs which run out of retries are now moved to a dead
   job queue.  These jobs must be retried manually or they will expire
@@ -249,7 +1363,7 @@ Sidekiq::Client.via(ConnectionPool.new { Redis.new }) do
 end
 ```
   **Sharding support does require a breaking change to client-side
-middleware, see 3.0-Upgrade.md.**
+middleware, see docs/3.0-Upgrade.md.**
 - New Chinese, Greek, Swedish and Czech translations for the Web UI.
 - Updated most languages translations for the new UI features.
 - **Remove official Capistrano integration** - this integration has been
@@ -377,10 +1491,10 @@ middleware, see 3.0-Upgrade.md.**
   appear to be doing any work. [#1194]
 - Sidekiq's testing behavior is now dynamic.  You can choose between
   `inline` and `fake` behavior in your tests. See
-[Testing](https://github.com/mperham/sidekiq/wiki/Testing) for detail. [#1193]
+[Testing](https://github.com/sidekiq/sidekiq/wiki/Testing) for detail. [#1193]
 - The Retries table has a new column for the error message.
 - The Web UI topbar now contains the status and live poll button.
-- Orphaned worker records are now auto-vacuumed when you vist the
+- Orphaned worker records are now auto-vacuumed when you visit the
   Workers page in the Web UI.
 - Sidekiq.default\_worker\_options allows you to configure default
   options for all Sidekiq worker types.