sidekiq 7.1.4 → 8.0.9

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 0a6064918f9c33be1d21f890b9cff080969fb44a916fe67ccb36958fe0a3b8f3
-  data.tar.gz: baf268f21f27e0dac2fc287f247910afe493e4dcaee21aa769ccebbed03e7699
+  metadata.gz: ef51643693879cd574d6edd54ef41443defc4825c855d5eed0e07d42e35aa1cd
+  data.tar.gz: 2d9acb8d89326185356ee9818ca543e8a6992bac0a00262e06aa9394052978c1
 SHA512:
-  metadata.gz: c8f4e3caaeab143f20fdd592dc7939f09a302642867bddb4fcd02cbbaa0e734d7685ee3ad02543d3bb3ba6cbedaf2cbab3577a849c03d2f351aa7918d6eab73f
-  data.tar.gz: 1cf71898600afb872ee717be6496da7a146aa8e2f528b6c73e3630a64f71869bc23bfa2d7130f021c7f5e12b473ed3b537cebb4a8fedcfbe517af48dde5bbea6
+  metadata.gz: 761f705abeb6b5deb31591d07d9d9ea0d79d5d6f74367f622842994aa0a57866bd3af9add1089ce2b29d8dd4a9cd110d839036504a1ae0830ac59dbca186ff26
+  data.tar.gz: 5df2e3b47cb7ed56a51ff90b1a8d23ea96f74e9fd64a9116d249a266c7a074ef2a4bac09d66202c66a4f631c6f15bcae167df8ae733f43020f0b53886ae3e564

data/Changes.md

@@ -2,6 +2,334 @@
 
 [Sidekiq Changes](https://github.com/sidekiq/sidekiq/blob/main/Changes.md) | [Sidekiq Pro Changes](https://github.com/sidekiq/sidekiq/blob/main/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/sidekiq/sidekiq/blob/main/Ent-Changes.md)
 
+8.0.9
+----------
+
+- Implement idle Redis connection reaping, will be activated in 8.1 [#6663]
+- Updated `Sidekiq::Process` API to provide capsule data. The `queues` and `weights` 
+  data will be removed from Redis in Sidekiq 8.1, as this data can now be found in the
+  `capsules` element. [#6295]
+- Restore bulk action buttons on Scheduled, Retry and Dead tabs [#6833, deve1212]
+- Support logging additional job attributes [#6846, bschrag620]
+- Fix display of long job args [#6836]
+- Create development lifecycle (`docs/sdlc.md`) and security (`docs/SECURITY.md`) policy
+  documentation for Sidekiq's current workflows
+
+8.0.8
+----------
+
+- Allow an optional global iteration max runtime. After executing for this length of time,
+  Sidekiq will re-queue the job to continue execution at a later time [#6819, fatkodima]
+```ruby
+Sidekiq.configure_server do |cfg|
+  cfg[:max_iteration_runtime] = 600 # ten minutes
+end
+```
+- Add `discarded_at` attribute when discarding a job so death handlers can distinguish between
+  a job which was killed and one that was discarded. [#6820, gstokkink]
+- `perform_bulk` now accepts an `:at` array of times to schedule each job at the corresponding time.
+  `perform_bulk(args: [[1], [2]], at: [Time.now, Time.now + 1])` [#6790, fatkodima]
+- `perform_bulk` now accepts a `:spread_interval` value to schedule jobs over
+  the next N seconds. `perform_bulk(..., spread_interval: 60)` [#6792, fatkodima]
+- Fix unintended display of flash messages in the Web UI due to session key collision
+- Add support for lazy load hooks [#6825]
+
+8.0.7
+----------
+
+- The `:discard` option for `sidekiq_retries_exhausted` and `sidekiq_retry_in`
+  now calls death handlers, otherwise it could break other Sidekiq
+  functionality. [#6741]
+- Provide a Plain log formatter which does not colorize output [#6778]
+- Job iteration now exposes `current_object` for easy access within the `around_iteration` callback [#6774]
+- Fix JS race condition which could skip confirmation dialogs when Live Polling [#6768]
+- Fix edge case which could lose CurrentAttributes [#6767]
+- Update UK locale [#6776]
+
+8.0.6
+----------
+
+- Adjust transactional client to use ActiveRecord 7.2's support for
+  `after_all_transactions_commit` when available. [#6765, rewritten]
+- Fix Rails 7.0 and 7.1 compatibility [#6746, mlarraz]
+- Flush metrics at `:exit` [#6764]
+
+8.0.5
+----------
+
+- Add `stopping?` method to AJ adapter for compatibility with the new AJ::Continuations feature [#6732]
+- Further improvements to Rails boot compatibility [#6710]
+- Add ability to disable CSRF middleware. SameSite cookies prevent
+  CSRF in a cleaner manner and are default in most browsers now.
+  CSRF code will be removed in Sidekiq 9.0. [#6739]
+
+8.0.4
+----------
+
+- Adjust Rails integration for various edge cases [6713]
+- Flush job iteration state when an error is raised [#6704]
+- Update Accept-Language parsing in Web UI [#6721]
+- Remove fixed-width in Web UI [#6686]
+- Adjust CSRF middleware ordering [#6688]
+- Support proxies when POSTing profiles to profiler.firefox.com [#6687]
+- Dont swallow NoMethodErrors in CurrentAttributes [#6685]
+
+8.0.3
+----------
+
+- Configure Vernier output directory [#6674]
+- Rework Rails integration [#6669]
+- Implement flash messages for the Web UI [#6675]
+
+8.0.2
+----------
+
+- Add `on(:exit)` event to run code right before the Sidekiq process exits [#6637]
+- Metrics page crashes with Rack 3.1+ [#6646]
+
+8.0.1
+----------
+
+- Relax Redis requirement to 7.0 for compatibility with AWS and Ubuntu 24.04 LTS. [#6630]
+
+8.0.0
+----------
+
+- **WARNING** The underlying class name for Active Jobs has changed from `ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper` to `Sidekiq::ActiveJob::Wrapper`.
+  The old name will still work in 8.x.
+- **WARNING** The `created_at`, `enqueued_at`, `failed_at` and `retried_at` attributes are now stored as epoch milliseconds, rather than epoch floats.
+  This is meant to avoid precision issues with JSON and JavaScript's 53-bit Floats.
+  Example: `"created_at" => 1234567890.123456` -> `"created_at" => 1234567890123`.
+- **NEW FEATURE** Job Profiling is now supported with [Vernier](https://vernier.prof)
+  which makes it really easy to performance tune your slow jobs.
+  The Web UI contains a new **Profiles** tab to view any collected profile data.
+  Please read the new [Profiling](https://github.com/sidekiq/sidekiq/wiki/Profiling) wiki page for details.
+- **NEW FEATURE** Job Metrics now store up to 72 hours of data and the Web UI allows display of 24/48/72 hours. [#6614]
+- CurrentAttribute support now uses `ActiveJob::Arguments` to serialize the context object, supporting Symbols and GlobalID.
+  The change should be backwards compatible. [#6510]
+- Freshen up `Sidekiq::Web` to simplify the code and improve security [#6532]
+  The CSS has been rewritten from scratch to remove the Bootstrap framework.
+- Add `on_cancel` callback for iterable jobs [#6607]
+- Add `cursor` reader to get the current cursor inside iterable jobs [#6606]
+- Default error logging has been modified to use Ruby's `Exception#detailed_message` and `#full_message` APIs.
+- CI now runs against Redis, Dragonfly and Valkey.
+- Job tags now allow custom CSS display [#6595]
+- The Web UI's language picker now shows options in the native language
+- Remove global variable usage within the codebase
+- Colorize and adjust logging for easier reading
+- Adjust Sidekiq's default thread priority to -1 for a 50ms timeslice.
+  This can help avoid TimeoutErrors when Sidekiq is overloaded. [#6543]
+- Use `Logger#with_level`, remove Sidekiq's custom impl
+- Remove `base64` gem dependency
+- Support: (Dragonfly 1.27+, Valkey 7.2+, Redis 7.2+), Ruby 3.2+, Rails 7.0+
+
+7.3.10
+----------
+
+- Deprecate Redis :password as a String to avoid log disclosure. [#6625]
+  Use a Proc instead: `config.redis = { password: ->(username) { "password" } }`
+
+7.3.9
+----------
+
+- Only require activejob if necessary [#6584]
+  You might get `uninitialized constant Sidekiq::ActiveJob` if you
+  `require 'sidekiq'` before `require 'rails'`.
+- Fix iterable job cancellation [#6589]
+- Web UI accessibility improvements [#6604]
+
+7.3.8
+----------
+
+- Fix dead tag links [#6554]
+- Massive Web UI performance improvement, some pages up to 15x faster [#6555]
+
+7.3.7
+----------
+
+- Backport `Sidekiq::Web.configure` for compatibility with 8.0 [#6532]
+- Backport `url_params(key)` and `route_params(key)` for compatibility with 8.0 [#6532]
+- Various fixes for UI filtering [#6508]
+- Tune `inspect` for internal S::Components to keep size managable [#6553]
+
+7.3.6
+----------
+
+- Forward compatibility fixes for Ruby 3.4
+- Filtering in the Web UI now works via GET so you can bookmark a filtered view. [#6497]
+
+7.3.5
+----------
+
+- Reimplement `retry_all` and `kill_all` API methods to use ZPOPMIN,
+  approximately 30-60% faster. [#6481]
+- Add preload testing binary at `examples/testing/sidekiq_boot` to verify your Rails app boots correctly with Sidekiq Enterprise's app preloading.
+- Fix circular require with ActiveJob adapter [#6477]
+- Fix potential race condition leading to incorrect serialized values for CurrentAttributes [#6475]
+- Restore missing elapsed time when default job logging is disabled
+
+7.3.4
+----------
+
+- Fix FrozenError when starting Sidekiq [#6470]
+
+7.3.3
+----------
+
+- Freeze global configuration once boot is complete, to avoid configuration race conditions [#6466, #6465]
+- Sidekiq now warns if a job iteration takes longer than the `-t` timeout setting (defaults to 25 seconds)
+- Iteration callbacks now have easy access to job arguments via the `arguments` method:
+```ruby
+def on_stop
+  p arguments # => `[123, "string", {"key" => "value"}]`
+  id, str, hash = arguments
+end
+```
+- Iterable jobs can be cancelled via `Sidekiq::Client#cancel!`:
+```ruby
+c = Sidekiq::Client.new
+jid = c.push("class" => SomeJob, "args" => [123])
+c.cancel!(jid) # => true
+```
+- Take over support for ActiveJob's `:sidekiq` adapter [#6430, fatkodima]
+- Ensure CurrentAttributes are in scope when creating batch callbacks [#6455]
+- Add `Sidekiq.gem_version` API.
+- Update Ukranian translations
+
+7.3.2
+----------
+
+- Adjust ActiveRecord batch iteration to restart an interrupted batch from the beginning.
+  Each batch should be processed as a single transaction in order to be idempotent. [#6405]
+- Fix typo in Sidekiq::DeadSet#kill [#6397]
+- Fix CSS issue with bottom bar in Web UI [#6414]
+
+7.3.1
+----------
+
+- Don't count job interruptions as failures in metrics [#6386]
+- Add frozen string literal to a number of .rb files.
+- Fix frozen string error with style_tag and script_tag [#6371]
+- Fix an error on Ruby 2.7 because of usage of `Hash#except` [#6376]
+
+7.3.0
+----------
+
+- **NEW FEATURE** Add `Sidekiq::IterableJob`, iteration support for long-running jobs. [#6286, fatkodima]
+  Iterable jobs are interruptible and can restart quickly if
+  running during a deploy. You must ensure that `each_iteration`
+  doesn't take more than Sidekiq's `-t` timeout (default: 25 seconds). Iterable jobs must not implement `perform`.
+```ruby
+class ProcessArrayJob
+  include Sidekiq::IterableJob
+  def build_enumerator(*args, **kwargs)
+    array_enumerator(args, **kwargs)
+  end
+  def each_iteration(arg)
+    puts arg
+  end
+end
+ProcessArrayJob.perform_async(1, 2, 3)
+```
+See the [Iteration](//github.com/sidekiq/sidekiq/wiki/Iteration) wiki page and the RDoc in `Sidekiq::IterableJob`.
+This feature should be considered BETA until the next minor release.
+- **SECURITY** The Web UI no longer allows extensions to use `<script>`.
+  Adjust CSP to disallow inline scripts within the Web UI. Please see
+  `examples/webui-ext` for how to register Web UI extensions and use
+  dynamic CSS and JS. This will make Sidekiq immune to XSS attacks. [#6270]
+- Add config option, `:skip_default_job_logging` to disable Sidekiq's default
+  start/finish job logging. [#6200]
+- Allow `Sidekiq::Limiter.redis` to use Redis Cluster [#6288]
+- Retain CurrentAttributeѕ after inline execution [#6307]
+- Ignore non-existent CurrentAttributes attributes when restoring [#6341]
+- Raise default Redis {read,write,connect} timeouts from 1 to 3 seconds
+  to minimize ReadTimeoutErrors [#6162]
+- Add `logger` as a dependency since it will become bundled in Ruby 3.5 [#6320]
+- Ignore unsupported locales in the Web UI [#6313]
+
+7.2.4
+----------
+
+- Fix XSS in metrics filtering introduced in 7.2.0, CVE-2024-32887
+  Thanks to @UmerAdeemCheema for the security report.
+
+7.2.3
+----------
+
+- [Support Dragonfly.io](https://www.mikeperham.com/2024/02/01/supporting-dragonfly/) as an alternative Redis implementation
+- Fix error unpacking some compressed error backtraces [#6241]
+- Fix potential heartbeat data leak [#6227]
+- Add ability to find a currently running work by jid [#6212, fatkodima]
+
+7.2.2
+----------
+
+- Add `Process.warmup` call in Ruby 3.3+
+- Batch jobs now skip transactional push [#6160]
+
+7.2.1
+----------
+
+- Add `Sidekiq::Work` type which replaces the raw Hash as the third parameter in
+  `Sidekiq::WorkSet#each { |pid, tid, hash| ... }` [#6145]
+- **DEPRECATED**: direct access to the attributes within the `hash` block parameter above.
+  The `Sidekiq::Work` instance contains accessor methods to get at the same data, e.g.
+```ruby
+work["queue"] # Old
+work.queue # New
+```
+- Fix Ruby 3.3 warnings around `base64` gem [#6151, earlopain]
+
+7.2.0
+----------
+
+- `sidekiq_retries_exhausted` can return `:discard` to avoid the deadset
+  and all death handlers [#6091]
+- Metrics filtering by job class in Web UI [#5974]
+- Better readability and formatting for numbers within the Web UI [#6080]
+- Add explicit error if user code tries to nest test modes [#6078]
+```ruby
+Sidekiq::Testing.inline! # global setting
+Sidekiq::Testing.fake! do # override within block
+  # ok
+  Sidekiq::Testing.inline! do # can't override the override
+    # not ok, nested
+  end
+end
+```
+- **SECURITY** Forbid inline JavaScript execution in Web UI [#6074]
+- Adjust redis-client adapter to avoid `method_missing` [#6083]
+  This can result in app code breaking if your app's Redis API usage was
+  depending on Sidekiq's adapter to correct invalid redis-client API usage.
+  One example:
+```ruby
+# bad, not redis-client native
+# Unsupported command argument type: TrueClass (TypeError)
+Sidekiq.redis { |c| c.set("key", "value", nx: true, ex: 15) }
+# good
+Sidekiq.redis { |c| c.set("key", "value", "nx", "ex", 15) }
+```
+
+7.1.6
+----------
+
+- The block forms of testing modes (inline, fake) are now thread-safe so you can have
+  a multithreaded test suite which uses different modes for different tests. [#6069]
+- Fix breakage with non-Proc error handlers [#6065]
+
+7.1.5
+----------
+
+- **FEATURE**: Job filtering within the Web UI. This feature has been open
+  sourced from Sidekiq Pro. [#6052]
+- **API CHANGE** Error handlers now take three arguments `->(ex, context, config)`.
+  The previous calling convention will work until Sidekiq 8.0 but will print
+  out a deprecation warning. [#6051]
+- Fix issue with the `batch_size` and `at` options in `S::Client.push_bulk` [#6040]
+- Fix inline testing firing batch callbacks early [#6057]
+- Use new log broadcast API in Rails 7.1 [#6054]
+- Crash if user tries to use RESP2 `protocol: 2` [#6061]
+
 7.1.4
 ----------
 
@@ -130,6 +458,11 @@ end
 - Job Execution metrics!!!
 - See `docs/7.0-Upgrade.md` for release notes
 
+6.5.{10,11,12}
+----------
+
+- Fixes for Rails 7.1 [#6067, #6070]
+
 6.5.9
 ----------
 

data/README.md

@@ -4,21 +4,23 @@ Sidekiq
 [![Gem Version](https://badge.fury.io/rb/sidekiq.svg)](https://rubygems.org/gems/sidekiq)
 ![Build](https://github.com/sidekiq/sidekiq/workflows/CI/badge.svg)
 
-Simple, efficient background processing for Ruby.
+Simple, efficient background jobs for Ruby.
 
 Sidekiq uses threads to handle many jobs at the same time in the
-same process.  It does not require Rails but will integrate tightly with
-Rails to make background processing dead simple.
+same process. Sidekiq can be used by any Ruby application.
 
 
 Requirements
 -----------------
 
-- Redis: 6.2+
-- Ruby: MRI 2.7+ or JRuby 9.3+.
+- Redis: Redis 7.0+, Valkey 7.2+ or Dragonfly 1.27+
+- Ruby: MRI 3.2+ or JRuby 9.4+.
 
-Sidekiq 7.0 supports Rails 6.0+ but does not require it.
+Sidekiq 8.0 supports Rails and Active Job 7.0+.
 
+Sidekiq supports [Valkey](https://valkey.io) and [Dragonfly](https://www.dragonflydb.io) as Redis alternatives.
+Redis 7.2.4 is considered to be the canonical implementation.
+Incompatibilities with that version are considered bugs.
 
 Installation
 -----------------
@@ -42,6 +44,7 @@ The benchmark in `bin/sidekiqload` creates 500,000 no-op jobs and drains them as
 This requires a lot of Redis network I/O and JSON parsing.
 This benchmark is IO-bound so we increase the concurrency to 25.
 If your application is sending lots of emails or performing other network-intensive work, you could see a similar benefit but be careful not to saturate the CPU.
+Real world applications will rarely if ever need to use concurrency greater than 10.
 
 Version | Time to process 500k jobs | Throughput (jobs/sec) | Ruby | Concurrency | Job Type
 -----------------|------|---------|---------|------------------------|---
@@ -62,7 +65,7 @@ Want to Upgrade?
 Use `bundle up sidekiq` to upgrade Sidekiq and all its dependencies.
 Upgrade notes between each major version can be found in the `docs/` directory.
 
-I also sell Sidekiq Pro and Sidekiq Enterprise, extensions to Sidekiq which provide more
+I also sell [Sidekiq Pro](https://billing.contribsys.com/spro/) and [Sidekiq Enterprise](https://billing.contribsys.com/sent/new.cgi), extensions to Sidekiq which provide more
 features, a commercial-friendly license and allow you to support high
 quality open source development all at the same time.  Please see the
 [Sidekiq](https://sidekiq.org/) homepage for more detail.
@@ -71,7 +74,7 @@ quality open source development all at the same time.  Please see the
 Problems?
 -----------------
 
-**Please do not directly email any Sidekiq committers with questions or problems.**
+**Do not directly email any Sidekiq committers with questions or problems.**
 A community is best served when discussions are held in public.
 
 If you have a problem, please review the [FAQ](https://github.com/sidekiq/sidekiq/wiki/FAQ) and [Troubleshooting](https://github.com/sidekiq/sidekiq/wiki/Problems-and-Troubleshooting) wiki pages.
@@ -83,24 +86,24 @@ You can purchase at https://sidekiq.org; email support@contribsys.com for help.
 Useful resources:
 
 * Product documentation is in the [wiki](https://github.com/sidekiq/sidekiq/wiki).
-* Occasional announcements are made to the [@sidekiq](https://twitter.com/sidekiq) Twitter account.
+* Occasional announcements are made to the [@sidekiq](https://ruby.social/@sidekiq) Mastodon account.
 * The [Sidekiq tag](https://stackoverflow.com/questions/tagged/sidekiq) on Stack Overflow has lots of useful Q & A.
 
-Every Friday morning is Sidekiq office hour: I video chat and answer questions.
+Every Thursday morning is Sidekiq Office Hour: I video chat and answer questions.
 See the [Sidekiq support page](https://sidekiq.org/support.html) for details.
 
 Contributing
 -----------------
 
-Please see [the contributing guidelines](https://github.com/sidekiq/sidekiq/blob/main/.github/contributing.md).
+See [the contributing guidelines](https://github.com/sidekiq/sidekiq/blob/main/.github/contributing.md).
 
 License
 -----------------
 
-Please see [LICENSE.txt](https://github.com/sidekiq/sidekiq/blob/main/LICENSE.txt) for licensing details.
+See [LICENSE.txt](https://github.com/sidekiq/sidekiq/blob/main/LICENSE.txt) for licensing details.
 The license for Sidekiq Pro and Sidekiq Enterprise can be found in [COMM-LICENSE.txt](https://github.com/sidekiq/sidekiq/blob/main/COMM-LICENSE.txt).
 
 Author
 -----------------
 
-Mike Perham, [@getajobmike](https://twitter.com/getajobmike) / [@sidekiq](https://twitter.com/sidekiq), [https://www.mikeperham.com](https://www.mikeperham.com) / [https://www.contribsys.com](https://www.contribsys.com)
+Mike Perham, [mastodon](https://ruby.social/@getajobmike), [https://www.mikeperham.com](https://www.mikeperham.com) / [https://www.contribsys.com](https://www.contribsys.com)