sidekiq_cleaner 5.3.6

data/Ent-Changes.md

@@ -0,0 +1,238 @@
+# Sidekiq Enterprise Changelog
+
+[Sidekiq Changes](https://github.com/mperham/sidekiq/blob/master/Changes.md) | [Sidekiq Pro Changes](https://github.com/mperham/sidekiq/blob/master/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/mperham/sidekiq/blob/master/Ent-Changes.md)
+
+Please see [http://sidekiq.org/](http://sidekiq.org/) for more details and how to buy.
+
+1.8.1
+-------------
+
+- Fix excessive lock reclaims with concurrent limiter [#4105]
+- Add ES translations, see issues [#3949](https://github.com/mperham/sidekiq/issues/3949) and [#3951](https://github.com/mperham/sidekiq/issues/3951) to add your own language.
+
+1.8.0
+-------------
+
+- Require Sidekiq Pro 4.0 and Sidekiq 5.2.
+- Refactor historical metrics API to use revamped Statsd support in Sidekiq Pro
+- Add a gauge to historical metrics for `default` queue latency [#4079]
+
+1.7.2
+-------------
+
+- Add PT and JA translations
+- Fix elapsed time calculations to use monotonic clock [#4000, sj26]
+- Fix edge case where flapping leadership would cause old periodic
+  jobs to be fired once [#3974]
+- Add support for sidekiqswarm memory monitoring on FreeBSD [#3884]
+
+1.7.1
+-------------
+
+- Fix Lua error in concurrent rate limiter under heavy contention
+- Remove superfluous `freeze` calls on Strings [#3759]
+
+1.7.0
+-------------
+
+- **NEW FEATURE** [Rolling restarts](https://github.com/mperham/sidekiq/wiki/Ent-Rolling-Restarts) - great for long running jobs!
+- Adjust middleware so unique jobs that don't push aren't registered in a Batch [#3662]
+- Add new unlimited rate limiter, useful for testing [#3743]
+```ruby
+limiter = Sidekiq::Limiter.unlimited(...any args...)
+```
+
+1.6.1
+-------------
+
+- Fix crash in rate limiter middleware when used with custom exceptions [#3604]
+
+1.6.0
+-------------
+
+- Show process "leader" tag on Busy page, requires Sidekiq 5.0.2 [#2867]
+- Capture custom metrics with the `save_history` API. [#2815]
+- Implement new `unique_until: 'start'` policy option. [#3471]
+
+1.5.4
+-------------
+
+- Fix broken Cron page in Web UI [#3458]
+
+1.5.3
+-------------
+
+- Remove dependency on the algorithms gem [#3446]
+- Allow user to specify max memory in megabytes with SIDEKIQ\_MAXMEM\_MB [#3451]
+- Implement logic to detect app startup failure, sidekiqswarm will exit
+  rather than try to restart the app forever [#3450]
+- Another fix for doubly-encrypted arguments [#3368]
+
+1.5.2
+-------------
+
+- Fix encrypted arguments double-encrypted by retry or rate limiting [#3368]
+- Fix leak in concurrent rate limiter, run this in Rails console to clean up existing data [#3323]
+```ruby
+expiry = 1.month.to_i; Sidekiq::Limiter.redis { |c| c.scan_each(match: "lmtr-cfree-*") { |key| c.expire(key, expiry) } }
+```
+
+1.5.1
+-------------
+
+- Fix issue with census startup when not using Bundler configuration for
+  source credentials.
+
+1.5.0
+-------------
+
+- Add new web authorization API [#3251]
+- Update all sidekiqswarm env vars to use SIDEKIQ\_ prefix [#3218]
+- Add census reporting, the leader will ping contribsys nightly with aggregate usage metrics
+
+1.4.0
+-------------
+
+- No functional changes, require latest Sidekiq and Sidekiq Pro versions
+
+1.3.2
+-------------
+
+- Upgrade encryption to use OpenSSL's more secure GCM mode. [#3060]
+
+1.3.1
+-------------
+
+- Fix multi-process memory monitoring on CentOS 6.x [#3063]
+- Polish the new encryption feature a bit.
+
+1.3.0
+-------------
+
+- **BETA** [New encryption feature](https://github.com/mperham/sidekiq/wiki/Ent-Encryption)
+  which automatically encrypts the last argument of a Worker, aka the secret bag.
+
+1.2.4
+-------------
+
+- Fix issue causing some minutely jobs to execute every other minute.
+- Log a warning if slow periodic processing causes us to miss a clock tick.
+
+1.2.3
+-------------
+
+- Periodic jobs could stop executing until process restart if Redis goes down [#3047]
+
+1.2.2
+-------------
+
+- Add API to check if a unique lock is present. See [#2932] for details.
+- Tune concurrent limiters to minimize thread thrashing under heavy contention. [#2944]
+- Add option for tuning which Bundler groups get preloaded with `sidekiqswarm` [#3025]
+```
+SIDEKIQ_PRELOAD=default,production bin/sidekiqswarm ...
+# Use an empty value for maximum application compatibility
+SIDEKIQ_PRELOAD= bin/sidekiqswarm ...
+```
+
+1.2.1
+-------------
+
+- Multi-Process mode can now monitor the RSS memory of children and
+  restart any that grow too large.  To limit children to 1GB each:
+```
+MAXMEM_KB=1048576 COUNT=2 bundle exec sidekiqswarm ...
+```
+
+1.2.0
+-------------
+
+- **NEW FEATURE** Multi-process mode!  Sidekiq Enterprise can now fork multiple worker
+  processes, enabling significant memory savings.  See the [wiki
+documentation](https://github.com/mperham/sidekiq/wiki/Ent-Multi-Process) for details.
+
+
+0.7.10
+-------------
+
+- More precise gemspec dependency versioning
+
+1.1.0
+-------------
+
+- **NEW FEATURE** Historical queue metrics, [documented in the wiki](https://github.com/mperham/sidekiq/wiki/Ent-Historical-Metrics) [#2719]
+
+0.7.9, 1.0.2
+-------------
+
+- Window limiters can now accept arbitrary window sizes [#2686]
+- Fix race condition in window limiters leading to non-stop OverLimit [#2704]
+- Fix invalid overage counts when nesting concurrent limiters
+
+1.0.1
+----------
+
+- Fix crash in periodic subsystem when a follower shuts down, thanks
+  to @justinko for reporting.
+
+1.0.0
+----------
+
+- Enterprise 1.x targets Sidekiq 4.x.
+- Rewrite several features to remove Celluloid dependency.  No
+  functional changes.
+
+0.7.8
+----------
+
+- Fix `unique_for: false` [#2658]
+
+
+0.7.7
+----------
+
+- Enterprise 0.x targets Sidekiq 3.x.
+- Fix racy shutdown event which could lead to disappearing periodic
+  jobs, requires Sidekiq >= 3.5.3.
+- Add new :leader event which is fired when a process gains leadership.
+
+0.7.6
+----------
+
+- Redesign how overrated jobs are rescheduled to avoid creating new
+  jobs. [#2619]
+
+0.7.5
+----------
+
+- Fix dynamic creation of concurrent limiters [#2617]
+
+0.7.4
+----------
+- Add additional check to prevent duplicate periodic job creation
+- Allow user-specified TTLs for rate limiters [#2607]
+- Paginate rate limiter index page [#2606]
+
+0.7.3
+----------
+
+- Rework `Sidekiq::Limiter` redis handling to match global redis handling.
+- Allow user to customize rate limit backoff logic and handle custom
+  rate limit errors.
+- Fix scalability issue with Limiter index page.
+
+0.7.2
+----------
+
+- Fix typo which prevented limiters with '0' in their names.
+
+0.7.1
+----------
+
+- Fix issue where unique scheduled jobs can't be enqueued upon schedule
+  due to the existing unique lock. [#2499]
+
+0.7.0
+----------
+
+Initial release.

data/Gemfile

@@ -0,0 +1,23 @@
+source 'https://rubygems.org'
+
+gemspec
+
+gem 'rake'
+gem 'redis-namespace'
+gem 'rails', '~> 5.2'
+gem 'sqlite3', '~> 1.3.6', platforms: :ruby
+gem 'activerecord-jdbcsqlite3-adapter', platforms: :jruby
+
+group :test do
+  gem 'minitest'
+  gem 'simplecov'
+end
+
+group :development, :test do
+  gem 'pry-byebug', platforms: :mri
+end
+
+group :load_test do
+  gem 'hiredis'
+  gem 'toxiproxy'
+end

data/LICENSE

@@ -0,0 +1,9 @@
+Copyright (c) Contributed Systems LLC
+
+Sidekiq is an Open Source project licensed under the terms of
+the LGPLv3 license.  Please see <http://www.gnu.org/licenses/lgpl-3.0.html>
+for license text.
+
+Sidekiq Pro has a commercial-friendly license allowing private forks
+and modifications of Sidekiq.  Please see https://sidekiq.org/products/pro.html for
+more detail.  You can find the commercial license terms in COMM-LICENSE.

data/Pro-2.0-Upgrade.md

@@ -0,0 +1,138 @@
+# Upgrading to Sidekiq Pro 2.0
+
+Sidekiq Pro 2.0 allows nested batches for more complex job workflows
+and provides a new reliable scheduler which uses Lua to guarantee
+atomicity and much higher performance.
+
+It also removes deprecated APIs, changes the batch data format and
+how features are activated.  Read carefully to ensure your upgrade goes
+smoothly.
+
+Sidekiq Pro 2.0 requires Sidekiq 3.3.2 or greater.  Redis 2.8 is
+recommended; Redis 2.4 or 2.6 will work but some functionality will not be
+available.
+
+**Note that you CANNOT go back to Pro 1.x once you've created batches
+with 2.x.  The new batches will not process correctly with 1.x.**
+
+**If you are on a version of Sidekiq Pro <1.5, you should upgrade to the
+latest 1.x version and run it for a week before upgrading to 2.0.**
+
+## Nested Batches
+
+Batches can now be nested within the `jobs` method.
+This feature enables Sidekiq Pro to handle workflow processing of any size
+and complexity!
+
+```ruby
+a = Sidekiq::Batch.new
+a.on(:success, SomeCallback)
+a.jobs do
+  SomeWork.perform_async
+
+  b = Sidekiq::Batch.new
+  b.on(:success, MyCallback)
+  b.jobs do
+    OtherWork.perform_async
+  end
+end
+```
+
+Parent batch callbacks are not processed until all child batch callbacks have
+run successfully.  In the example above, `MyCallback` will always fire
+before `SomeCallback` because `b` is considered a child of `a`.
+
+Of course you can dynamically add child batches while a batch job is executing.
+
+```ruby
+def perform(*args)
+  do_something(args)
+
+  if more_work?
+    # Sidekiq::Worker#batch returns the Batch this job is part of.
+    batch.jobs do
+      b = Sidekiq::Batch.new
+      b.on(:success, MyCallback)
+      b.jobs do
+        OtherWork.perform_async
+      end
+    end
+  end
+end
+```
+
+More context: [#1485]
+
+## Batch Data
+
+The batch data model was overhauled.  Batch data should take
+significantly less space in Redis now.  A simple benchmark shows 25%
+savings but real world savings should be even greater.
+
+* Batch 2.x BIDs are 14 character URL-safe Base64-encoded strings, e.g.
+  "vTF1-9QvLPnREQ".  Batch 1.x BIDs were 16 character hex-encoded
+  strings, e.g. "4a3fc67d30370edf".
+* In 1.x, batch data was not removed until it naturally expired in Redis.
+  In 2.x, all data for a batch is removed from Redis once the batch has
+  run any success callbacks.
+* Because of the former point, batch expiry is no longer a concern.
+  Batch expiry is hardcoded to 30 days and is no longer user-tunable.
+* Failed batch jobs no longer automatically store any associated
+  backtrace in Redis.
+
+**There's no data migration required.  Sidekiq Pro 2.0 transparently handles
+both old and new format.**
+
+More context: [#2130]
+
+## Reliability
+
+2.0 brings a new reliable scheduler which uses Lua inside Redis so enqueuing
+scheduled jobs is atomic.  Benchmarks show it 50x faster when enqueuing
+lots of jobs.
+
+**Two caveats**:
+- Client-side middleware is not executed
+  for each job when enqueued with the reliable scheduler.  No Sidekiq or
+  Sidekiq Pro functionality is affected by this change but some 3rd party
+  plugins might be.
+- The Lua script used inside the reliable scheduler is not safe for use
+  with Redis Cluster or other multi-master Redis solutions.
+  It is safe to use with Redis Sentinel or a typical master/slave replication setup.
+
+**You no longer require anything to use the Reliability features.**
+
+* Activate reliable fetch and/or the new reliable scheduler:
+```ruby
+Sidekiq.configure_server do |config|
+  config.reliable_fetch!
+  config.reliable_scheduler!
+end
+```
+* Activate reliable push:
+```ruby
+Sidekiq::Client.reliable_push!
+```
+
+More context: [#2130]
+
+## Other Changes
+
+* You must require `sidekiq/pro/notifications` if you want to use the
+  existing notification schemes.  I don't recommend using them as the
+  newer-style `Sidekiq::Batch#on` method is simpler and more flexible.
+* Several classes have been renamed.  Generally these classes are ones
+  you should not need to require/use in your own code, e.g. the Batch
+  middleware.
+* You can add `attr_accessor :jid` to a Batch callback class and Sidekiq
+  Pro will set it to the jid of the callback job. [#2178]
+* There's now an official API to iterate all known Batches [#2191]
+```ruby
+Sidekiq::BatchSet.new.each {|status| p status.bid }
+```
+* The Web UI now shows the Sidekiq Pro version in the footer. [#1991]
+
+## Thanks
+
+Adam Prescott, Luke van der Hoeven and Jon Hyman all provided valuable
+feedback during the release process.  Thank you guys!

data/Pro-3.0-Upgrade.md

@@ -0,0 +1,44 @@
+# Welcome to Sidekiq Pro 3.0!
+
+Sidekiq Pro 3.0 is designed to work with Sidekiq 4.0.
+
+## What's New
+
+* **Redis 2.8.0 or greater is required.**  Redis 2.8 was released two years
+  ago and contains **many** useful features which Sidekiq couldn't
+  leverage until now.  **Redis 3.0.3 or greater is recommended** for large
+  scale use.
+
+* Sidekiq Pro no longer uses Celluloid.  If your application code uses Celluloid,
+  you will need to pull it in yourself.
+
+* Pausing and unpausing queues is now instantaneous, no more polling!
+
+* Reliable fetch has been re-implemented due to the fetch changes in
+  Sidekiq 4.0.
+
+* Support for platforms without persistent hostnames.  Since the reliable\_fetch
+  algorithm requires a persistent hostname, an alternative reliability
+algorithm is now available for platforms like Heroku and Docker:
+```ruby
+Sidekiq.configure_server do |config|
+  config.timed_fetch!
+end
+```
+  The wiki contains [much more detail about each reliability option](https://github.com/mperham/sidekiq/wiki/Pro-Reliability-Server).
+
+* The old 'sidekiq/notifications' features have been removed.
+
+## Upgrade
+
+First, make sure you are using Redis 2.8 or greater. Next:
+
+* Upgrade to the latest Sidekiq Pro 2.x.
+```ruby
+gem 'sidekiq-pro', '< 3'
+```
+* Fix any deprecation warnings you see.
+* Upgrade to 3.x.
+```ruby
+gem 'sidekiq-pro', '< 4'
+```

data/Pro-4.0-Upgrade.md

@@ -0,0 +1,35 @@
+# Welcome to Sidekiq Pro 4.0!
+
+Sidekiq Pro 4.0 is designed to work with Sidekiq 5.0.
+
+## What's New
+
+* Batches now "die" if any of their jobs die.  You can enumerate the set
+  of dead batches and their associated dead jobs.  The success callback
+  for a dead batch will never fire unless these jobs are fixed.
+```ruby
+Sidekiq::Batch::DeadSet.new.each do |status|
+  status.dead? # => true
+  status.dead_jobs # => [...]
+end
+```
+This API allows you to enumerate the batches which need help.
+If you fix the issue and the dead jobs succeed, the batch will succeed.
+* The older `reliable_fetch` and `timed_fetch` algorithms have been
+  removed.  Only super\_fetch is available in 4.0.
+* The statsd middleware has been tweaked to remove support for legacy,
+  pre-3.6.0 configuration and add relevant tags.
+* Requires Sidekiq 5.0.5+.
+
+## Upgrade
+
+* Upgrade to the latest Sidekiq Pro 3.x.
+```ruby
+gem 'sidekiq-pro', '< 4'
+```
+* Fix any deprecation warnings you see.
+* Upgrade to 4.x.
+```ruby
+gem 'sidekiq-pro', '< 5'
+```
+