sidekiq 6.4.2 → 6.5.8

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of sidekiq might be problematic. Click here for more details.

Files changed (61) hide show
  1. checksums.yaml +4 -4
  2. data/Changes.md +73 -0
  3. data/bin/sidekiqload +17 -5
  4. data/lib/sidekiq/api.rb +196 -45
  5. data/lib/sidekiq/cli.rb +45 -32
  6. data/lib/sidekiq/client.rb +5 -5
  7. data/lib/sidekiq/component.rb +65 -0
  8. data/lib/sidekiq/delay.rb +1 -1
  9. data/lib/sidekiq/fetch.rb +18 -16
  10. data/lib/sidekiq/job_retry.rb +60 -39
  11. data/lib/sidekiq/job_util.rb +7 -3
  12. data/lib/sidekiq/launcher.rb +24 -21
  13. data/lib/sidekiq/logger.rb +1 -1
  14. data/lib/sidekiq/manager.rb +23 -20
  15. data/lib/sidekiq/metrics/deploy.rb +47 -0
  16. data/lib/sidekiq/metrics/query.rb +153 -0
  17. data/lib/sidekiq/metrics/shared.rb +94 -0
  18. data/lib/sidekiq/metrics/tracking.rb +134 -0
  19. data/lib/sidekiq/middleware/chain.rb +82 -38
  20. data/lib/sidekiq/middleware/current_attributes.rb +18 -12
  21. data/lib/sidekiq/middleware/i18n.rb +2 -0
  22. data/lib/sidekiq/middleware/modules.rb +21 -0
  23. data/lib/sidekiq/paginator.rb +10 -2
  24. data/lib/sidekiq/processor.rb +21 -15
  25. data/lib/sidekiq/rails.rb +5 -5
  26. data/lib/sidekiq/redis_client_adapter.rb +154 -0
  27. data/lib/sidekiq/redis_connection.rb +78 -47
  28. data/lib/sidekiq/ring_buffer.rb +29 -0
  29. data/lib/sidekiq/scheduled.rb +53 -24
  30. data/lib/sidekiq/testing.rb +1 -1
  31. data/lib/sidekiq/transaction_aware_client.rb +45 -0
  32. data/lib/sidekiq/version.rb +1 -1
  33. data/lib/sidekiq/web/action.rb +3 -3
  34. data/lib/sidekiq/web/application.rb +21 -5
  35. data/lib/sidekiq/web/helpers.rb +15 -2
  36. data/lib/sidekiq/web.rb +5 -1
  37. data/lib/sidekiq/worker.rb +6 -2
  38. data/lib/sidekiq.rb +87 -18
  39. data/sidekiq.gemspec +2 -2
  40. data/web/assets/javascripts/application.js +1 -1
  41. data/web/assets/javascripts/chart.min.js +13 -0
  42. data/web/assets/javascripts/chartjs-plugin-annotation.min.js +7 -0
  43. data/web/assets/javascripts/dashboard.js +0 -17
  44. data/web/assets/javascripts/graph.js +16 -0
  45. data/web/assets/javascripts/metrics.js +262 -0
  46. data/web/assets/stylesheets/application.css +44 -1
  47. data/web/locales/el.yml +43 -19
  48. data/web/locales/en.yml +7 -0
  49. data/web/locales/ja.yml +7 -0
  50. data/web/locales/pt-br.yml +27 -9
  51. data/web/locales/zh-cn.yml +36 -11
  52. data/web/locales/zh-tw.yml +32 -7
  53. data/web/views/_nav.erb +1 -1
  54. data/web/views/busy.erb +7 -2
  55. data/web/views/dashboard.erb +1 -0
  56. data/web/views/metrics.erb +69 -0
  57. data/web/views/metrics_for_job.erb +87 -0
  58. data/web/views/queue.erb +5 -1
  59. metadata +33 -8
  60. data/lib/sidekiq/exception_handler.rb +0 -27
  61. data/lib/sidekiq/util.rb +0 -108
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 70d899b76bdd764a2ec7e5f23b6d056494b58897daaf49db672daad3e0f60237
4
- data.tar.gz: daae0430ecaf8eb7f172c70e628581c20a17e1a7cb1a58170b9082fc3dde1ea2
3
+ metadata.gz: fde4ce1db65a0c90d5fd32cff914e33fd238e52ce2f219d03bdc4d6accbdd645
4
+ data.tar.gz: fd62df7cd374ea7bf2977b44da41c771da1deb8b84046e745fb042f66060eba0
5
5
  SHA512:
6
- metadata.gz: 930a0feb7ff47473eb995f8ba695468498afecb311c25296c93042ecfa00cd2bf48e19efd5ba775b4895c065705a344043e0c0e5d7b4115c2dceb1ec7cf341f9
7
- data.tar.gz: 14ab645609e7c9f3fa954432097cd46d6e9e74e5bddc9ed1541697d13f3399add407ffb5af23c89c82651793c97e2c54b8fd4f845ca20287ce6bf033fd9caed2
6
+ metadata.gz: 1b6b3089616dbc25ce2e182ceca53c00b31b6a77adeda402b67436f1021dfc301e9f7ac2e1337e405ffdf0f6aab875fe6b228fdbceaa6e50685781cb5899e80e
7
+ data.tar.gz: 37f2a103a2247c3541036044fd3ddb6e589eff483427b2f7c6ed02c6c349ec7a850c2adfdf868a22f018cf49bca3f94b65e3ecc8c46db9262c84b71c65593ddc
data/Changes.md CHANGED
@@ -2,6 +2,79 @@
2
2
 
3
3
  [Sidekiq Changes](https://github.com/mperham/sidekiq/blob/main/Changes.md) | [Sidekiq Pro Changes](https://github.com/mperham/sidekiq/blob/main/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/mperham/sidekiq/blob/main/Ent-Changes.md)
4
4
 
5
+ 6.5.8
6
+ ----------
7
+
8
+ - Fail if using a bad version of scout_apm [#5616]
9
+ - Add pagination to Busy page [#5556]
10
+ - Speed up WorkSet#each [#5559]
11
+ - Adjust CurrentAttributes to work with the String class name so we aren't referencing
12
+ the Class within a Rails initializer [#5536]
13
+
14
+ 6.5.7
15
+ ----------
16
+
17
+ - Updates for JA and ZH locales
18
+ - Further optimizations for scheduled polling [#5513]
19
+
20
+ 6.5.6
21
+ ----------
22
+
23
+ - Fix deprecation warnings with redis-rb 4.8.0 [#5484]
24
+ - Lock redis-rb to < 5.0 as we are moving to redis-client in Sidekiq 7.0
25
+
26
+ 6.5.5
27
+ ----------
28
+
29
+ - Fix require issue with job_retry.rb [#5462]
30
+ - Improve Sidekiq::Web compatibility with Rack 3.x
31
+
32
+ 6.5.4
33
+ ----------
34
+
35
+ - Fix invalid code on Ruby 2.5 [#5460]
36
+ - Fix further metrics dependency issues [#5457]
37
+
38
+ 6.5.3
39
+ ----------
40
+
41
+ - Don't require metrics code without explicit opt-in [#5456]
42
+
43
+ 6.5.2
44
+ ----------
45
+
46
+ - [Job Metrics are under active development, help wanted!](https://github.com/mperham/sidekiq/wiki/Metrics#contributing) **BETA**
47
+ - Add `Context` column on queue page which shows any CurrentAttributes [#5450]
48
+ - `sidekiq_retry_in` may now return `:discard` or `:kill` to dynamically stop job retries [#5406]
49
+ - Smarter sorting of processes in /busy Web UI [#5398]
50
+ - Fix broken hamburger menu in mobile UI [#5428]
51
+ - Require redis-rb 4.5.0. Note that Sidekiq will break if you use the
52
+ [`Redis.exists_returns_integer = false`](https://github.com/redis/redis-rb/blob/master/CHANGELOG.md#450) flag. [#5394]
53
+
54
+ 6.5.1
55
+ ----------
56
+
57
+ - Fix `push_bulk` breakage [#5387]
58
+
59
+ 6.5.0
60
+ ---------
61
+
62
+ - Substantial refactoring of Sidekiq server internals, part of a larger effort
63
+ to reduce Sidekiq's internal usage of global methods and data, see [docs/global_to_local.md](docs/global_to_local.md) and [docs/middleware.md](docs/middleware.md).
64
+ - **Add beta support for the `redis-client` gem**. This will become the default Redis driver in Sidekiq 7.0. [#5298]
65
+ Read more: https://github.com/mperham/sidekiq/wiki/Using-redis-client
66
+ - **Add beta support for DB transaction-aware client** [#5291]
67
+ Add this line to your initializer and any jobs created during a transaction
68
+ will only be pushed to Redis **after the transaction commits**. You will need to add the
69
+ `after_commit_everywhere` gem to your Gemfile.
70
+ ```ruby
71
+ Sidekiq.transactional_push!
72
+ ```
73
+ This feature does not have a lot of production usage yet; please try it out and let us
74
+ know if you have any issues. It will be fully supported in Sidekiq 7.0 or removed if it
75
+ proves problematic.
76
+ - Fix regression with middleware arguments [#5312]
77
+
5
78
  6.4.2
6
79
  ---------
7
80
 
data/bin/sidekiqload CHANGED
@@ -11,6 +11,10 @@ Bundler.require(:default, :load_test)
11
11
  require_relative "../lib/sidekiq/cli"
12
12
  require_relative "../lib/sidekiq/launcher"
13
13
 
14
+ if ENV["SIDEKIQ_REDIS_CLIENT"]
15
+ Sidekiq::RedisConnection.adapter = :redis_client
16
+ end
17
+
14
18
  Sidekiq.configure_server do |config|
15
19
  config.options[:concurrency] = 10
16
20
  config.redis = {db: 13, port: 6380}
@@ -85,7 +89,7 @@ def Process.rss
85
89
  `ps -o rss= -p #{Process.pid}`.chomp.to_i
86
90
  end
87
91
 
88
- iter = 50
92
+ iter = 10
89
93
  count = 10_000
90
94
 
91
95
  iter.times do
@@ -118,16 +122,24 @@ Monitoring = Thread.new do
118
122
  end
119
123
  end
120
124
 
125
+ def with_latency(latency, &block)
126
+ Sidekiq.logger.error "Simulating #{latency}ms of latency between Sidekiq and redis"
127
+ if latency > 0
128
+ Toxiproxy[:redis].downstream(:latency, latency: latency).apply(&block)
129
+ else
130
+ yield
131
+ end
132
+ end
133
+
121
134
  begin
122
135
  # RubyProf::exclude_threads = [ Monitoring ]
123
136
  # RubyProf.start
124
137
  events = Sidekiq.options[:lifecycle_events][:startup]
125
138
  events.each(&:call)
126
139
  events.clear
127
-
128
- Sidekiq.logger.error "Simulating 1ms of latency between Sidekiq and redis"
129
- Toxiproxy[:redis].downstream(:latency, latency: 1).apply do
130
- launcher = Sidekiq::Launcher.new(Sidekiq.options)
140
+
141
+ with_latency(Integer(ENV.fetch("LATENCY", "1"))) do
142
+ launcher = Sidekiq::Launcher.new(Sidekiq)
131
143
  launcher.run
132
144
 
133
145
  while readable_io = IO.select([self_read])