sidekiq 6.4.2 → 6.5.12

Sign up to get free protection for your applications and to get access to all the features.
Files changed (62) hide show
  1. checksums.yaml +4 -4
  2. data/Changes.md +89 -0
  3. data/bin/sidekiqload +17 -5
  4. data/lib/sidekiq/api.rb +196 -45
  5. data/lib/sidekiq/cli.rb +46 -32
  6. data/lib/sidekiq/client.rb +6 -6
  7. data/lib/sidekiq/component.rb +65 -0
  8. data/lib/sidekiq/delay.rb +1 -1
  9. data/lib/sidekiq/fetch.rb +18 -16
  10. data/lib/sidekiq/job_retry.rb +60 -39
  11. data/lib/sidekiq/job_util.rb +7 -3
  12. data/lib/sidekiq/launcher.rb +24 -21
  13. data/lib/sidekiq/logger.rb +1 -1
  14. data/lib/sidekiq/manager.rb +23 -20
  15. data/lib/sidekiq/metrics/deploy.rb +47 -0
  16. data/lib/sidekiq/metrics/query.rb +153 -0
  17. data/lib/sidekiq/metrics/shared.rb +94 -0
  18. data/lib/sidekiq/metrics/tracking.rb +134 -0
  19. data/lib/sidekiq/middleware/chain.rb +82 -38
  20. data/lib/sidekiq/middleware/current_attributes.rb +18 -12
  21. data/lib/sidekiq/middleware/i18n.rb +2 -0
  22. data/lib/sidekiq/middleware/modules.rb +21 -0
  23. data/lib/sidekiq/monitor.rb +1 -1
  24. data/lib/sidekiq/paginator.rb +11 -3
  25. data/lib/sidekiq/processor.rb +21 -15
  26. data/lib/sidekiq/rails.rb +12 -13
  27. data/lib/sidekiq/redis_client_adapter.rb +154 -0
  28. data/lib/sidekiq/redis_connection.rb +78 -47
  29. data/lib/sidekiq/ring_buffer.rb +29 -0
  30. data/lib/sidekiq/scheduled.rb +53 -24
  31. data/lib/sidekiq/testing.rb +1 -1
  32. data/lib/sidekiq/transaction_aware_client.rb +45 -0
  33. data/lib/sidekiq/version.rb +1 -1
  34. data/lib/sidekiq/web/action.rb +3 -3
  35. data/lib/sidekiq/web/application.rb +21 -5
  36. data/lib/sidekiq/web/helpers.rb +18 -5
  37. data/lib/sidekiq/web.rb +5 -1
  38. data/lib/sidekiq/worker.rb +8 -4
  39. data/lib/sidekiq.rb +87 -18
  40. data/sidekiq.gemspec +2 -2
  41. data/web/assets/javascripts/application.js +2 -1
  42. data/web/assets/javascripts/chart.min.js +13 -0
  43. data/web/assets/javascripts/chartjs-plugin-annotation.min.js +7 -0
  44. data/web/assets/javascripts/dashboard.js +0 -17
  45. data/web/assets/javascripts/graph.js +16 -0
  46. data/web/assets/javascripts/metrics.js +262 -0
  47. data/web/assets/stylesheets/application.css +44 -1
  48. data/web/locales/el.yml +43 -19
  49. data/web/locales/en.yml +7 -0
  50. data/web/locales/ja.yml +7 -0
  51. data/web/locales/pt-br.yml +27 -9
  52. data/web/locales/zh-cn.yml +36 -11
  53. data/web/locales/zh-tw.yml +32 -7
  54. data/web/views/_nav.erb +1 -1
  55. data/web/views/busy.erb +7 -2
  56. data/web/views/dashboard.erb +1 -0
  57. data/web/views/metrics.erb +69 -0
  58. data/web/views/metrics_for_job.erb +87 -0
  59. data/web/views/queue.erb +5 -1
  60. metadata +34 -9
  61. data/lib/sidekiq/exception_handler.rb +0 -27
  62. data/lib/sidekiq/util.rb +0 -108
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 70d899b76bdd764a2ec7e5f23b6d056494b58897daaf49db672daad3e0f60237
4
- data.tar.gz: daae0430ecaf8eb7f172c70e628581c20a17e1a7cb1a58170b9082fc3dde1ea2
3
+ metadata.gz: 55c6f9a8c0f2810bcbe7744c95204893d73f534666f6091c74dcb5199e7a4a28
4
+ data.tar.gz: c42690ef0d1876c94eb51fb68319275d11f10169826180db921b34d6334a1a54
5
5
  SHA512:
6
- metadata.gz: 930a0feb7ff47473eb995f8ba695468498afecb311c25296c93042ecfa00cd2bf48e19efd5ba775b4895c065705a344043e0c0e5d7b4115c2dceb1ec7cf341f9
7
- data.tar.gz: 14ab645609e7c9f3fa954432097cd46d6e9e74e5bddc9ed1541697d13f3399add407ffb5af23c89c82651793c97e2c54b8fd4f845ca20287ce6bf033fd9caed2
6
+ metadata.gz: 461b559e18cbdf8fe6ba20ab9fa38d08fd3b7b8d14dce7a7f9369d678e92bd25c07734bc14b0167f83589c0f03501897195b3fdf9cfd5de5a60f039bf7436f98
7
+ data.tar.gz: ce0490b438a22a71c37e5a65193a7728e3297b437730d30a3653807ff1e89476db1f8c62d8de50c0c8cebbbce2ba6fb65f110855e071c62746fe8697a0f636e1
data/Changes.md CHANGED
@@ -2,6 +2,95 @@
2
2
 
3
3
  [Sidekiq Changes](https://github.com/mperham/sidekiq/blob/main/Changes.md) | [Sidekiq Pro Changes](https://github.com/mperham/sidekiq/blob/main/Pro-Changes.md) | [Sidekiq Enterprise Changes](https://github.com/mperham/sidekiq/blob/main/Ent-Changes.md)
4
4
 
5
+ 6.5.11
6
+ ----------
7
+
8
+ - Fix for Rails 7.1 [#6067]
9
+
10
+ 6.5.10
11
+ ----------
12
+
13
+ - Web UI DoS vector [#6045] CVE-2023-26141
14
+ - Fix broadcast logger with Rails 7.1 [#6054]
15
+
16
+ 6.5.9
17
+ ----------
18
+
19
+ - Ensure Sidekiq.options[:environment] == RAILS_ENV [#5932]
20
+
21
+ 6.5.8
22
+ ----------
23
+
24
+ - Fail if using a bad version of scout_apm [#5616]
25
+ - Add pagination to Busy page [#5556]
26
+ - Speed up WorkSet#each [#5559]
27
+ - Adjust CurrentAttributes to work with the String class name so we aren't referencing
28
+ the Class within a Rails initializer [#5536]
29
+
30
+ 6.5.7
31
+ ----------
32
+
33
+ - Updates for JA and ZH locales
34
+ - Further optimizations for scheduled polling [#5513]
35
+
36
+ 6.5.6
37
+ ----------
38
+
39
+ - Fix deprecation warnings with redis-rb 4.8.0 [#5484]
40
+ - Lock redis-rb to < 5.0 as we are moving to redis-client in Sidekiq 7.0
41
+
42
+ 6.5.5
43
+ ----------
44
+
45
+ - Fix require issue with job_retry.rb [#5462]
46
+ - Improve Sidekiq::Web compatibility with Rack 3.x
47
+
48
+ 6.5.4
49
+ ----------
50
+
51
+ - Fix invalid code on Ruby 2.5 [#5460]
52
+ - Fix further metrics dependency issues [#5457]
53
+
54
+ 6.5.3
55
+ ----------
56
+
57
+ - Don't require metrics code without explicit opt-in [#5456]
58
+
59
+ 6.5.2
60
+ ----------
61
+
62
+ - [Job Metrics are under active development, help wanted!](https://github.com/mperham/sidekiq/wiki/Metrics#contributing) **BETA**
63
+ - Add `Context` column on queue page which shows any CurrentAttributes [#5450]
64
+ - `sidekiq_retry_in` may now return `:discard` or `:kill` to dynamically stop job retries [#5406]
65
+ - Smarter sorting of processes in /busy Web UI [#5398]
66
+ - Fix broken hamburger menu in mobile UI [#5428]
67
+ - Require redis-rb 4.5.0. Note that Sidekiq will break if you use the
68
+ [`Redis.exists_returns_integer = false`](https://github.com/redis/redis-rb/blob/master/CHANGELOG.md#450) flag. [#5394]
69
+
70
+ 6.5.1
71
+ ----------
72
+
73
+ - Fix `push_bulk` breakage [#5387]
74
+
75
+ 6.5.0
76
+ ---------
77
+
78
+ - Substantial refactoring of Sidekiq server internals, part of a larger effort
79
+ to reduce Sidekiq's internal usage of global methods and data, see [docs/global_to_local.md](docs/global_to_local.md) and [docs/middleware.md](docs/middleware.md).
80
+ - **Add beta support for the `redis-client` gem**. This will become the default Redis driver in Sidekiq 7.0. [#5298]
81
+ Read more: https://github.com/mperham/sidekiq/wiki/Using-redis-client
82
+ - **Add beta support for DB transaction-aware client** [#5291]
83
+ Add this line to your initializer and any jobs created during a transaction
84
+ will only be pushed to Redis **after the transaction commits**. You will need to add the
85
+ `after_commit_everywhere` gem to your Gemfile.
86
+ ```ruby
87
+ Sidekiq.transactional_push!
88
+ ```
89
+ This feature does not have a lot of production usage yet; please try it out and let us
90
+ know if you have any issues. It will be fully supported in Sidekiq 7.0 or removed if it
91
+ proves problematic.
92
+ - Fix regression with middleware arguments [#5312]
93
+
5
94
  6.4.2
6
95
  ---------
7
96
 
data/bin/sidekiqload CHANGED
@@ -11,6 +11,10 @@ Bundler.require(:default, :load_test)
11
11
  require_relative "../lib/sidekiq/cli"
12
12
  require_relative "../lib/sidekiq/launcher"
13
13
 
14
+ if ENV["SIDEKIQ_REDIS_CLIENT"]
15
+ Sidekiq::RedisConnection.adapter = :redis_client
16
+ end
17
+
14
18
  Sidekiq.configure_server do |config|
15
19
  config.options[:concurrency] = 10
16
20
  config.redis = {db: 13, port: 6380}
@@ -85,7 +89,7 @@ def Process.rss
85
89
  `ps -o rss= -p #{Process.pid}`.chomp.to_i
86
90
  end
87
91
 
88
- iter = 50
92
+ iter = 10
89
93
  count = 10_000
90
94
 
91
95
  iter.times do
@@ -118,16 +122,24 @@ Monitoring = Thread.new do
118
122
  end
119
123
  end
120
124
 
125
+ def with_latency(latency, &block)
126
+ Sidekiq.logger.error "Simulating #{latency}ms of latency between Sidekiq and redis"
127
+ if latency > 0
128
+ Toxiproxy[:redis].downstream(:latency, latency: latency).apply(&block)
129
+ else
130
+ yield
131
+ end
132
+ end
133
+
121
134
  begin
122
135
  # RubyProf::exclude_threads = [ Monitoring ]
123
136
  # RubyProf.start
124
137
  events = Sidekiq.options[:lifecycle_events][:startup]
125
138
  events.each(&:call)
126
139
  events.clear
127
-
128
- Sidekiq.logger.error "Simulating 1ms of latency between Sidekiq and redis"
129
- Toxiproxy[:redis].downstream(:latency, latency: 1).apply do
130
- launcher = Sidekiq::Launcher.new(Sidekiq.options)
140
+
141
+ with_latency(Integer(ENV.fetch("LATENCY", "1"))) do
142
+ launcher = Sidekiq::Launcher.new(Sidekiq)
131
143
  launcher.run
132
144
 
133
145
  while readable_io = IO.select([self_read])