autotuner 0.1.0 → 1.0.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.rubocop.yml +3 -0
- data/Gemfile +1 -1
- data/LICENSE.txt +1 -1
- data/README.md +55 -15
- data/lib/autotuner/configuration.rb +35 -0
- data/lib/{gc_tuner → autotuner}/data_structure/data_points.rb +9 -3
- data/lib/autotuner/gc_context.rb +31 -0
- data/lib/{gc_tuner → autotuner}/heuristic/base.rb +11 -3
- data/lib/autotuner/heuristic/gc_compact.rb +60 -0
- data/lib/autotuner/heuristic/heap_size_warmup.rb +130 -0
- data/lib/autotuner/heuristic/malloc.rb +121 -0
- data/lib/autotuner/heuristic/oldmalloc.rb +89 -0
- data/lib/autotuner/heuristic/remembered_wb_unprotected_objects.rb +103 -0
- data/lib/autotuner/heuristics.rb +12 -0
- data/lib/{gc_tuner → autotuner}/rack_plugin.rb +2 -2
- data/lib/autotuner/report/base.rb +37 -0
- data/lib/autotuner/report/multiple_environment_variables.rb +35 -0
- data/lib/autotuner/report/single_environment_variable.rb +29 -0
- data/lib/autotuner/report/string.rb +13 -0
- data/lib/autotuner/request_collector.rb +99 -0
- data/lib/{gc_tuner/request_collector.rb → autotuner/request_context.rb} +8 -17
- data/lib/autotuner/system_context.rb +21 -0
- data/lib/autotuner/version.rb +5 -0
- data/lib/autotuner.rb +29 -0
- metadata +25 -15
- data/lib/gc_tuner/configuration.rb +0 -20
- data/lib/gc_tuner/gc_context.rb +0 -17
- data/lib/gc_tuner/heuristic/size_pool_warmup.rb +0 -131
- data/lib/gc_tuner/heuristics.rb +0 -27
- data/lib/gc_tuner/version.rb +0 -5
- data/lib/gc_tuner.rb +0 -18
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 2a0cbc0a71a4fe3322d56efe07228bc9e9f7abc0e63f5d6449d86bcc8d7e4475
|
4
|
+
data.tar.gz: a07a9184f3615a5e4da4a7d66ffae945e5da377e8cb089a0fcd1b40735c3d093
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: de54a820d1e5674b1a0b14d2519a2bd89e28f0d7d11690cf90e83a1ace67c829a2c9f019d4487952a4d2432146a857dce6905e82513273267ba7835beff2fcaa
|
7
|
+
data.tar.gz: fd56fb285f351006ac7091c24db412a96518d82771beebcbcbd16107d41e0a2a4af382b5ce2a3a719a21fb96aeea20bbd421e429eba568b3cd1d34a39ab58048
|
data/.rubocop.yml
CHANGED
data/Gemfile
CHANGED
data/LICENSE.txt
CHANGED
data/README.md
CHANGED
@@ -1,34 +1,74 @@
|
|
1
|
-
#
|
1
|
+
# Autotuner
|
2
2
|
|
3
|
-
|
4
|
-
|
5
|
-
Welcome to your new gem! In this directory, you'll find the files you need to be able to package up your Ruby library into a gem. Put your Ruby code in the file `lib/gc_tuner`. To experiment with that code, run `bin/console` for an interactive prompt.
|
3
|
+
Autotuner is a tool designed to help you tune the garbage collector of your Rails app. Autotuner integrates into Rack as a middleware and collects data from the garbage collector between requests. It will then intelligently provide suggestions to tune the garbage collector for faster bootup, warmup, and response times.
|
6
4
|
|
7
5
|
## Installation
|
8
6
|
|
9
|
-
|
7
|
+
To install the gem, add it to the application's Gemfile by executing:
|
8
|
+
|
9
|
+
```
|
10
|
+
$ bundle add autotuner
|
11
|
+
```
|
12
|
+
|
13
|
+
## Quick start
|
14
|
+
|
15
|
+
1. Open the `config.ru` file in your Rails app and add the following line immediately above `run(Rails.application)`:
|
16
|
+
```ruby
|
17
|
+
use(Autotuner::RackPlugin)
|
18
|
+
```
|
19
|
+
1. Create an initializer in `config/initializers/autotuner.rb`:
|
20
|
+
```ruby
|
21
|
+
# Enable autotuner. Alternatively, call Autotuner.sample_ratio= with a value
|
22
|
+
# between 0 and 1.0 to sample on a portion of instances.
|
23
|
+
Autotuner.enabled = true
|
24
|
+
|
25
|
+
# This callback is called whenever a suggestion is provided by this gem.
|
26
|
+
# You can output this report to your logging pipeline, stdout, a file,
|
27
|
+
# or somewhere else!
|
28
|
+
Autotuner.reporter = proc do |report|
|
29
|
+
Rails.logger.info(report.to_s)
|
30
|
+
end
|
31
|
+
|
32
|
+
# This (optional) callback is called to provide metrics that can give you
|
33
|
+
# insights about the performance of your app. It's recommended to send this
|
34
|
+
# data to your observability service (e.g. Datadog, Prometheus, New Relic, etc).
|
35
|
+
Autotuner.metrics_reporter = proc do |metrics|
|
36
|
+
# stats is a hash of metric name (string) to integer value.
|
37
|
+
metrics.each do |key, val|
|
38
|
+
StatsD.gauge(key, val)
|
39
|
+
end
|
40
|
+
end
|
41
|
+
```
|
42
|
+
|
43
|
+
## Experimenting with tuning suggestions
|
10
44
|
|
11
|
-
|
45
|
+
While autotuner aims to comprehensively analyze your traffic to give the suggestion, not all of the suggestions it provides will be perfect. There will be cases where the suggestions it provides may result in undesired outcomes. Therefore, it is NOT recommended to blindly apply suggestions from autotuner, but rather use a scientific approach to experiment with the suggestions. There are a few steps to this.
|
12
46
|
|
13
|
-
|
47
|
+
1. Before any suggestions from autotuner is applied, make sure you are collecting system metrics from your Rails app. Send this data to your observability service so you can measure average, 50th percentile, 99th percentile, and 99.9th percentile data.
|
14
48
|
|
15
|
-
|
49
|
+
You can use `Autotuner.metrics_reporter` to collect important metrics from your app, including: GC time, number of major and minor GC cycles, request time, and number of heap pages allocated.
|
50
|
+
1. Establish an experimental group of machines/containers in production. Since response times and the state of the garbage collector are highly variable, it's much easier and more reliable to compare two groups at the same time rather than across different time periods with different traffic patterns.
|
16
51
|
|
17
|
-
|
52
|
+
You can do this by assigning a random number to each machine/container at boot, and using that number to determine the group it belongs in. Depending on the traffic of your app, you may want to place between 5% (high traffic apps) to 50% (low traffic apps) in the experimental group.
|
53
|
+
1. Apply suggestions from Autotuner one at a time in the experimental group, and observe the impacts of the tuning. You may want to observe the impact over a few days to a week, including warmup performance after a new deploy and long periods of no deploys (such as a weekend).
|
18
54
|
|
19
|
-
|
55
|
+
If you observe that the suggestion provides positive improvements, then also apply the suggestion to the default group and experiment with the next tuning suggestion provided by Autotuner.
|
20
56
|
|
21
|
-
|
57
|
+
Some suggestions may provide a trade-off. For example, it may improve average response time at the expense of worse extremes (99th or 99.9th percentile). It is up to you to determine whether the trade-off is worth it.
|
22
58
|
|
23
|
-
|
59
|
+
Some suggestions may cause a decrease in performance. In that case, discard the suggestion and experiment with the next suggestion provided by Autotuner.
|
24
60
|
|
25
|
-
|
61
|
+
## Configuration
|
26
62
|
|
27
|
-
|
63
|
+
- `Autotuner.enabled=`: (required, unless `Autotuner.sample_ratio` is set) Sets whether autotuner is enabled or not. When autotuner is disabled, data is not collected and suggestions are not given. Defaults to `false`.
|
64
|
+
- `Autotuner.sample_ratio=`: (optional) Sets the portion of instances where autotuner is enabled. Pass a value between 0 (enabled on no intances) and 1.0 (enabled on all instances). Note that this does not sample reqeusts, but rather samples the portion of instances that have autotuner enabled (it will be enabled for all requests on those instances). Do not configure `Autotuner.enabled=` when you use this option.
|
65
|
+
- `Autotuner.reporter=`: (required) Callback called when a heuristic is ready to give a suggestion. The callback will be called with one argument which will be an instance of `Autotuner::Report::Base`. Call `#to_s` on this object to get a string containing instructions and recommendations. You must set this when autotuner is enabled.
|
66
|
+
- `Autotuner.debug_reporter=`: (optional) Callback to periodically emit debug messages of internal state of heuristics. The callback will be called with one argument which will be a hash with the heuristic name as the key and the debug message as the value. Regular users do not need to configure this as this is only useful for debugging purposes.
|
67
|
+
- `Autotuner.metrics_reporter=`: (optional) Callback to emit useful metrics about your service. The callback will be called with a hash containing the metric names (string) as the key and integer values.
|
28
68
|
|
29
69
|
## Contributing
|
30
70
|
|
31
|
-
Bug reports and pull requests are welcome on GitHub at https://github.com/
|
71
|
+
Bug reports and pull requests are welcome on GitHub at https://github.com/Shopify/autotuner.
|
32
72
|
|
33
73
|
## License
|
34
74
|
|
@@ -0,0 +1,35 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Autotuner
|
4
|
+
module Configuration
|
5
|
+
DATA_POINTS_COUNT = 1_000
|
6
|
+
|
7
|
+
attr_reader :sample_ratio
|
8
|
+
attr_accessor :reporter
|
9
|
+
attr_accessor :metrics_reporter
|
10
|
+
|
11
|
+
# Set this callback to report debug information periodically.
|
12
|
+
attr_accessor :debug_reporter
|
13
|
+
|
14
|
+
def enabled?
|
15
|
+
@enabled
|
16
|
+
end
|
17
|
+
|
18
|
+
def enabled=(enabled)
|
19
|
+
raise ArgumentError, "cannot configure `enabled` when `sample_ratio` is configured" if sample_ratio
|
20
|
+
|
21
|
+
@enabled = enabled
|
22
|
+
end
|
23
|
+
|
24
|
+
def sample_ratio=(ratio)
|
25
|
+
raise ArgumentError, "`ratio` must be between 0 and 1.0" unless (0..1.0).include?(ratio)
|
26
|
+
if enabled? || enabled? == false
|
27
|
+
raise ArgumentError, "cannot configure `sample_ratio` when `enabled` is configured"
|
28
|
+
end
|
29
|
+
|
30
|
+
@sample_ratio = ratio
|
31
|
+
|
32
|
+
@enabled = rand < ratio
|
33
|
+
end
|
34
|
+
end
|
35
|
+
end
|
@@ -1,6 +1,6 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
|
-
module
|
3
|
+
module Autotuner
|
4
4
|
module DataStructure
|
5
5
|
class DataPoints
|
6
6
|
STABLE_RATIO = 0.5
|
@@ -74,8 +74,14 @@ module GCTuner
|
|
74
74
|
(Math.sqrt((length * sum_x_2) - (sum_x**2)) * Math.sqrt((length * sum_y_2) - (sum_y**2)))
|
75
75
|
end
|
76
76
|
|
77
|
-
def
|
78
|
-
|
77
|
+
def debug_state
|
78
|
+
{
|
79
|
+
samples: @samples,
|
80
|
+
length: @length,
|
81
|
+
compression_ratio: @compression_ratio,
|
82
|
+
temp_sample: @temp_sample,
|
83
|
+
temp_sample_count: @temp_sample_count,
|
84
|
+
}
|
79
85
|
end
|
80
86
|
|
81
87
|
private
|
@@ -0,0 +1,31 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Autotuner
|
4
|
+
class GCContext
|
5
|
+
HAS_STAT_HEAP = GC.respond_to?(:stat_heap)
|
6
|
+
|
7
|
+
attr_reader :stat, :latest_gc_info
|
8
|
+
|
9
|
+
if HAS_STAT_HEAP
|
10
|
+
attr_reader :stat_heap
|
11
|
+
end
|
12
|
+
|
13
|
+
def initialize
|
14
|
+
@stat = GC.stat
|
15
|
+
@latest_gc_info = GC.latest_gc_info
|
16
|
+
|
17
|
+
if HAS_STAT_HEAP
|
18
|
+
@stat_heap = GC.stat_heap
|
19
|
+
end
|
20
|
+
end
|
21
|
+
|
22
|
+
def update
|
23
|
+
GC.stat(@stat)
|
24
|
+
GC.latest_gc_info(@latest_gc_info)
|
25
|
+
|
26
|
+
if HAS_STAT_HEAP
|
27
|
+
GC.stat_heap(nil, @stat_heap)
|
28
|
+
end
|
29
|
+
end
|
30
|
+
end
|
31
|
+
end
|
@@ -1,6 +1,6 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
|
-
module
|
3
|
+
module Autotuner
|
4
4
|
module Heuristic
|
5
5
|
class Base
|
6
6
|
class << self
|
@@ -19,15 +19,23 @@ module GCTuner
|
|
19
19
|
end
|
20
20
|
end
|
21
21
|
|
22
|
+
def initialize(system_context)
|
23
|
+
@system_context = system_context
|
24
|
+
end
|
25
|
+
|
26
|
+
def name
|
27
|
+
raise NotImplementedError
|
28
|
+
end
|
29
|
+
|
22
30
|
def call(request_time, before_gc_context, after_gc_context)
|
23
31
|
raise NotImplementedError
|
24
32
|
end
|
25
33
|
|
26
|
-
def
|
34
|
+
def tuning_report
|
27
35
|
raise NotImplementedError
|
28
36
|
end
|
29
37
|
|
30
|
-
def
|
38
|
+
def debug_state
|
31
39
|
raise NotImplementedError
|
32
40
|
end
|
33
41
|
end
|
@@ -0,0 +1,60 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Autotuner
|
4
|
+
module Heuristic
|
5
|
+
class GCCompact < Base
|
6
|
+
class << self
|
7
|
+
private
|
8
|
+
|
9
|
+
def supported?
|
10
|
+
true
|
11
|
+
end
|
12
|
+
end
|
13
|
+
|
14
|
+
def initialize(_system_context)
|
15
|
+
super
|
16
|
+
|
17
|
+
@called_gc_compact = nil
|
18
|
+
end
|
19
|
+
|
20
|
+
def name
|
21
|
+
"GCCompact"
|
22
|
+
end
|
23
|
+
|
24
|
+
def call(request_context)
|
25
|
+
return unless @called_gc_compact.nil?
|
26
|
+
|
27
|
+
@called_gc_compact = request_context.before_gc_context.stat[:compact_count] > 0
|
28
|
+
end
|
29
|
+
|
30
|
+
def tuning_report
|
31
|
+
return if @called_gc_compact
|
32
|
+
|
33
|
+
# Don't give suggestion twice
|
34
|
+
@called_gc_compact = true
|
35
|
+
|
36
|
+
Report::String.new(<<~MSG)
|
37
|
+
The following suggestion runs compaction at boot time, which reduces fragmentation inside of the Ruby heap. This can improve performance and reduce memory usage in forking web servers.
|
38
|
+
|
39
|
+
Before forking your web server, run the following Ruby code:
|
40
|
+
|
41
|
+
3.times { GC.start }
|
42
|
+
GC.compact
|
43
|
+
|
44
|
+
For example, in Puma, add the following code into config/puma.rb:
|
45
|
+
|
46
|
+
before_fork do
|
47
|
+
3.times { GC.start }
|
48
|
+
GC.compact
|
49
|
+
end
|
50
|
+
MSG
|
51
|
+
end
|
52
|
+
|
53
|
+
def debug_state
|
54
|
+
{
|
55
|
+
called_gc_compact: @called_gc_compact,
|
56
|
+
}
|
57
|
+
end
|
58
|
+
end
|
59
|
+
end
|
60
|
+
end
|
@@ -0,0 +1,130 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Autotuner
|
4
|
+
module Heuristic
|
5
|
+
class HeapSizeWarmup < Base
|
6
|
+
class << self
|
7
|
+
private
|
8
|
+
|
9
|
+
def supported?
|
10
|
+
# Ruby 3.2 uses multiple heaps but does not support the
|
11
|
+
# RUBY_GC_HEAP_%d_INIT_SLOTS environment variables, so we cannot
|
12
|
+
# accurately tune the heap size.
|
13
|
+
!RUBY_VERSION.start_with?("3.2.")
|
14
|
+
end
|
15
|
+
end
|
16
|
+
|
17
|
+
# Ruby 3.3.0 and later have support for RUBY_GC_HEAP_%d_INIT_SLOTS
|
18
|
+
SUPPORT_MULTI_HEAP_P = RUBY_VERSION >= "3.3.0"
|
19
|
+
|
20
|
+
HEAP_NAMES =
|
21
|
+
if SUPPORT_MULTI_HEAP_P
|
22
|
+
GC.stat_heap.keys.map(&:to_s).freeze
|
23
|
+
else
|
24
|
+
[nil]
|
25
|
+
end
|
26
|
+
|
27
|
+
HEAP_SIZE_CONFIGURATION_DELTA_RATIO = 0.01
|
28
|
+
HEAP_SIZE_CONFIGURATION_DELTA = 1_000
|
29
|
+
|
30
|
+
REPORT_ASSIST_MESSAGE = <<~MSG
|
31
|
+
The following suggestions adjusts the size of heap at boot time, which can improve bootup speed and reduce the time taken for the app to reach peak performance.
|
32
|
+
MSG
|
33
|
+
|
34
|
+
def initialize(_system_context)
|
35
|
+
super
|
36
|
+
|
37
|
+
@heaps_data = Array.new(HEAP_NAMES.length)
|
38
|
+
HEAP_NAMES.length.times do |i|
|
39
|
+
@heaps_data[i] = DataStructure::DataPoints.new(Configuration::DATA_POINTS_COUNT)
|
40
|
+
end
|
41
|
+
|
42
|
+
@given_suggestion = false
|
43
|
+
end
|
44
|
+
|
45
|
+
def name
|
46
|
+
"HeapSizeWarmup"
|
47
|
+
end
|
48
|
+
|
49
|
+
def call(request_context)
|
50
|
+
# We only want to collect data at boot until plateau
|
51
|
+
return if @given_suggestion
|
52
|
+
|
53
|
+
@heaps_data.each_with_index do |data, i|
|
54
|
+
value =
|
55
|
+
if SUPPORT_MULTI_HEAP_P
|
56
|
+
request_context.after_gc_context.stat_heap[i][:heap_eden_slots]
|
57
|
+
else
|
58
|
+
request_context.after_gc_context.stat[:heap_available_slots]
|
59
|
+
end
|
60
|
+
|
61
|
+
data.insert(value)
|
62
|
+
end
|
63
|
+
end
|
64
|
+
|
65
|
+
def tuning_report
|
66
|
+
# Don't give suggestions twice
|
67
|
+
return if @given_suggestion
|
68
|
+
# The request time should plateau
|
69
|
+
return unless @system_context.request_time_data.plateaued?
|
70
|
+
|
71
|
+
@given_suggestion = true
|
72
|
+
|
73
|
+
env_names = []
|
74
|
+
suggested_values = []
|
75
|
+
configured_values = []
|
76
|
+
HEAP_NAMES.each_with_index do |heap_name, i|
|
77
|
+
env_name = env_name_for_heap(heap_name)
|
78
|
+
|
79
|
+
data = @heaps_data[i]
|
80
|
+
suggested_value = data.samples[data.length - 1].to_i
|
81
|
+
|
82
|
+
env_val = ENV[env_name]
|
83
|
+
configured_value = env_val&.to_i
|
84
|
+
|
85
|
+
if configured_value
|
86
|
+
diff = (suggested_value - configured_value).abs
|
87
|
+
|
88
|
+
# Don't report this if it's within the ratio
|
89
|
+
next if diff <= configured_value * HEAP_SIZE_CONFIGURATION_DELTA_RATIO
|
90
|
+
# Don't report this if it's within the delta
|
91
|
+
next if diff <= HEAP_SIZE_CONFIGURATION_DELTA
|
92
|
+
end
|
93
|
+
|
94
|
+
env_names << env_name
|
95
|
+
suggested_values << suggested_value
|
96
|
+
configured_values << configured_value
|
97
|
+
end
|
98
|
+
|
99
|
+
# Don't generate report if there is nothing to report
|
100
|
+
return if suggested_values.empty?
|
101
|
+
|
102
|
+
Report::MultipleEnvironmentVariables.new(REPORT_ASSIST_MESSAGE, env_names, suggested_values, configured_values)
|
103
|
+
end
|
104
|
+
|
105
|
+
def debug_state
|
106
|
+
state = {
|
107
|
+
given_suggestion: @given_suggestion,
|
108
|
+
}
|
109
|
+
|
110
|
+
# Don't output @heaps_data because there is too much data.
|
111
|
+
|
112
|
+
HEAP_NAMES.each do |heap_name|
|
113
|
+
env_var = env_name_for_heap(heap_name)
|
114
|
+
env_val = ENV[env_var]
|
115
|
+
state[:"ENV[#{env_var}]"] = env_val if env_val
|
116
|
+
end
|
117
|
+
|
118
|
+
state
|
119
|
+
end
|
120
|
+
|
121
|
+
def env_name_for_heap(heap_name)
|
122
|
+
if SUPPORT_MULTI_HEAP_P
|
123
|
+
"RUBY_GC_HEAP_#{heap_name}_INIT_SLOTS"
|
124
|
+
else
|
125
|
+
"RUBY_GC_HEAP_INIT_SLOTS"
|
126
|
+
end
|
127
|
+
end
|
128
|
+
end
|
129
|
+
end
|
130
|
+
end
|
@@ -0,0 +1,121 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Autotuner
|
4
|
+
module Heuristic
|
5
|
+
class Malloc < Base
|
6
|
+
class << self
|
7
|
+
private
|
8
|
+
|
9
|
+
def supported?
|
10
|
+
true
|
11
|
+
end
|
12
|
+
end
|
13
|
+
|
14
|
+
MALLOC_GC_RATIO_THRESHOLD = 0.1
|
15
|
+
MIN_MALLOC_GC = 10
|
16
|
+
|
17
|
+
# From the GC_MALLOC_LIMIT_MIN macro
|
18
|
+
# https://github.com/ruby/ruby/blob/3874381c4483ba7794ac2abf157e265546f9bfa7/gc.c#L312C9-L312C9
|
19
|
+
DEFAULT_MALLOC_LIMIT = 16 * 1024 * 1024
|
20
|
+
# From the GC_MALLOC_LIMIT_MAX macro
|
21
|
+
# https://github.com/ruby/ruby/blob/3874381c4483ba7794ac2abf157e265546f9bfa7/gc.c#L315C9-L315C28
|
22
|
+
DEFAULT_MALLOC_LIMIT_MAX = 32 * 1024 * 1024
|
23
|
+
|
24
|
+
LIMIT_ENV = "RUBY_GC_MALLOC_LIMIT"
|
25
|
+
LIMIT_MAX_ENV = "RUBY_GC_MALLOC_LIMIT_MAX"
|
26
|
+
|
27
|
+
attr_reader :minor_gc_count
|
28
|
+
attr_reader :malloc_gc_count
|
29
|
+
|
30
|
+
def initialize(_system_context)
|
31
|
+
super
|
32
|
+
|
33
|
+
@minor_gc_count = 0
|
34
|
+
@malloc_gc_count = 0
|
35
|
+
|
36
|
+
@given_suggestion = false
|
37
|
+
end
|
38
|
+
|
39
|
+
def name
|
40
|
+
"Malloc"
|
41
|
+
end
|
42
|
+
|
43
|
+
def call(request_context)
|
44
|
+
# gc_by is only useful if we ran at least one minor GC during the request.
|
45
|
+
if request_context.after_gc_context.stat[:minor_gc_count] ==
|
46
|
+
request_context.before_gc_context.stat[:minor_gc_count]
|
47
|
+
return
|
48
|
+
end
|
49
|
+
# gc_by is only useful if it wasn't a major GC.
|
50
|
+
# It is a major GC when where is a major_by reason set.
|
51
|
+
return if request_context.after_gc_context.latest_gc_info[:major_by]
|
52
|
+
|
53
|
+
@minor_gc_count += 1
|
54
|
+
@malloc_gc_count += 1 if request_context.after_gc_context.latest_gc_info[:gc_by] == :malloc
|
55
|
+
end
|
56
|
+
|
57
|
+
def tuning_report
|
58
|
+
# Don't give suggestions twice.
|
59
|
+
return if @given_suggestion
|
60
|
+
# Don't report if there's very few data points
|
61
|
+
return if malloc_gc_count < MIN_MALLOC_GC
|
62
|
+
|
63
|
+
malloc_gc_ratio = malloc_gc_count.to_f / minor_gc_count
|
64
|
+
# Don't report if there's very few malloc GC.
|
65
|
+
return if malloc_gc_ratio <= MALLOC_GC_RATIO_THRESHOLD
|
66
|
+
|
67
|
+
@given_suggestion = true
|
68
|
+
|
69
|
+
Report::MultipleEnvironmentVariables.new(
|
70
|
+
<<~MSG,
|
71
|
+
The following suggestions reduces the number of minor garbage collection cycles, specifically a cycle called "malloc". Your app runs malloc cycles in approximately #{format("%.2f", malloc_gc_ratio * 100)}% of all minor garbage collection cycles.
|
72
|
+
|
73
|
+
Reducing minor garbage collection cycles can help reduce response times. The following tuning values aims to reduce malloc garbage collection cycles by setting it to a higher value. This may cause a slight increase in memory usage. You should monitor memory usage carefully to ensure your app is not running out of memory.
|
74
|
+
MSG
|
75
|
+
[LIMIT_ENV, LIMIT_MAX_ENV],
|
76
|
+
# Suggest to double the limit and max
|
77
|
+
[suggested_malloc_limit, suggested_malloc_limit_max],
|
78
|
+
[configured_malloc_limit, configured_malloc_limit_max],
|
79
|
+
)
|
80
|
+
end
|
81
|
+
|
82
|
+
def debug_state
|
83
|
+
{
|
84
|
+
given_suggestion: @given_suggestion,
|
85
|
+
minor_gc_count: minor_gc_count,
|
86
|
+
malloc_gc_count: malloc_gc_count,
|
87
|
+
}
|
88
|
+
end
|
89
|
+
|
90
|
+
private
|
91
|
+
|
92
|
+
def configured_malloc_limit
|
93
|
+
ENV[LIMIT_ENV]&.to_i
|
94
|
+
end
|
95
|
+
|
96
|
+
def configured_malloc_limit_max
|
97
|
+
ENV[LIMIT_MAX_ENV]&.to_i
|
98
|
+
end
|
99
|
+
|
100
|
+
def suggested_malloc_limit
|
101
|
+
if !configured_malloc_limit
|
102
|
+
DEFAULT_MALLOC_LIMIT * 2
|
103
|
+
elsif configured_malloc_limit < DEFAULT_MALLOC_LIMIT
|
104
|
+
DEFAULT_MALLOC_LIMIT
|
105
|
+
else
|
106
|
+
configured_malloc_limit * 2
|
107
|
+
end
|
108
|
+
end
|
109
|
+
|
110
|
+
def suggested_malloc_limit_max
|
111
|
+
if !configured_malloc_limit_max
|
112
|
+
DEFAULT_MALLOC_LIMIT_MAX * 2
|
113
|
+
elsif configured_malloc_limit_max < DEFAULT_MALLOC_LIMIT_MAX
|
114
|
+
DEFAULT_MALLOC_LIMIT_MAX
|
115
|
+
else
|
116
|
+
configured_malloc_limit_max * 2
|
117
|
+
end
|
118
|
+
end
|
119
|
+
end
|
120
|
+
end
|
121
|
+
end
|
@@ -0,0 +1,89 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Autotuner
|
4
|
+
module Heuristic
|
5
|
+
class Oldmalloc < Base
|
6
|
+
class << self
|
7
|
+
private
|
8
|
+
|
9
|
+
def supported?
|
10
|
+
true
|
11
|
+
end
|
12
|
+
end
|
13
|
+
|
14
|
+
OLDMALLOC_GC_RATIO_THRESHOLD = 0.01
|
15
|
+
MIN_OLDMALLOC_GC = 10
|
16
|
+
|
17
|
+
LIMIT_ENV = "RUBY_GC_OLDMALLOC_LIMIT"
|
18
|
+
LIMIT_MAX_ENV = "RUBY_GC_OLDMALLOC_LIMIT_MAX"
|
19
|
+
|
20
|
+
# Except for aggressively decreasing memory usage, it doesn't make sense
|
21
|
+
# for Rails apps to run oldmalloc major GC cycles. So we suggest an
|
22
|
+
# extremely high value (around 1TB here) to essentially disable it.
|
23
|
+
LIMIT_ENV_SUGGESTED_VALUE = 1_000_000_000_000
|
24
|
+
LIMIT_MAX_SUGGESTED_VALUE = 1_000_000_000_000
|
25
|
+
|
26
|
+
attr_reader :major_gc_count
|
27
|
+
attr_reader :oldmalloc_gc_count
|
28
|
+
|
29
|
+
def initialize(_system_context)
|
30
|
+
super
|
31
|
+
|
32
|
+
@major_gc_count = 0
|
33
|
+
@oldmalloc_gc_count = 0
|
34
|
+
|
35
|
+
@given_suggestion = false
|
36
|
+
end
|
37
|
+
|
38
|
+
def name
|
39
|
+
"Oldmalloc"
|
40
|
+
end
|
41
|
+
|
42
|
+
def call(request_context)
|
43
|
+
# major_by is only useful if we ran at least one major GC during the request
|
44
|
+
if request_context.after_gc_context.stat[:major_gc_count] ==
|
45
|
+
request_context.before_gc_context.stat[:major_gc_count]
|
46
|
+
return
|
47
|
+
end
|
48
|
+
|
49
|
+
# Technically, we could run more than one major GC in the request, but
|
50
|
+
# since we don't have information about the other major GC, we'll treat
|
51
|
+
# it as if there was only one major GC.
|
52
|
+
@major_gc_count += 1
|
53
|
+
@oldmalloc_gc_count += 1 if request_context.after_gc_context.latest_gc_info[:major_by] == :oldmalloc
|
54
|
+
end
|
55
|
+
|
56
|
+
def tuning_report
|
57
|
+
# Don't give suggestions twice
|
58
|
+
return if @given_suggestion
|
59
|
+
# Don't report if there's very few data points
|
60
|
+
return if @oldmalloc_gc_count < MIN_OLDMALLOC_GC
|
61
|
+
|
62
|
+
oldmalloc_gc_ratio = @oldmalloc_gc_count.to_f / @major_gc_count
|
63
|
+
# Don't report if there's very few oldmalloc GC
|
64
|
+
return if oldmalloc_gc_ratio <= OLDMALLOC_GC_RATIO_THRESHOLD
|
65
|
+
|
66
|
+
@given_suggestion = true
|
67
|
+
|
68
|
+
Report::MultipleEnvironmentVariables.new(
|
69
|
+
<<~MSG,
|
70
|
+
The following suggestions reduces the number of major garbage collection cycles, specifically a cycle called "oldmalloc". Your app runs oldmalloc cycles in approximately #{format("%.2f", oldmalloc_gc_ratio * 100)}% of all major garbage collection cycles.
|
71
|
+
|
72
|
+
Reducing major garbage collection cycles can help reduce response times, especially for the extremes (e.g. p95 or p99 response times). The following tuning values aims to disable oldmalloc garbage collection cycles by setting it to an extremely high value. This may cause a slight increase in memory usage. You should monitor memory usage carefully to ensure your app is not running out of memory.
|
73
|
+
MSG
|
74
|
+
[LIMIT_ENV, LIMIT_MAX_ENV],
|
75
|
+
[LIMIT_ENV_SUGGESTED_VALUE, LIMIT_MAX_SUGGESTED_VALUE],
|
76
|
+
[ENV[LIMIT_ENV]&.to_i, ENV[LIMIT_MAX_ENV]&.to_i],
|
77
|
+
)
|
78
|
+
end
|
79
|
+
|
80
|
+
def debug_state
|
81
|
+
{
|
82
|
+
given_suggestion: @given_suggestion,
|
83
|
+
major_gc_count: @major_gc_count,
|
84
|
+
oldmalloc_gc_count: @oldmalloc_gc_count,
|
85
|
+
}
|
86
|
+
end
|
87
|
+
end
|
88
|
+
end
|
89
|
+
end
|