bc-prometheus-ruby 0.2.4 → 0.5.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +28 -0
- data/README.md +133 -10
- data/bc-prometheus-ruby.gemspec +7 -6
- data/lib/bigcommerce/prometheus.rb +3 -0
- data/lib/bigcommerce/prometheus/client.rb +16 -8
- data/lib/bigcommerce/prometheus/collectors/base.rb +20 -10
- data/lib/bigcommerce/prometheus/configuration.rb +33 -19
- data/lib/bigcommerce/prometheus/instrumentors/hutch.rb +8 -0
- data/lib/bigcommerce/prometheus/instrumentors/resque.rb +7 -1
- data/lib/bigcommerce/prometheus/instrumentors/web.rb +1 -1
- data/lib/bigcommerce/prometheus/integrations/railtie.rb +1 -1
- data/lib/bigcommerce/prometheus/integrations/resque.rb +3 -3
- data/lib/bigcommerce/prometheus/server.rb +10 -4
- data/lib/bigcommerce/prometheus/servers/thin/controllers/not_found_controller.rb +1 -1
- data/lib/bigcommerce/prometheus/servers/thin/controllers/send_metrics_controller.rb +1 -0
- data/lib/bigcommerce/prometheus/servers/thin/server.rb +5 -4
- data/lib/bigcommerce/prometheus/servers/thin/server_metrics.rb +1 -1
- data/lib/bigcommerce/prometheus/type_collectors/base.rb +1 -0
- data/lib/bigcommerce/prometheus/type_collectors/resque.rb +2 -0
- data/lib/bigcommerce/prometheus/version.rb +1 -1
- metadata +29 -29
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 4a5bc3a1ae57f782c0f94d6cf490830fce96e78ab0f782a4761cad8dcc5850ae
|
4
|
+
data.tar.gz: 694ede704dc83a7c9fd0f1abfd4300bd7e464b464b305833be44c7b06bf62bd5
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 20a0ccb53fc272f5f80425503fe492908c6cca2a01186b0489141091239671e848d4930ba29c54ada04c78149ebbda6f2f240880e0da24d2bc7a5b7b98c2b8bc
|
7
|
+
data.tar.gz: ebe5d43de15abe3d933050add69fb785280506bbe6f6acab6065eee215534aeb61bd025eaec545ca9bd682083b7751e01a1569c80daa43313dfaeac72b7d5163
|
data/CHANGELOG.md
CHANGED
@@ -2,6 +2,34 @@ Changelog for the bc-prometheus-ruby gem.
|
|
2
2
|
|
3
3
|
### Pending Release
|
4
4
|
|
5
|
+
### 0.5.1
|
6
|
+
|
7
|
+
- Fix keywords argument issue with Collectors::Base and Ruby 3.0+
|
8
|
+
|
9
|
+
### 0.5.0
|
10
|
+
|
11
|
+
- Add configuration to disable the Railtie that activates the web instrumentor automatically. This allows applications to choose how and when this is initialized.
|
12
|
+
- Bump prometheus_exporter gem
|
13
|
+
- Start testing against Ruby 3.0
|
14
|
+
|
15
|
+
### 0.4.0
|
16
|
+
|
17
|
+
- Add configuration to control Thin web server thread pool size. Note that the default number of threads is changing from 20 to 3. You can configure this using an environment variable or initializer.
|
18
|
+
- Update rubocop to 1.0
|
19
|
+
|
20
|
+
### 0.3.1
|
21
|
+
|
22
|
+
- Update prometheus_exporter dependency to ~> 0.5 to fix memory leaks
|
23
|
+
|
24
|
+
### 0.3.0
|
25
|
+
|
26
|
+
- Support for only Ruby 2.6+ going forward
|
27
|
+
- Updated README around custom metrics and collectors
|
28
|
+
- Add ability to pass custom resque and hutch Collectors/TypeCollectors
|
29
|
+
- Add ENV support for all configuration elements
|
30
|
+
- Fix issue where base collector did not use Bigcommerce::Prometheus.client
|
31
|
+
- Expose new `push` method for Collectors::Base to ease use of custom ad hoc metrics
|
32
|
+
|
5
33
|
### 0.2.4
|
6
34
|
|
7
35
|
- Fix cant modify frozen array error when using bc-prometheus-ruby outside a web process
|
data/README.md
CHANGED
@@ -1,6 +1,6 @@
|
|
1
1
|
# bc-prometheus-ruby - Drop-in Prometheus metrics
|
2
2
|
|
3
|
-
[![CircleCI](https://circleci.com/gh/bigcommerce/bc-prometheus-ruby.svg?style=svg&circle-token=fc3e2c4405a1f53a31e298f0ef981c2d0dfdee90)](https://circleci.com/gh/bigcommerce/bc-prometheus-ruby) [![Gem Version](https://badge.fury.io/rb/bc-prometheus-ruby.svg)](https://badge.fury.io/rb/bc-prometheus-ruby) [![Documentation](https://inch-ci.org/github/bigcommerce/bc-prometheus-ruby.svg?branch=
|
3
|
+
[![CircleCI](https://circleci.com/gh/bigcommerce/bc-prometheus-ruby.svg?style=svg&circle-token=fc3e2c4405a1f53a31e298f0ef981c2d0dfdee90)](https://circleci.com/gh/bigcommerce/bc-prometheus-ruby) [![Gem Version](https://badge.fury.io/rb/bc-prometheus-ruby.svg)](https://badge.fury.io/rb/bc-prometheus-ruby) [![Documentation](https://inch-ci.org/github/bigcommerce/bc-prometheus-ruby.svg?branch=main)](https://inch-ci.org/github/bigcommerce/bc-prometheus-ruby?branch=main)
|
4
4
|
|
5
5
|
## Installation
|
6
6
|
|
@@ -39,15 +39,138 @@ Bigcommerce::Prometheus::Instrumentors::Resque.new(app: Rails.application).start
|
|
39
39
|
|
40
40
|
After requiring the main file, you can further configure with:
|
41
41
|
|
42
|
-
| Option | Description | Default |
|
43
|
-
| ------ | ----------- | ------- |
|
44
|
-
| client_custom_labels | A hash of custom labels to send with each client request | `{}` |
|
45
|
-
| client_max_queue_size | The max amount of metrics to send before flushing | 10000 |
|
46
|
-
| client_thread_sleep | How often to sleep the worker thread that manages the client buffer (seconds) | 0.5 |
|
47
|
-
| puma_collection_frequency | How often to poll puma collection metrics (seconds) | 30 |
|
48
|
-
| server_host | The host to run the exporter on | 0.0.0.0 |
|
49
|
-
| server_port | The port to run the exporter on | 9394 |
|
50
|
-
|
|
42
|
+
| Option | Description | Default | Environment Variable |
|
43
|
+
| ------ | ----------- | ------- | -------------------- |
|
44
|
+
| client_custom_labels | A hash of custom labels to send with each client request | `{}` | None |
|
45
|
+
| client_max_queue_size | The max amount of metrics to send before flushing | `10000` | `ENV['PROMETHEUS_CLIENT_MAX_QUEUE_SIZE']` |
|
46
|
+
| client_thread_sleep | How often to sleep the worker thread that manages the client buffer (seconds) | `0.5` | `ENV['PROMETHEUS_CLIENT_THREAD_SLEEP']` |
|
47
|
+
| puma_collection_frequency | How often to poll puma collection metrics (seconds) | `30` | `ENV['PROMETHEUS_PUMA_COLLECTION_FREQUENCY']` |
|
48
|
+
| server_host | The host to run the exporter on | `"0.0.0.0"` | `ENV['PROMETHEUS_SERVER_HOST']` |
|
49
|
+
| server_port | The port to run the exporter on | `9394` | `ENV['PROMETHEUS_SERVER_PORT']` |
|
50
|
+
| server_thread_pool_size | The number of threads used for the exporter server | `3` | `ENV['PROMETHEUS_SERVER_THREAD_POOL_SIZE']` |
|
51
|
+
| process_name | What the current process name is (used in logging) | `"unknown"` | `ENV['PROCESS']` |
|
52
|
+
| railtie_disabled | Opt out flag for Railtie; use `Bigcommerce::Prometheus::Instrumentors::Web.new(app: Rails.application).start` in your app's code to start it up yourself | `0` | `ENV['PROMETHEUS_DISABLE_RAILTIE']` |
|
53
|
+
|
54
|
+
## Custom Collectors
|
55
|
+
|
56
|
+
To create custom metrics and collectors, simply create two files: a collector (the class that runs and collects metrics),
|
57
|
+
and the type collector, which runs on the threaded prometheus server and
|
58
|
+
|
59
|
+
### Type Collector
|
60
|
+
|
61
|
+
First, create a type collector. Note that the "type" of this will be the full name of the class, with `TypeCollector`
|
62
|
+
stripped. This is important later. Our example here will have a "type" of "app".
|
63
|
+
|
64
|
+
```ruby
|
65
|
+
class AppTypeCollector < ::Bigcommerce::Prometheus::TypeCollectors::Base
|
66
|
+
def build_metrics
|
67
|
+
{
|
68
|
+
honks: PrometheusExporter::Metric::Counter.new('honks', 'Running counter of honks'),
|
69
|
+
points: PrometheusExporter::Metric::Gauge.new('points', 'Current amount of points')
|
70
|
+
}
|
71
|
+
end
|
72
|
+
|
73
|
+
def collect_metrics(data:, labels: {})
|
74
|
+
metric(:points).observe(data.fetch('points', 0))
|
75
|
+
metric(:honks).observe(1, labels) if data.fetch('honks', 0).to_i.positive?
|
76
|
+
end
|
77
|
+
end
|
78
|
+
```
|
79
|
+
|
80
|
+
There are two important methods here: `build_metrics`, which registers the different metrics you want to measure, and
|
81
|
+
`collect_metrics`, which actually takes in the metrics and prepares them to be rendered so that Prometheus can scrape
|
82
|
+
them.
|
83
|
+
|
84
|
+
Note also in the example the different ways of observing Gauges vs Counters.
|
85
|
+
|
86
|
+
### Collector
|
87
|
+
|
88
|
+
Next, create a collector. Your "type" of the Collector must match the type collector above, so that bc-prometheus-ruby
|
89
|
+
knows how to map the metrics to the right TypeCollector. This is inferred from the class name. Here, it is "app":
|
90
|
+
|
91
|
+
```ruby
|
92
|
+
class AppCollector < ::Bigcommerce::Prometheus::Collectors::Base
|
93
|
+
def honk!
|
94
|
+
push(
|
95
|
+
honks: 1,
|
96
|
+
custom_labels: {
|
97
|
+
volume: 'loud'
|
98
|
+
}
|
99
|
+
)
|
100
|
+
end
|
101
|
+
|
102
|
+
def collect(metrics)
|
103
|
+
metrics[:points] = rand(1..100)
|
104
|
+
metrics
|
105
|
+
end
|
106
|
+
end
|
107
|
+
```
|
108
|
+
|
109
|
+
There are two types of metrics here: on-demand, and polled. Let's look at the first:
|
110
|
+
|
111
|
+
#### On-Demand Metrics
|
112
|
+
|
113
|
+
To issue an on-demand metric (usually a counter) that then automatically updates, in your application code, you would
|
114
|
+
then run:
|
115
|
+
|
116
|
+
```ruby
|
117
|
+
app_collector = AppCollector.new
|
118
|
+
app_collector.honk!
|
119
|
+
```
|
120
|
+
|
121
|
+
This will "push" the metrics to our `AppTypeCollector` instance, which will render them as:
|
122
|
+
|
123
|
+
```
|
124
|
+
# HELP ruby_honks Running counter of honks
|
125
|
+
# TYPE ruby_honks counter
|
126
|
+
ruby_honks{volume="loud"} 2
|
127
|
+
```
|
128
|
+
|
129
|
+
As you can see this will respect any custom labels we push in as well.
|
130
|
+
|
131
|
+
### Polling Metrics
|
132
|
+
|
133
|
+
Using our same AppCollector, if you note the `collect` method: this method will run on a 15 second polled basis
|
134
|
+
(the frequency of which is configurable in the initializer of the AppCollector). Here we're just spitting out random
|
135
|
+
points, so it'll look something like this:
|
136
|
+
|
137
|
+
```
|
138
|
+
# HELP ruby_points Current amount of points
|
139
|
+
# TYPE ruby_points gauge
|
140
|
+
ruby_points 42
|
141
|
+
```
|
142
|
+
|
143
|
+
### Registering Our Collectors
|
144
|
+
|
145
|
+
Each different type of integration will need to have the collectors passed into them, where appropriate. For example,
|
146
|
+
if we want these collectors to run on our web, resque, and hutch processes, we'll need to:
|
147
|
+
|
148
|
+
```ruby
|
149
|
+
::Bigcommerce::Prometheus.configure do |c|
|
150
|
+
c.web_collectors = [AppCollector]
|
151
|
+
c.web_type_collectors = [AppTypeCollector.new]
|
152
|
+
c.resque_collectors = [AppCollector]
|
153
|
+
c.resque_type_collectors = [AppTypeCollector.new]
|
154
|
+
c.hutch_collectors = [AppCollector]
|
155
|
+
c.hutch_type_collectors = [AppTypeCollector.new]
|
156
|
+
end
|
157
|
+
```
|
158
|
+
|
159
|
+
#### Custom Server Integrations
|
160
|
+
|
161
|
+
For custom integrations that initialize their own server, you'll need to pass your TypeCollector instance via the
|
162
|
+
`.add_type_collector` method on the prometheus server instance before starting it:
|
163
|
+
|
164
|
+
```ruby
|
165
|
+
server = ::Bigcommerce::Prometheus::Server.new
|
166
|
+
Bigcommerce::Prometheus.web_type_collectors.each do |tc|
|
167
|
+
server.add_type_collector(tc)
|
168
|
+
end
|
169
|
+
|
170
|
+
# and for polling:
|
171
|
+
|
172
|
+
AppCollector.start
|
173
|
+
```
|
51
174
|
|
52
175
|
## License
|
53
176
|
|
data/bc-prometheus-ruby.gemspec
CHANGED
@@ -15,7 +15,7 @@
|
|
15
15
|
# COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
|
16
16
|
# OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
17
17
|
#
|
18
|
-
|
18
|
+
$LOAD_PATH.push File.expand_path('lib', __dir__)
|
19
19
|
require 'bigcommerce/prometheus/version'
|
20
20
|
|
21
21
|
Gem::Specification.new do |spec|
|
@@ -31,17 +31,18 @@ Gem::Specification.new do |spec|
|
|
31
31
|
|
32
32
|
spec.files = Dir['README.md', 'CHANGELOG.md', 'CODE_OF_CONDUCT.md', 'lib/**/*', 'bc-prometheus-ruby.gemspec']
|
33
33
|
spec.require_paths = ['lib']
|
34
|
+
spec.required_ruby_version = '>= 2.6'
|
34
35
|
|
35
|
-
spec.add_development_dependency 'rake', '>= 10.0'
|
36
|
-
spec.add_development_dependency 'rspec', '>= 3.8'
|
37
|
-
spec.add_development_dependency 'rspec_junit_formatter', '>= 0.4'
|
38
36
|
spec.add_development_dependency 'bundler-audit', '>= 0.6'
|
39
37
|
spec.add_development_dependency 'null-logger', '>= 0.1'
|
40
38
|
spec.add_development_dependency 'pry', '>= 0.12'
|
41
|
-
spec.add_development_dependency '
|
39
|
+
spec.add_development_dependency 'rake', '>= 10.0'
|
40
|
+
spec.add_development_dependency 'rspec', '>= 3.8'
|
41
|
+
spec.add_development_dependency 'rspec_junit_formatter', '>= 0.4'
|
42
|
+
spec.add_development_dependency 'rubocop', '>= 1.0'
|
42
43
|
spec.add_development_dependency 'simplecov', '>= 0.16'
|
43
44
|
|
44
45
|
spec.add_runtime_dependency 'bigcommerce-multitrap', '~> 0.1'
|
45
|
-
spec.add_runtime_dependency 'prometheus_exporter', '~> 0.
|
46
|
+
spec.add_runtime_dependency 'prometheus_exporter', '~> 0.7'
|
46
47
|
spec.add_runtime_dependency 'thin', '~> 1.7'
|
47
48
|
end
|
@@ -24,16 +24,24 @@ module Bigcommerce
|
|
24
24
|
include Singleton
|
25
25
|
include Loggable
|
26
26
|
|
27
|
-
|
27
|
+
##
|
28
|
+
# @param [String] host
|
29
|
+
# @param [Integer] port
|
30
|
+
# @param [Integer] max_queue_size
|
31
|
+
# @param [Integer|Float] thread_sleep
|
32
|
+
# @param [Hash] custom_labels
|
33
|
+
# @param [String] process_name
|
34
|
+
#
|
35
|
+
def initialize(host: nil, port: nil, max_queue_size: nil, thread_sleep: nil, custom_labels: nil, process_name: nil)
|
28
36
|
super(
|
29
|
-
host: Bigcommerce::Prometheus.server_host,
|
30
|
-
port: Bigcommerce::Prometheus.server_port,
|
31
|
-
max_queue_size: Bigcommerce::Prometheus.client_max_queue_size,
|
32
|
-
thread_sleep: Bigcommerce::Prometheus.client_thread_sleep,
|
33
|
-
custom_labels: Bigcommerce::Prometheus.client_custom_labels
|
37
|
+
host: host || Bigcommerce::Prometheus.server_host,
|
38
|
+
port: port || Bigcommerce::Prometheus.server_port,
|
39
|
+
max_queue_size: max_queue_size || Bigcommerce::Prometheus.client_max_queue_size,
|
40
|
+
thread_sleep: thread_sleep || Bigcommerce::Prometheus.client_thread_sleep,
|
41
|
+
custom_labels: custom_labels || Bigcommerce::Prometheus.client_custom_labels
|
34
42
|
)
|
35
43
|
PrometheusExporter::Client.default = self
|
36
|
-
@process_name = ::Bigcommerce::Prometheus.process_name
|
44
|
+
@process_name = process_name || ::Bigcommerce::Prometheus.process_name
|
37
45
|
end
|
38
46
|
|
39
47
|
##
|
@@ -55,7 +63,7 @@ module Bigcommerce
|
|
55
63
|
|
56
64
|
##
|
57
65
|
# @param [String] path
|
58
|
-
# @return [URI]
|
66
|
+
# @return [Module<URI>]
|
59
67
|
#
|
60
68
|
def uri_path(path)
|
61
69
|
URI("http://#{@host}:#{@port}#{path}")
|
@@ -25,8 +25,8 @@ module Bigcommerce
|
|
25
25
|
##
|
26
26
|
# Start the collector
|
27
27
|
#
|
28
|
-
def self.start(
|
29
|
-
process_collector = new(
|
28
|
+
def self.start(args, &block)
|
29
|
+
process_collector = new(**args, &block)
|
30
30
|
|
31
31
|
stop if @thread
|
32
32
|
|
@@ -49,15 +49,15 @@ module Bigcommerce
|
|
49
49
|
end
|
50
50
|
|
51
51
|
##
|
52
|
-
# @param [
|
52
|
+
# @param [Bigcommerce::Prometheus::Client] client
|
53
53
|
# @param [String] type
|
54
54
|
# @param [Integer] frequency
|
55
55
|
# @param [Hash] options
|
56
56
|
#
|
57
57
|
def initialize(client: nil, type: nil, frequency: nil, options: nil)
|
58
|
-
@client = client ||
|
58
|
+
@client = client || Bigcommerce::Prometheus.client
|
59
59
|
@type = type || self.class.to_s.downcase.gsub('::', '_').gsub('collector', '')
|
60
|
-
@frequency = frequency ||
|
60
|
+
@frequency = frequency || Bigcommerce::Prometheus.collector_collection_frequency
|
61
61
|
@options = options || {}
|
62
62
|
@logger = Bigcommerce::Prometheus.logger
|
63
63
|
end
|
@@ -66,12 +66,9 @@ module Bigcommerce
|
|
66
66
|
# Run the collector and send stats
|
67
67
|
#
|
68
68
|
def run
|
69
|
-
metrics = {
|
69
|
+
metrics = {}
|
70
70
|
metrics = collect(metrics)
|
71
|
-
|
72
|
-
@client.send_json(metrics)
|
73
|
-
rescue StandardError => e
|
74
|
-
@logger.error("Prometheus Exporter Failed To Collect #{@type} Stats #{e}")
|
71
|
+
push(metrics)
|
75
72
|
ensure
|
76
73
|
sleep @frequency
|
77
74
|
end
|
@@ -85,6 +82,19 @@ module Bigcommerce
|
|
85
82
|
def collect(metrics = {})
|
86
83
|
metrics
|
87
84
|
end
|
85
|
+
|
86
|
+
private
|
87
|
+
|
88
|
+
##
|
89
|
+
# @param [Hash] metric
|
90
|
+
#
|
91
|
+
def push(metric)
|
92
|
+
metric[:type] = @type unless metric.key?(:type)
|
93
|
+
@logger.debug("[bigcommerce-prometheus] Pushing #{metric[:type]} metrics to type collector: #{metric.inspect}")
|
94
|
+
@client.send_json(metric)
|
95
|
+
rescue StandardError => e
|
96
|
+
@logger.error("[bigcommerce-prometheus] Prometheus Exporter failed to send #{metric[:type]} stats to type collector #{e}")
|
97
|
+
end
|
88
98
|
end
|
89
99
|
end
|
90
100
|
end
|
@@ -23,20 +23,38 @@ module Bigcommerce
|
|
23
23
|
module Configuration
|
24
24
|
VALID_CONFIG_KEYS = {
|
25
25
|
logger: nil,
|
26
|
+
enabled: ENV.fetch('PROMETHEUS_ENABLED', 1).to_i.positive?,
|
27
|
+
|
28
|
+
# Client configuration
|
26
29
|
client_custom_labels: nil,
|
27
|
-
client_max_queue_size: 10_000,
|
28
|
-
client_thread_sleep: 0.5,
|
29
|
-
|
30
|
-
|
31
|
-
|
32
|
-
|
33
|
-
|
34
|
-
|
35
|
-
|
36
|
-
|
37
|
-
|
30
|
+
client_max_queue_size: ENV.fetch('PROMETHEUS_CLIENT_MAX_QUEUE_SIZE', 10_000).to_i,
|
31
|
+
client_thread_sleep: ENV.fetch('PROMETHEUS_CLIENT_THREAD_SLEEP', 0.5).to_f,
|
32
|
+
|
33
|
+
# Integration configuration
|
34
|
+
puma_collection_frequency: ENV.fetch('PROMETHEUS_PUMA_COLLECTION_FREQUENCY', 30).to_i,
|
35
|
+
puma_process_label: ENV.fetch('PROMETHEUS_PUMA_PROCESS_LABEL', 'web').to_s,
|
36
|
+
resque_collection_frequency: ENV.fetch('PROMETHEUS_RESQUE_COLLECTION_FREQUENCY', 30).to_i,
|
37
|
+
resque_process_label: ENV.fetch('PROMETHEUS_REQUEST_PROCESS_LABEL', 'resque').to_s,
|
38
|
+
|
39
|
+
# Server configuration
|
40
|
+
not_found_body: ENV.fetch('PROMETHEUS_SERVER_NOT_FOUND_BODY', 'Not Found! The Prometheus Ruby Exporter only listens on /metrics and /send-metrics').to_s,
|
41
|
+
server_host: ENV.fetch('PROMETHEUS_SERVER_HOST', '0.0.0.0').to_s,
|
42
|
+
server_port: ENV.fetch('PROMETHEUS_SERVER_PORT', PrometheusExporter::DEFAULT_PORT).to_i,
|
43
|
+
server_timeout: ENV.fetch('PROMETHEUS_DEFAULT_TIMEOUT', PrometheusExporter::DEFAULT_TIMEOUT).to_i,
|
44
|
+
server_prefix: ENV.fetch('PROMETHEUS_DEFAULT_PREFIX', PrometheusExporter::DEFAULT_PREFIX).to_s,
|
45
|
+
server_thread_pool_size: ENV.fetch('PROMETHEUS_SERVER_THREAD_POOL_SIZE', 3).to_i,
|
46
|
+
|
47
|
+
# Custom collector configuration
|
48
|
+
collector_collection_frequency: ENV.fetch('PROMETHEUS_DEFAULT_COLLECTOR_COLLECTION_FREQUENCY_SEC', 15).to_i,
|
49
|
+
hutch_collectors: [],
|
50
|
+
hutch_type_collectors: [],
|
51
|
+
resque_collectors: [],
|
52
|
+
resque_type_collectors: [],
|
38
53
|
web_collectors: [],
|
39
|
-
web_type_collectors: []
|
54
|
+
web_type_collectors: [],
|
55
|
+
|
56
|
+
# Additional configuration
|
57
|
+
railtie_disabled: ENV.fetch('PROMETHEUS_DISABLE_RAILTIE', 0).to_i.positive?
|
40
58
|
}.freeze
|
41
59
|
|
42
60
|
attr_accessor *VALID_CONFIG_KEYS.keys
|
@@ -62,6 +80,7 @@ module Bigcommerce
|
|
62
80
|
reset unless @configured
|
63
81
|
yield self
|
64
82
|
@configured = true
|
83
|
+
self
|
65
84
|
end
|
66
85
|
|
67
86
|
##
|
@@ -85,12 +104,7 @@ module Bigcommerce
|
|
85
104
|
send("#{k}=".to_sym, v)
|
86
105
|
end
|
87
106
|
determine_logger
|
88
|
-
self.enabled = ENV.fetch('PROMETHEUS_ENABLED', 1).to_i.positive?
|
89
|
-
self.server_host = ENV.fetch('PROMETHEUS_SERVER_HOST', '0.0.0.0').to_s
|
90
|
-
self.server_port = ENV.fetch('PROMETHEUS_SERVER_PORT', PrometheusExporter::DEFAULT_PORT).to_i
|
91
107
|
|
92
|
-
self.puma_process_label = ENV.fetch('PROMETHEUS_PUMA_PROCESS_LABEL', 'web').to_s
|
93
|
-
self.puma_collection_frequency = ENV.fetch('PROMETHEUS_PUMA_COLLECTION_FREQUENCY', 30).to_i
|
94
108
|
self.web_type_collectors = []
|
95
109
|
end
|
96
110
|
|
@@ -109,8 +123,8 @@ module Bigcommerce
|
|
109
123
|
self.logger = Application.logger
|
110
124
|
else
|
111
125
|
require 'logger'
|
112
|
-
self.logger = ::Logger.new(
|
113
|
-
|
126
|
+
self.logger = ::Logger.new($stdout)
|
127
|
+
logger.level = ::Logger::Severity::INFO
|
114
128
|
end
|
115
129
|
end
|
116
130
|
|
@@ -31,6 +31,8 @@ module Bigcommerce
|
|
31
31
|
@server_port = Bigcommerce::Prometheus.server_port
|
32
32
|
@server_timeout = Bigcommerce::Prometheus.server_timeout
|
33
33
|
@server_prefix = Bigcommerce::Prometheus.server_prefix
|
34
|
+
@collectors = Bigcommerce::Prometheus.hutch_collectors || []
|
35
|
+
@type_collectors = Bigcommerce::Prometheus.hutch_type_collectors || []
|
34
36
|
end
|
35
37
|
|
36
38
|
##
|
@@ -44,6 +46,9 @@ module Bigcommerce
|
|
44
46
|
|
45
47
|
server.add_type_collector(PrometheusExporter::Server::ActiveRecordCollector.new)
|
46
48
|
server.add_type_collector(PrometheusExporter::Server::HutchCollector.new)
|
49
|
+
@type_collectors.each do |tc|
|
50
|
+
server.add_type_collector(tc)
|
51
|
+
end
|
47
52
|
server.start
|
48
53
|
setup_middleware
|
49
54
|
rescue StandardError => e
|
@@ -65,6 +70,9 @@ module Bigcommerce
|
|
65
70
|
require 'hutch'
|
66
71
|
::Hutch::Config.set(:tracer, PrometheusExporter::Instrumentation::Hutch)
|
67
72
|
@app.middleware.unshift(PrometheusExporter::Middleware, client: Bigcommerce::Prometheus.client)
|
73
|
+
@collectors.each(&:start)
|
74
|
+
rescue StandardError => e
|
75
|
+
logger.warn "[bigcommerce-prometheus][#{@process_name}] Failed to setup hutch prometheus middleware: #{e.message}"
|
68
76
|
end
|
69
77
|
end
|
70
78
|
end
|
@@ -31,6 +31,8 @@ module Bigcommerce
|
|
31
31
|
@server_port = Bigcommerce::Prometheus.server_port
|
32
32
|
@server_timeout = Bigcommerce::Prometheus.server_timeout
|
33
33
|
@server_prefix = Bigcommerce::Prometheus.server_prefix
|
34
|
+
@collectors = Bigcommerce::Prometheus.resque_collectors || []
|
35
|
+
@type_collectors = Bigcommerce::Prometheus.resque_type_collectors || []
|
34
36
|
end
|
35
37
|
|
36
38
|
##
|
@@ -44,6 +46,9 @@ module Bigcommerce
|
|
44
46
|
|
45
47
|
server.add_type_collector(PrometheusExporter::Server::ActiveRecordCollector.new)
|
46
48
|
server.add_type_collector(Bigcommerce::Prometheus::TypeCollectors::Resque.new)
|
49
|
+
@type_collectors.each do |tc|
|
50
|
+
server.add_type_collector(tc)
|
51
|
+
end
|
47
52
|
server.start
|
48
53
|
setup_middleware
|
49
54
|
rescue StandardError => e
|
@@ -63,7 +68,8 @@ module Bigcommerce
|
|
63
68
|
def setup_middleware
|
64
69
|
logger.info "[bigcommerce-prometheus][#{@process_name}] Setting up resque prometheus middleware"
|
65
70
|
::Resque.before_first_fork do
|
66
|
-
::Bigcommerce::Prometheus::Integrations::Resque.start
|
71
|
+
::Bigcommerce::Prometheus::Integrations::Resque.start(client: Bigcommerce::Prometheus.client)
|
72
|
+
@collectors.each(&:start)
|
67
73
|
end
|
68
74
|
end
|
69
75
|
end
|
@@ -77,7 +77,7 @@ module Bigcommerce
|
|
77
77
|
def setup_after_fork
|
78
78
|
@app.config.after_fork_callbacks = [] unless @app.config.after_fork_callbacks
|
79
79
|
@app.config.after_fork_callbacks << lambda do
|
80
|
-
::Bigcommerce::Prometheus::Integrations::Puma.start
|
80
|
+
::Bigcommerce::Prometheus::Integrations::Puma.start(client: Bigcommerce::Prometheus.client)
|
81
81
|
@collectors.each(&:start)
|
82
82
|
end
|
83
83
|
end
|
@@ -23,7 +23,7 @@ module Bigcommerce
|
|
23
23
|
#
|
24
24
|
class Railtie < ::Rails::Railtie
|
25
25
|
initializer 'zzz.bc_prometheus_ruby.configure_rails_initialization' do |app|
|
26
|
-
Bigcommerce::Prometheus::Instrumentors::Web.new(app: app).start
|
26
|
+
Bigcommerce::Prometheus::Instrumentors::Web.new(app: app).start unless ::Bigcommerce::Prometheus.railtie_disabled
|
27
27
|
end
|
28
28
|
end
|
29
29
|
end
|
@@ -25,13 +25,13 @@ module Bigcommerce
|
|
25
25
|
##
|
26
26
|
# Start the resque integration
|
27
27
|
#
|
28
|
-
def self.start
|
28
|
+
def self.start(client: nil)
|
29
29
|
::PrometheusExporter::Instrumentation::Process.start(
|
30
|
-
client: ::Bigcommerce::Prometheus.client,
|
30
|
+
client: client || ::Bigcommerce::Prometheus.client,
|
31
31
|
type: ::Bigcommerce::Prometheus.resque_process_label
|
32
32
|
)
|
33
33
|
::Bigcommerce::Prometheus::Collectors::Resque.start(
|
34
|
-
client: ::Bigcommerce::Prometheus.client,
|
34
|
+
client: client || ::Bigcommerce::Prometheus.client,
|
35
35
|
frequency: ::Bigcommerce::Prometheus.resque_collection_frequency
|
36
36
|
)
|
37
37
|
end
|
@@ -26,15 +26,21 @@ module Bigcommerce
|
|
26
26
|
# @param [Integer] port
|
27
27
|
# @param [Integer] timeout
|
28
28
|
# @param [String] prefix
|
29
|
+
# @param [Integer] thread_pool_size
|
29
30
|
#
|
30
|
-
def initialize(host: nil, port: nil, timeout: nil, prefix: nil, logger: nil)
|
31
|
+
def initialize(host: nil, port: nil, timeout: nil, prefix: nil, logger: nil, thread_pool_size: nil)
|
31
32
|
@host = host || ::Bigcommerce::Prometheus.server_host
|
32
33
|
@port = (port || ::Bigcommerce::Prometheus.server_port).to_i
|
33
34
|
@timeout = (timeout || ::Bigcommerce::Prometheus.server_timeout).to_i
|
34
35
|
@prefix = (prefix || ::PrometheusExporter::DEFAULT_PREFIX).to_s
|
35
36
|
@process_name = ::Bigcommerce::Prometheus.process_name
|
36
37
|
@logger = logger || ::Bigcommerce::Prometheus.logger
|
37
|
-
@server = ::Bigcommerce::Prometheus::Servers::Thin::Server.new(
|
38
|
+
@server = ::Bigcommerce::Prometheus::Servers::Thin::Server.new(
|
39
|
+
port: @port,
|
40
|
+
timeout: @timeout,
|
41
|
+
logger: @logger,
|
42
|
+
thread_pool_size: (thread_pool_size || ::Bigcommerce::Prometheus.server_thread_pool_size).to_i
|
43
|
+
)
|
38
44
|
@running = false
|
39
45
|
::PrometheusExporter::Metric::Base.default_prefix = @prefix
|
40
46
|
setup_signal_handlers
|
@@ -51,7 +57,7 @@ module Bigcommerce
|
|
51
57
|
end
|
52
58
|
@running = true
|
53
59
|
|
54
|
-
@logger.info "[bigcommerce-prometheus][#{@process_name}] Prometheus exporter started on #{@host}:#{@port}"
|
60
|
+
@logger.info "[bigcommerce-prometheus][#{@process_name}] Prometheus exporter started on #{@host}:#{@port} with #{@server.threadpool_size} threads"
|
55
61
|
|
56
62
|
@server
|
57
63
|
rescue ::StandardError => e
|
@@ -78,7 +84,7 @@ module Bigcommerce
|
|
78
84
|
@server.stop!
|
79
85
|
@run_thread.kill
|
80
86
|
@running = false
|
81
|
-
|
87
|
+
$stdout.puts "[bigcommerce-prometheus][#{@process_name}] Prometheus exporter cleanly shut down"
|
82
88
|
rescue ::StandardError => e
|
83
89
|
warn "[bigcommerce-prometheus][#{@process_name}] Failed to stop exporter: #{e.message}"
|
84
90
|
end
|
@@ -26,7 +26,7 @@ module Bigcommerce
|
|
26
26
|
class NotFoundController < BaseController
|
27
27
|
def call
|
28
28
|
@response.status = 404
|
29
|
-
@response.body <<
|
29
|
+
@response.body << Bigcommerce::Prometheus.not_found_body
|
30
30
|
end
|
31
31
|
end
|
32
32
|
end
|
@@ -23,14 +23,15 @@ module Bigcommerce
|
|
23
23
|
# Thin adapter for server
|
24
24
|
#
|
25
25
|
class Server < ::Thin::Server
|
26
|
-
def initialize(port
|
27
|
-
@port = port || ::
|
28
|
-
@host = host ||
|
29
|
-
@timeout = timeout || ::
|
26
|
+
def initialize(port: nil, host: nil, timeout: nil, logger: nil, thread_pool_size: nil)
|
27
|
+
@port = port || ::Bigcommerce::Prometheus.server_port
|
28
|
+
@host = host || ::Bigcommerce::Prometheus.server_host
|
29
|
+
@timeout = timeout || ::Bigcommerce::Prometheus.server_timeout
|
30
30
|
@logger = logger || ::Bigcommerce::Prometheus.logger
|
31
31
|
@rack_app = ::Bigcommerce::Prometheus::Servers::Thin::RackApp.new(timeout: timeout, logger: logger)
|
32
32
|
super(@host, @port, @rack_app)
|
33
33
|
::Thin::Logging.logger = @logger
|
34
|
+
self.threadpool_size = (thread_pool_size || ::Bigcommerce::Prometheus.server_thread_pool_size).to_i
|
34
35
|
end
|
35
36
|
|
36
37
|
##
|
@@ -35,7 +35,7 @@ module Bigcommerce
|
|
35
35
|
@metrics_total = ::PrometheusExporter::Metric::Counter.new('collector_metrics_total', 'Total metrics processed by exporter.')
|
36
36
|
@sessions_total = ::PrometheusExporter::Metric::Counter.new('collector_sessions_total', 'Total send_metric sessions processed by exporter.')
|
37
37
|
@bad_metrics_total = ::PrometheusExporter::Metric::Counter.new('collector_bad_metrics_total', 'Total mis-handled metrics by collector.')
|
38
|
-
@collector_working_gauge = ::PrometheusExporter::Metric::Gauge.new('collector_working', 'Is the
|
38
|
+
@collector_working_gauge = ::PrometheusExporter::Metric::Gauge.new('collector_working', 'Is the main process collector able to collect metrics')
|
39
39
|
@collector_rss_gauge = ::PrometheusExporter::Metric::Gauge.new('collector_rss', 'total memory used by collector process')
|
40
40
|
end
|
41
41
|
|
@@ -29,6 +29,7 @@ module Bigcommerce
|
|
29
29
|
# @param [Hash] default_labels
|
30
30
|
#
|
31
31
|
def initialize(type: nil, default_labels: {})
|
32
|
+
super()
|
32
33
|
@type = type || self.class.to_s.downcase.gsub('::', '_').gsub('typecollector', '')
|
33
34
|
@default_labels = default_labels || {}
|
34
35
|
@metrics = build_metrics
|
metadata
CHANGED
@@ -1,113 +1,113 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: bc-prometheus-ruby
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.5.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Shaun McCormick
|
8
|
-
autorequire:
|
8
|
+
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2021-04-15 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
|
-
name:
|
14
|
+
name: bundler-audit
|
15
15
|
requirement: !ruby/object:Gem::Requirement
|
16
16
|
requirements:
|
17
17
|
- - ">="
|
18
18
|
- !ruby/object:Gem::Version
|
19
|
-
version: '
|
19
|
+
version: '0.6'
|
20
20
|
type: :development
|
21
21
|
prerelease: false
|
22
22
|
version_requirements: !ruby/object:Gem::Requirement
|
23
23
|
requirements:
|
24
24
|
- - ">="
|
25
25
|
- !ruby/object:Gem::Version
|
26
|
-
version: '
|
26
|
+
version: '0.6'
|
27
27
|
- !ruby/object:Gem::Dependency
|
28
|
-
name:
|
28
|
+
name: null-logger
|
29
29
|
requirement: !ruby/object:Gem::Requirement
|
30
30
|
requirements:
|
31
31
|
- - ">="
|
32
32
|
- !ruby/object:Gem::Version
|
33
|
-
version: '
|
33
|
+
version: '0.1'
|
34
34
|
type: :development
|
35
35
|
prerelease: false
|
36
36
|
version_requirements: !ruby/object:Gem::Requirement
|
37
37
|
requirements:
|
38
38
|
- - ">="
|
39
39
|
- !ruby/object:Gem::Version
|
40
|
-
version: '
|
40
|
+
version: '0.1'
|
41
41
|
- !ruby/object:Gem::Dependency
|
42
|
-
name:
|
42
|
+
name: pry
|
43
43
|
requirement: !ruby/object:Gem::Requirement
|
44
44
|
requirements:
|
45
45
|
- - ">="
|
46
46
|
- !ruby/object:Gem::Version
|
47
|
-
version: '0.
|
47
|
+
version: '0.12'
|
48
48
|
type: :development
|
49
49
|
prerelease: false
|
50
50
|
version_requirements: !ruby/object:Gem::Requirement
|
51
51
|
requirements:
|
52
52
|
- - ">="
|
53
53
|
- !ruby/object:Gem::Version
|
54
|
-
version: '0.
|
54
|
+
version: '0.12'
|
55
55
|
- !ruby/object:Gem::Dependency
|
56
|
-
name:
|
56
|
+
name: rake
|
57
57
|
requirement: !ruby/object:Gem::Requirement
|
58
58
|
requirements:
|
59
59
|
- - ">="
|
60
60
|
- !ruby/object:Gem::Version
|
61
|
-
version: '0
|
61
|
+
version: '10.0'
|
62
62
|
type: :development
|
63
63
|
prerelease: false
|
64
64
|
version_requirements: !ruby/object:Gem::Requirement
|
65
65
|
requirements:
|
66
66
|
- - ">="
|
67
67
|
- !ruby/object:Gem::Version
|
68
|
-
version: '0
|
68
|
+
version: '10.0'
|
69
69
|
- !ruby/object:Gem::Dependency
|
70
|
-
name:
|
70
|
+
name: rspec
|
71
71
|
requirement: !ruby/object:Gem::Requirement
|
72
72
|
requirements:
|
73
73
|
- - ">="
|
74
74
|
- !ruby/object:Gem::Version
|
75
|
-
version: '
|
75
|
+
version: '3.8'
|
76
76
|
type: :development
|
77
77
|
prerelease: false
|
78
78
|
version_requirements: !ruby/object:Gem::Requirement
|
79
79
|
requirements:
|
80
80
|
- - ">="
|
81
81
|
- !ruby/object:Gem::Version
|
82
|
-
version: '
|
82
|
+
version: '3.8'
|
83
83
|
- !ruby/object:Gem::Dependency
|
84
|
-
name:
|
84
|
+
name: rspec_junit_formatter
|
85
85
|
requirement: !ruby/object:Gem::Requirement
|
86
86
|
requirements:
|
87
87
|
- - ">="
|
88
88
|
- !ruby/object:Gem::Version
|
89
|
-
version: '0.
|
89
|
+
version: '0.4'
|
90
90
|
type: :development
|
91
91
|
prerelease: false
|
92
92
|
version_requirements: !ruby/object:Gem::Requirement
|
93
93
|
requirements:
|
94
94
|
- - ">="
|
95
95
|
- !ruby/object:Gem::Version
|
96
|
-
version: '0.
|
96
|
+
version: '0.4'
|
97
97
|
- !ruby/object:Gem::Dependency
|
98
98
|
name: rubocop
|
99
99
|
requirement: !ruby/object:Gem::Requirement
|
100
100
|
requirements:
|
101
101
|
- - ">="
|
102
102
|
- !ruby/object:Gem::Version
|
103
|
-
version: '0
|
103
|
+
version: '1.0'
|
104
104
|
type: :development
|
105
105
|
prerelease: false
|
106
106
|
version_requirements: !ruby/object:Gem::Requirement
|
107
107
|
requirements:
|
108
108
|
- - ">="
|
109
109
|
- !ruby/object:Gem::Version
|
110
|
-
version: '0
|
110
|
+
version: '1.0'
|
111
111
|
- !ruby/object:Gem::Dependency
|
112
112
|
name: simplecov
|
113
113
|
requirement: !ruby/object:Gem::Requirement
|
@@ -142,14 +142,14 @@ dependencies:
|
|
142
142
|
requirements:
|
143
143
|
- - "~>"
|
144
144
|
- !ruby/object:Gem::Version
|
145
|
-
version: '0.
|
145
|
+
version: '0.7'
|
146
146
|
type: :runtime
|
147
147
|
prerelease: false
|
148
148
|
version_requirements: !ruby/object:Gem::Requirement
|
149
149
|
requirements:
|
150
150
|
- - "~>"
|
151
151
|
- !ruby/object:Gem::Version
|
152
|
-
version: '0.
|
152
|
+
version: '0.7'
|
153
153
|
- !ruby/object:Gem::Dependency
|
154
154
|
name: thin
|
155
155
|
requirement: !ruby/object:Gem::Requirement
|
@@ -203,7 +203,7 @@ homepage: https://github.com/bigcommerce/bc-prometheus-ruby
|
|
203
203
|
licenses:
|
204
204
|
- MIT
|
205
205
|
metadata: {}
|
206
|
-
post_install_message:
|
206
|
+
post_install_message:
|
207
207
|
rdoc_options: []
|
208
208
|
require_paths:
|
209
209
|
- lib
|
@@ -211,15 +211,15 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
211
211
|
requirements:
|
212
212
|
- - ">="
|
213
213
|
- !ruby/object:Gem::Version
|
214
|
-
version: '
|
214
|
+
version: '2.6'
|
215
215
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
216
216
|
requirements:
|
217
217
|
- - ">="
|
218
218
|
- !ruby/object:Gem::Version
|
219
219
|
version: '0'
|
220
220
|
requirements: []
|
221
|
-
rubygems_version: 3.
|
222
|
-
signing_key:
|
221
|
+
rubygems_version: 3.2.3
|
222
|
+
signing_key:
|
223
223
|
specification_version: 4
|
224
224
|
summary: Simple integration of ruby and puma servers with prometheus
|
225
225
|
test_files: []
|