invoca-metrics 0.0.2 → 1.14.1
Sign up to get free protection for your applications and to get access to all the features.
Potentially problematic release.
This version of invoca-metrics might be problematic. Click here for more details.
- checksums.yaml +4 -4
- data/LICENSE.txt +22 -0
- data/README.md +161 -0
- data/invoca-metrics.gemspec +34 -0
- data/lib/invoca/metrics/batch.rb +38 -0
- data/lib/invoca/metrics/client.rb +171 -0
- data/lib/invoca/metrics/direct_metric.rb +68 -0
- data/lib/invoca/metrics/gauge_cache.rb +80 -0
- data/lib/invoca/metrics/prometheus/configuration.rb +109 -0
- data/lib/invoca/metrics/prometheus/declare_metrics/base.rb +52 -0
- data/lib/invoca/metrics/prometheus/declare_metrics/counter.rb +26 -0
- data/lib/invoca/metrics/prometheus/declare_metrics/dsl.rb +61 -0
- data/lib/invoca/metrics/prometheus/declare_metrics/gauge.rb +32 -0
- data/lib/invoca/metrics/prometheus/declare_metrics/histogram.rb +50 -0
- data/lib/invoca/metrics/prometheus/metrics_registry.rb +43 -0
- data/lib/invoca/metrics/prometheus.rb +67 -0
- data/lib/invoca/metrics/statsd_client.rb +49 -0
- data/lib/invoca/metrics/version.rb +7 -0
- data/lib/invoca/metrics.rb +117 -0
- metadata +89 -8
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 90b8517fa182fbf84e8067d0f4727aac492ea493804f5913cbd0495a3c1a1a88
|
4
|
+
data.tar.gz: f694ecc164dc329381989fbce06d5ea91fb416707d54ca9a571eeda286c3ab52
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: db2b4c7f76eb190ce6e2ff8231f5eb1f39103fde83e314d4eb40d18c1b01a73820e0b253be0969bdcbf179ea8cbdb4466845182c952ee1a159f736486bd041ed
|
7
|
+
data.tar.gz: b48fd3176d18a7957c6cf98e88513044c275195f8ab93036192f0599b9c338d84c615b94ad8465f468318f4513ecd42fca381a2ef180d17541c09b352c689a2e
|
data/LICENSE.txt
ADDED
@@ -0,0 +1,22 @@
|
|
1
|
+
Copyright (c) 2014 Cary Penniman
|
2
|
+
|
3
|
+
MIT License
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
6
|
+
a copy of this software and associated documentation files (the
|
7
|
+
"Software"), to deal in the Software without restriction, including
|
8
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
9
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
10
|
+
permit persons to whom the Software is furnished to do so, subject to
|
11
|
+
the following conditions:
|
12
|
+
|
13
|
+
The above copyright notice and this permission notice shall be
|
14
|
+
included in all copies or substantial portions of the Software.
|
15
|
+
|
16
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
17
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
18
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
19
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
20
|
+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
21
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
22
|
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,161 @@
|
|
1
|
+
# Invoca::Metrics
|
2
|
+
|
3
|
+
Publish metrics for observability!
|
4
|
+
|
5
|
+
## Dependencies
|
6
|
+
* Ruby >= 2.6
|
7
|
+
* ActiveSupport >= 4.2, < 7
|
8
|
+
|
9
|
+
## Usage
|
10
|
+
|
11
|
+
Mix in the `Invoca::Metrics::Source` module:
|
12
|
+
```ruby
|
13
|
+
class MyClass
|
14
|
+
include Invoca::Metrics::Source
|
15
|
+
...
|
16
|
+
end
|
17
|
+
```
|
18
|
+
|
19
|
+
Then declare the metrics that can be published by the class using a `declare_metrics` DSL block.
|
20
|
+
Each entry in the block starts with the metric type as the DSL method:
|
21
|
+
|
22
|
+
| Metric Type | Value | Metric Instance Methods |
|
23
|
+
|:-----------:|:----------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
24
|
+
| `counter` | monotonically incrementing floating point | `.increment(value = 1, **labels)` |
|
25
|
+
| `gauge` | arbitrary floating point | `.set(value, **labels)` |
|
26
|
+
| `histogram` | histogram counters with configurable bucket cutoffs | `.add(value, **labels)`<br/><br/>`.time(return_timing: false, **labels, &block)`<br/>yields to block and `add()`s the milliseconds spent to the histogram; returns either `block_value` or `[milliseconds, block_value]` depending on the `return_timing` argument |
|
27
|
+
|
28
|
+
After the metric type, the first argument is the metric name, given as a symbol. This is globally unique.
|
29
|
+
Note that other classes in the process are allowed to declare the same metric, as long as the type and labels are the same.
|
30
|
+
|
31
|
+
Each metric may have any number of labels declared in the optional `labels:` hash, with default values in case that label is not passed when the metric is published.
|
32
|
+
|
33
|
+
Note: An implicit label of `service: <service_name>` is always added. The `<service_name>` is a configured value.
|
34
|
+
|
35
|
+
Histograms have a required argument, `buckets: <array>`, where the array is the histogram bucket cutoffs.
|
36
|
+
|
37
|
+
```ruby
|
38
|
+
class MyClass
|
39
|
+
include Invoca::Metrics::Source
|
40
|
+
|
41
|
+
declare_metrics do
|
42
|
+
counter :request_total, labels: { status: '200' }
|
43
|
+
gauge :available_memory
|
44
|
+
histogram :processing_time, buckets: [30.seconds, 1.hour, 2.hours, 4.hours, 5.hours, 6.hours, 10.hours]
|
45
|
+
end
|
46
|
+
|
47
|
+
...
|
48
|
+
end
|
49
|
+
```
|
50
|
+
|
51
|
+
After declaring the metrics, you may use `prometheus_metrics.<metric_name>` (in either the class or instance) to publish the metric.
|
52
|
+
```ruby
|
53
|
+
prometheus_metrics.request_total.increment # increment the counter by 1
|
54
|
+
|
55
|
+
prometheus_metrics.request_total.increment(5) # increment the counter by 5
|
56
|
+
|
57
|
+
prometheus_metrics.available_memory.set(10.25) # set the gauge to 10.25
|
58
|
+
|
59
|
+
prometheus_metrics.processing_time.add(100) # Increment the bucket closest to the value 100
|
60
|
+
prometheus_metrics.processing_time.time { work } # measure the elapsed milliseconds to run the given block and increment the bucket closest to that time period in milliseconds
|
61
|
+
```
|
62
|
+
|
63
|
+
## Installation
|
64
|
+
|
65
|
+
Add this line to your application's Gemfile:
|
66
|
+
|
67
|
+
gem 'invoca-metrics'
|
68
|
+
|
69
|
+
And then execute:
|
70
|
+
|
71
|
+
$ bundle
|
72
|
+
|
73
|
+
Or install it yourself as:
|
74
|
+
|
75
|
+
$ gem install invoca-metrics
|
76
|
+
|
77
|
+
## Setup
|
78
|
+
|
79
|
+
### Default Setup
|
80
|
+
|
81
|
+
Add the following code to your application...
|
82
|
+
|
83
|
+
require 'invoca/metrics'
|
84
|
+
|
85
|
+
Invoca::Metrics.service_name = "tts-service"
|
86
|
+
Invoca::Metrics.server_name = Socket.gethostname
|
87
|
+
Invoca::Metrics.cluster_name = "production"
|
88
|
+
Invoca::Metrics.sub_server_name = "worker_process_1"
|
89
|
+
Invoca::Metrics.statsd_host = "255.0.0.123"
|
90
|
+
Invoca::Metrics.statsd_port = 200
|
91
|
+
|
92
|
+
Invoca::Metrics::Client.logger = logger
|
93
|
+
Invoca::Metrics::Client.log_send_failures = ['staging', 'production'].include(environment)
|
94
|
+
|
95
|
+
Note: the `service_name` setting is required; all others are optional.
|
96
|
+
|
97
|
+
### Multi Configuration
|
98
|
+
|
99
|
+
To set multiple configurations, supply a `config` hash with additional settings.
|
100
|
+
|
101
|
+
Invoca::Metrics.config = {
|
102
|
+
deployment_group: {
|
103
|
+
server_name: "deployment"
|
104
|
+
statsd_host: "255.0.0.456",
|
105
|
+
statsd_port: 300
|
106
|
+
},
|
107
|
+
region: {
|
108
|
+
server_name: "region_name"
|
109
|
+
statsd_host: "255.0.0.789",
|
110
|
+
statsd_port: 400,
|
111
|
+
sub_server_name: nil
|
112
|
+
cluster_name: nil
|
113
|
+
}
|
114
|
+
}
|
115
|
+
|
116
|
+
The settings (`[service_name, server_name, cluster_name, sub_server_name, statsd_host, statsd_port]`) directly set on `Invoca::Metrics` will be the default values supplied if the individual configuration does not supply the option.
|
117
|
+
It's highly suggested that each configuration has its own `statsd_host` and `statsd_port` along with unique naming since writing the same metric from one statsd node could be overwritten by the same metric from a separate node.
|
118
|
+
|
119
|
+
In order to set a configuration as the default, set the configuration key as `default_config_key`.
|
120
|
+
|
121
|
+
Invoca::Metrics.default_config_key = :deployment_group
|
122
|
+
|
123
|
+
Any keys missing from the `default_config_key` config will by default have the values from the keys set directly on `Invoca::Metrics`.
|
124
|
+
|
125
|
+
The full set of default values for the above example would then be
|
126
|
+
|
127
|
+
service_name: "my_event_worker"
|
128
|
+
cluster_name: "production"
|
129
|
+
sub_server_name: "worker_process_1"
|
130
|
+
server_name: "deployment"
|
131
|
+
statsd_host: "255.0.0.456",
|
132
|
+
statsd_port: 300
|
133
|
+
|
134
|
+
### Prometheus Metric Exporting Setup
|
135
|
+
There are 2 ways to set up metrics for being scraped by Prometheus: `:direct` and `:push`).
|
136
|
+
`:direct` will expose a web `server directly from the running process in order for metrics to be scraped while `:push` will push metrics over to an aggregate gateway which in turn implements a web server.
|
137
|
+
|
138
|
+
#### `:direct` Configuration Example
|
139
|
+
```ruby
|
140
|
+
Invoca::Metrics::Prometheus.configure do |config|
|
141
|
+
config.mode = :direct
|
142
|
+
config.port = 9394 # Port to expose metrics from
|
143
|
+
end
|
144
|
+
```
|
145
|
+
|
146
|
+
#### `:push` Configuration
|
147
|
+
```ruby
|
148
|
+
Invoca::Metrics::Prometheus.configure do |config|
|
149
|
+
config.mode = :push
|
150
|
+
config.host = '127.0.0.1' # IP address to push metrics to
|
151
|
+
config.port = 9394 # Port to push metrics to
|
152
|
+
end
|
153
|
+
```
|
154
|
+
|
155
|
+
## Contributing
|
156
|
+
|
157
|
+
1. Fork it
|
158
|
+
2. Create your feature branch (`git checkout -b my-new-feature`)
|
159
|
+
3. Commit your changes (`git commit -am 'Add some feature'`)
|
160
|
+
4. Push to the branch (`git push origin my-new-feature`)
|
161
|
+
5. Create new Pull Request
|
@@ -0,0 +1,34 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
lib = File.expand_path('lib', __dir__)
|
4
|
+
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
|
5
|
+
require 'invoca/metrics/version'
|
6
|
+
|
7
|
+
Gem::Specification.new do |spec|
|
8
|
+
spec.name = "invoca-metrics"
|
9
|
+
spec.version = Invoca::Metrics::VERSION
|
10
|
+
spec.authors = ["Invoca development"]
|
11
|
+
spec.email = ["development@invoca.com"]
|
12
|
+
spec.description = 'Invoca metrics reporting library'
|
13
|
+
spec.summary = 'Invoca metrics reporting library'
|
14
|
+
spec.homepage = "https://github.com/Invoca/invoca-metrics"
|
15
|
+
spec.license = "MIT"
|
16
|
+
|
17
|
+
spec.metadata['allowed_push_host'] = "https://rubygems.org"
|
18
|
+
|
19
|
+
spec.files = [
|
20
|
+
*Dir.glob('lib/**/*.rb'),
|
21
|
+
'README.md',
|
22
|
+
'LICENSE.txt',
|
23
|
+
'invoca-metrics.gemspec',
|
24
|
+
]
|
25
|
+
|
26
|
+
spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
|
27
|
+
spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
|
28
|
+
spec.require_paths = ["lib"]
|
29
|
+
|
30
|
+
spec.add_dependency "activesupport", ">= 4.2", "< 7"
|
31
|
+
spec.add_dependency "statsd-ruby", "~> 1.2.1"
|
32
|
+
spec.add_dependency "prometheus_exporter", "~> 2.0"
|
33
|
+
spec.add_dependency "contextual_logger", "~> 1.0"
|
34
|
+
end
|
@@ -0,0 +1,38 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'active_support'
|
4
|
+
require 'active_support/core_ext/module/delegation'
|
5
|
+
|
6
|
+
module Invoca
|
7
|
+
module Metrics
|
8
|
+
class Batch < Client
|
9
|
+
delegate :batch_size, :batch_size=, to: :statsd_client
|
10
|
+
|
11
|
+
# @param [Invoca::Metrics::Client] client requires a configured Client instance
|
12
|
+
# @param [Statsd::Batch] statsd_batch requires a configured Batch instance
|
13
|
+
def initialize(client, statsd_batch)
|
14
|
+
super(
|
15
|
+
hostname: client.hostname,
|
16
|
+
port: client.port,
|
17
|
+
cluster_name: client.cluster_name,
|
18
|
+
service_name: client.service_name,
|
19
|
+
server_label: client.server_label,
|
20
|
+
sub_server_name: client.sub_server_name,
|
21
|
+
namespace: client.namespace
|
22
|
+
)
|
23
|
+
@statsd_client = statsd_batch
|
24
|
+
end
|
25
|
+
|
26
|
+
# @yields [Batch] yields itself
|
27
|
+
#
|
28
|
+
# A convenience method to ensure that data is not lost in the event of an
|
29
|
+
# exception being thrown. Batches will be transmitted on the parent socket
|
30
|
+
# as soon as the batch is full, and when the block finishes.
|
31
|
+
def ensure_send
|
32
|
+
yield self
|
33
|
+
ensure
|
34
|
+
statsd_client.flush
|
35
|
+
end
|
36
|
+
end
|
37
|
+
end
|
38
|
+
end
|
@@ -0,0 +1,171 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'active_support'
|
4
|
+
require 'active_support/core_ext/module/delegation'
|
5
|
+
|
6
|
+
module Invoca
|
7
|
+
module Metrics
|
8
|
+
class Client
|
9
|
+
STATSD_DEFAULT_HOSTNAME = "127.0.0.1"
|
10
|
+
STATSD_DEFAULT_PORT = 8125
|
11
|
+
STATSD_METRICS_SEPARATOR = '.'
|
12
|
+
|
13
|
+
class << self
|
14
|
+
delegate :logger, :logger=, :log_send_failures, :log_send_failures=, to: StatsdClient
|
15
|
+
|
16
|
+
# Default values are required for backwards compatibility
|
17
|
+
def metrics(statsd_host: Invoca::Metrics.default_client_config[:statsd_host],
|
18
|
+
statsd_port: Invoca::Metrics.default_client_config[:statsd_port],
|
19
|
+
cluster_name: Invoca::Metrics.default_client_config[:cluster_name],
|
20
|
+
service_name: Invoca::Metrics.default_client_config[:service_name],
|
21
|
+
server_name: Invoca::Metrics.default_client_config[:server_name],
|
22
|
+
sub_server_name: Invoca::Metrics.default_client_config[:sub_server_name],
|
23
|
+
namespace: nil)
|
24
|
+
config = {
|
25
|
+
hostname: statsd_host || STATSD_DEFAULT_HOSTNAME,
|
26
|
+
port: statsd_port || STATSD_DEFAULT_PORT,
|
27
|
+
cluster_name: cluster_name,
|
28
|
+
service_name: service_name,
|
29
|
+
server_label: server_name,
|
30
|
+
sub_server_name: sub_server_name,
|
31
|
+
namespace: namespace
|
32
|
+
}.freeze
|
33
|
+
|
34
|
+
client_cache[config] ||= new(**config)
|
35
|
+
end
|
36
|
+
|
37
|
+
def reset_cache
|
38
|
+
@client_cache = {}
|
39
|
+
end
|
40
|
+
|
41
|
+
private
|
42
|
+
|
43
|
+
def client_cache
|
44
|
+
@client_cache ||= {}
|
45
|
+
end
|
46
|
+
end
|
47
|
+
|
48
|
+
attr_reader :hostname, :port, :server_label, :sub_server_name, :cluster_name, :service_name, :gauge_cache
|
49
|
+
delegate :batch_size, :namespace, :timing, :time, to: :statsd_client
|
50
|
+
|
51
|
+
def initialize(hostname:, port:, cluster_name: nil, service_name: nil, server_label: nil, sub_server_name: nil, namespace: nil)
|
52
|
+
@hostname = hostname
|
53
|
+
@port = port
|
54
|
+
@cluster_name = cluster_name
|
55
|
+
@service_name = service_name
|
56
|
+
@server_label = server_label
|
57
|
+
@sub_server_name = sub_server_name
|
58
|
+
|
59
|
+
@statsd_client = StatsdClient.new(@hostname, @port)
|
60
|
+
@statsd_client.namespace = namespace || [@cluster_name, @service_name].compact.join(STATSD_METRICS_SEPARATOR).presence
|
61
|
+
|
62
|
+
@gauge_cache = GaugeCache.register(gauge_cache_key, @statsd_client)
|
63
|
+
end
|
64
|
+
|
65
|
+
def gauge_cache_key
|
66
|
+
[
|
67
|
+
hostname,
|
68
|
+
port,
|
69
|
+
cluster_name,
|
70
|
+
service_name,
|
71
|
+
namespace,
|
72
|
+
server_name,
|
73
|
+
sub_server_name
|
74
|
+
].freeze
|
75
|
+
end
|
76
|
+
|
77
|
+
def server_name # For backwards compatibility
|
78
|
+
server_label
|
79
|
+
end
|
80
|
+
|
81
|
+
# This will store the gauge value passed in so that it is reported every GAUGE_REPORT_INTERVAL
|
82
|
+
# seconds and post the gauge at the same time to avoid delay in gauges being
|
83
|
+
def gauge(name, value)
|
84
|
+
log_usage(name, :gauge)
|
85
|
+
if (args = normalized_metric_name_and_value(name, value, "gauge"))
|
86
|
+
gauge_cache.set(*args)
|
87
|
+
statsd_client.gauge(*args)
|
88
|
+
end
|
89
|
+
end
|
90
|
+
|
91
|
+
def count(name, value = 1)
|
92
|
+
log_usage(name, :counter)
|
93
|
+
if (args = normalized_metric_name_and_value(name, value, "counter"))
|
94
|
+
statsd_client.count(*args)
|
95
|
+
end
|
96
|
+
end
|
97
|
+
|
98
|
+
alias counter count
|
99
|
+
|
100
|
+
def increment(name)
|
101
|
+
count(name, 1)
|
102
|
+
end
|
103
|
+
|
104
|
+
def decrement(name)
|
105
|
+
count(name, -1)
|
106
|
+
end
|
107
|
+
|
108
|
+
def set(name, value)
|
109
|
+
log_usage(name, :counter)
|
110
|
+
if (args = normalized_metric_name_and_value(name, value, nil))
|
111
|
+
statsd_client.set(*args)
|
112
|
+
end
|
113
|
+
end
|
114
|
+
|
115
|
+
def timer(name, milliseconds = nil, return_timing: false, &block)
|
116
|
+
name.present? or raise ArgumentError, "Must specify a metric name."
|
117
|
+
(!milliseconds.nil? ^ block_given?) or raise ArgumentError, "Must pass exactly one of milliseconds or block."
|
118
|
+
name_and_type = [name, "timer", server_label].join(STATSD_METRICS_SEPARATOR)
|
119
|
+
log_usage(name, :timer)
|
120
|
+
|
121
|
+
if milliseconds.nil?
|
122
|
+
result, block_time = time(name_and_type, &block)
|
123
|
+
return_timing ? [result, block_time] : result
|
124
|
+
else
|
125
|
+
timing(name_and_type, milliseconds)
|
126
|
+
end
|
127
|
+
end
|
128
|
+
|
129
|
+
def batch(&block)
|
130
|
+
statsd_client.batch do |batch|
|
131
|
+
Metrics::Batch.new(self, batch).ensure_send(&block)
|
132
|
+
end
|
133
|
+
end
|
134
|
+
|
135
|
+
# TODO: - implement transmit method
|
136
|
+
def transmit(message, extra_data = {})
|
137
|
+
# TODO: - we need to wire up exception data to a monitoring service
|
138
|
+
end
|
139
|
+
|
140
|
+
private
|
141
|
+
|
142
|
+
attr_reader :statsd_client
|
143
|
+
|
144
|
+
def normalized_metric_name_and_value(name, value, stat_type)
|
145
|
+
name.present? or raise ArgumentError, "Must specify a metric name."
|
146
|
+
extended_name = [name, stat_type, @server_label, @sub_server_name].compact.join(STATSD_METRICS_SEPARATOR)
|
147
|
+
if value
|
148
|
+
[extended_name, value]
|
149
|
+
else
|
150
|
+
[extended_name]
|
151
|
+
end
|
152
|
+
end
|
153
|
+
|
154
|
+
def log_usage(metric_name, metric_type)
|
155
|
+
if Invoca::Metrics.graphite_usage_logging_enabled
|
156
|
+
call_stack = caller_locations
|
157
|
+
unless call_stack.any? { |l| /invoca\/metrics\/prometheus/.match?(l.to_s) }
|
158
|
+
self.class.logger&.info(
|
159
|
+
"Deprecated usage of grapihite metrics",
|
160
|
+
invoca_metrics: {
|
161
|
+
metric_name: metric_name,
|
162
|
+
metric_type: metric_type,
|
163
|
+
metric_source: call_stack.find { |l| !/invoca\/metrics/.match?(l.to_s) }.to_s
|
164
|
+
}
|
165
|
+
)
|
166
|
+
end
|
167
|
+
end
|
168
|
+
end
|
169
|
+
end
|
170
|
+
end
|
171
|
+
end
|
@@ -0,0 +1,68 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Invoca
|
4
|
+
module Metrics
|
5
|
+
# Directly reports metrics without sending through graphite. Does not add process information to metric names.
|
6
|
+
class DirectMetric
|
7
|
+
attr_reader :name, :value, :tick
|
8
|
+
|
9
|
+
def initialize(name, value, tick = nil)
|
10
|
+
@name = name
|
11
|
+
@value = value
|
12
|
+
@tick = tick || self.class.rounded_tick
|
13
|
+
end
|
14
|
+
|
15
|
+
def to_s
|
16
|
+
"#{name} #{value} #{tick}"
|
17
|
+
end
|
18
|
+
|
19
|
+
PERIOD = 60
|
20
|
+
DEFAULT_PORT = 2003
|
21
|
+
DEFAULT_HOST = '127.0.0.1'
|
22
|
+
|
23
|
+
class << self
|
24
|
+
def report(metrics)
|
25
|
+
metrics_output = Array(metrics).map { |metric| "#{metric}\n" }.join
|
26
|
+
|
27
|
+
send_to_metric_host(metrics_output)
|
28
|
+
end
|
29
|
+
|
30
|
+
def generate_distribution(metric_prefix, metric_values, tick = nil)
|
31
|
+
fixed_tick = tick || rounded_tick
|
32
|
+
sorted_values = metric_values.sort
|
33
|
+
count = sorted_values.count
|
34
|
+
|
35
|
+
if count == 0
|
36
|
+
[
|
37
|
+
new("#{metric_prefix}.count", count, fixed_tick)
|
38
|
+
]
|
39
|
+
else
|
40
|
+
[
|
41
|
+
new("#{metric_prefix}.count", count, fixed_tick),
|
42
|
+
new("#{metric_prefix}.max", sorted_values[-1], fixed_tick),
|
43
|
+
new("#{metric_prefix}.min", sorted_values[0], fixed_tick),
|
44
|
+
new("#{metric_prefix}.median", sorted_values[count * 0.5], fixed_tick),
|
45
|
+
new("#{metric_prefix}.upper_90", sorted_values[count * 0.9], fixed_tick)
|
46
|
+
]
|
47
|
+
end
|
48
|
+
end
|
49
|
+
|
50
|
+
def rounded_tick
|
51
|
+
tick = Time.now.to_i
|
52
|
+
tick - (tick % PERIOD)
|
53
|
+
end
|
54
|
+
|
55
|
+
private
|
56
|
+
|
57
|
+
def send_to_metric_host(message)
|
58
|
+
host = ENV["DIRECT_METRIC_HOST"] || DEFAULT_HOST
|
59
|
+
port = (ENV["DIRECT_METRIC_PORT"] || DEFAULT_PORT).to_i
|
60
|
+
|
61
|
+
TCPSocket.open(host, port) do |tcp|
|
62
|
+
tcp.send(message, 0)
|
63
|
+
end
|
64
|
+
end
|
65
|
+
end
|
66
|
+
end
|
67
|
+
end
|
68
|
+
end
|
@@ -0,0 +1,80 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Invoca
|
4
|
+
module Metrics
|
5
|
+
class GaugeCache
|
6
|
+
GAUGE_REPORT_INTERVAL = 60.seconds
|
7
|
+
|
8
|
+
class << self
|
9
|
+
def register(cache_key, statsd_client)
|
10
|
+
registered_gauge_caches[cache_key] ||= new(statsd_client)
|
11
|
+
end
|
12
|
+
|
13
|
+
def reset
|
14
|
+
@registered_gauge_caches = {}
|
15
|
+
end
|
16
|
+
|
17
|
+
private
|
18
|
+
|
19
|
+
def registered_gauge_caches
|
20
|
+
@registered_gauge_caches ||= {}
|
21
|
+
end
|
22
|
+
end
|
23
|
+
|
24
|
+
attr_reader :cache
|
25
|
+
|
26
|
+
def initialize(statsd_client)
|
27
|
+
@statsd_client = statsd_client
|
28
|
+
@cache = {}
|
29
|
+
start_reporting_thread
|
30
|
+
end
|
31
|
+
|
32
|
+
# Atomic method for setting the value for a particular gauge
|
33
|
+
def set(metric, value)
|
34
|
+
@cache[metric] = value
|
35
|
+
end
|
36
|
+
|
37
|
+
# Reports all gauges that have been set in the cache
|
38
|
+
# To avoid "RuntimeError: can't add a new key into hash during iteration" from occurring we are
|
39
|
+
# temporarily duplicating the cache to iterate and send the batch of metrics
|
40
|
+
def report
|
41
|
+
@statsd_client.batch do |statsd_batch|
|
42
|
+
@cache.dup.each do |metric, value|
|
43
|
+
statsd_batch.gauge(metric, value) if value
|
44
|
+
end
|
45
|
+
end
|
46
|
+
end
|
47
|
+
|
48
|
+
def service_environment
|
49
|
+
@service_environment ||= ENV['RAILS_ENV'].presence || ENV['SERVICE_ENV'].presence || 'development'
|
50
|
+
end
|
51
|
+
|
52
|
+
private
|
53
|
+
|
54
|
+
def start_reporting_thread
|
55
|
+
Thread.new do
|
56
|
+
reporting_loop_with_rescue
|
57
|
+
end
|
58
|
+
end
|
59
|
+
|
60
|
+
def reporting_loop_with_rescue
|
61
|
+
reporting_loop
|
62
|
+
rescue Exception => ex
|
63
|
+
Invoca::Metrics::Client.logger.error("GaugeCache#reporting_loop_with_rescue rescued exception:\n#{ex.class}: #{ex.message}")
|
64
|
+
end
|
65
|
+
|
66
|
+
def reporting_loop
|
67
|
+
next_time = Time.now.to_i
|
68
|
+
loop do
|
69
|
+
next_time = (((next_time + GAUGE_REPORT_INTERVAL + 1) / GAUGE_REPORT_INTERVAL) * GAUGE_REPORT_INTERVAL) - 1
|
70
|
+
report
|
71
|
+
if (delay = next_time - Time.now.to_i) > 0
|
72
|
+
sleep(delay)
|
73
|
+
else
|
74
|
+
warn("Window to report gauge may have been missed.") unless service_environment == 'test'
|
75
|
+
end
|
76
|
+
end
|
77
|
+
end
|
78
|
+
end
|
79
|
+
end
|
80
|
+
end
|
@@ -0,0 +1,109 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'prometheus_exporter'
|
4
|
+
require 'prometheus_exporter/server'
|
5
|
+
require 'prometheus_exporter/client'
|
6
|
+
|
7
|
+
module Invoca
|
8
|
+
module Metrics
|
9
|
+
module Prometheus
|
10
|
+
class Configuration
|
11
|
+
AVAILABLE_MODES = [:direct, :push].freeze
|
12
|
+
AVAILABLE_METRIC_TYPES = [:counter, :gauge, :histogram].freeze
|
13
|
+
|
14
|
+
# The mode to use for exporting Prometheus metrics.
|
15
|
+
# Valid modes are:
|
16
|
+
# - `:direct` (default): Exports metrics directly using a threaded webserver.
|
17
|
+
# - `:push`: Exports metrics using a pushgateway.
|
18
|
+
#
|
19
|
+
# @return [Symbol]
|
20
|
+
attr_reader :mode
|
21
|
+
|
22
|
+
# The host to use for the Prometheus server. This setting
|
23
|
+
# is only used when in :push mode. Defaults to `localhost`.
|
24
|
+
#
|
25
|
+
# @return [String]
|
26
|
+
attr_accessor :host
|
27
|
+
|
28
|
+
# The port to use when exporting metrics. When in :direct mode
|
29
|
+
# this is used to specify the port to listen on. When in :push
|
30
|
+
# mode this is used to specify the port to push to. Defaults to 9394.
|
31
|
+
#
|
32
|
+
# @return [Integer]
|
33
|
+
attr_accessor :port
|
34
|
+
|
35
|
+
# The client to use for exporting Prometheus metrics.
|
36
|
+
#
|
37
|
+
# @return [PrometheusExporter::Client]
|
38
|
+
attr_reader :client
|
39
|
+
|
40
|
+
# The webserver that is exposed when in :direct mode.
|
41
|
+
#
|
42
|
+
# @return [PrometheusExporter::Server::WebServer]
|
43
|
+
attr_reader :server
|
44
|
+
|
45
|
+
def initialize
|
46
|
+
@host = 'localhost'
|
47
|
+
@port = 9394
|
48
|
+
end
|
49
|
+
|
50
|
+
def mode=(new_mode)
|
51
|
+
AVAILABLE_MODES.include?(new_mode) or raise ArgumentError, "Invalid mode #{new_mode.inspect}. Allowed modes: #{AVAILABLE_MODES}"
|
52
|
+
@mode = new_mode
|
53
|
+
end
|
54
|
+
|
55
|
+
# Initializes the client to use for exporting Prometheus metrics and
|
56
|
+
# freezes the configuration object so no more changes can be made to it.
|
57
|
+
def freeze
|
58
|
+
validate_configuration!
|
59
|
+
|
60
|
+
case mode
|
61
|
+
when :direct
|
62
|
+
initialize_direct_client
|
63
|
+
when :push
|
64
|
+
initialize_push_client
|
65
|
+
end
|
66
|
+
|
67
|
+
PrometheusExporter::Client.default = @client
|
68
|
+
|
69
|
+
super
|
70
|
+
end
|
71
|
+
|
72
|
+
# Registers a metric against the current Configuration. Once the metric object
|
73
|
+
# is registered, metrics can be reported and exported through the metric object.
|
74
|
+
#
|
75
|
+
# @param type [Symbol] The type of metric to register. Valid types are: :counter, :gauge, :histogram
|
76
|
+
# @param name [String] The name of the metric.
|
77
|
+
# @param help [String] The help text for the metric.
|
78
|
+
# @param opts [Hash] A hash of options to pass to the metric object. This is different depending on the
|
79
|
+
# type of metrics being registered.
|
80
|
+
#
|
81
|
+
# @return metric [PrometheusExporter::Metric]
|
82
|
+
def register_metric(type, name, help, opts = nil)
|
83
|
+
AVAILABLE_METRIC_TYPES.include?(type) or raise ArgumentError, "Invalid metric type #{type.inspect}. Allowed types: #{AVAILABLE_METRIC_TYPES}"
|
84
|
+
client.register(type, name, help, opts)
|
85
|
+
end
|
86
|
+
|
87
|
+
private
|
88
|
+
|
89
|
+
def initialize_direct_client
|
90
|
+
@server = PrometheusExporter::Server::WebServer.new(bind: '0.0.0.0', port: port)
|
91
|
+
@server.start
|
92
|
+
@client = PrometheusExporter::LocalClient.new(collector: @server.collector)
|
93
|
+
end
|
94
|
+
|
95
|
+
def initialize_push_client
|
96
|
+
@client = PrometheusExporter::Client.new(host: host, port: port)
|
97
|
+
end
|
98
|
+
|
99
|
+
def validate_configuration!
|
100
|
+
mode.present? or raise ArgumentError, "Mode must be provided. Valid modes: #{AVAILABLE_MODES}"
|
101
|
+
port.present? or raise ArgumentError, "Port must be provided when in #{mode.inspect} mode"
|
102
|
+
if mode == :push
|
103
|
+
host.present? or raise ArgumentError, "Host must be provided when in #{mode.inspect} mode"
|
104
|
+
end
|
105
|
+
end
|
106
|
+
end
|
107
|
+
end
|
108
|
+
end
|
109
|
+
end
|
@@ -0,0 +1,52 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Invoca
|
4
|
+
module Metrics
|
5
|
+
module Prometheus
|
6
|
+
module DeclareMetrics
|
7
|
+
class Base
|
8
|
+
|
9
|
+
attr_reader :name, :type, :source, :metric, :default_labels, :graphite
|
10
|
+
|
11
|
+
def initialize(name, type, source, metric, labels:, graphite:)
|
12
|
+
@name = name
|
13
|
+
@type = type
|
14
|
+
@source = source
|
15
|
+
@metric = metric
|
16
|
+
@default_labels = labels
|
17
|
+
@graphite = graphite
|
18
|
+
|
19
|
+
if labels
|
20
|
+
labels.is_a?(Hash) or raise ArgumentError, "Labels must be type Hash but got #{labels.inspect}"
|
21
|
+
end
|
22
|
+
end
|
23
|
+
|
24
|
+
# TODO: Could default_labels be renamed to labels?
|
25
|
+
def settings
|
26
|
+
{ type: type, labels: default_labels, graphite: graphite }
|
27
|
+
end
|
28
|
+
|
29
|
+
# TODO: This is temporary; the invoca-metrics metrics objects
|
30
|
+
# should be made immutable, in which case the entire object
|
31
|
+
# would be replaced rather than resetting the Prometheus metric
|
32
|
+
# object as is done here
|
33
|
+
def reset_metric(new_metric)
|
34
|
+
@metric = new_metric
|
35
|
+
end
|
36
|
+
|
37
|
+
private
|
38
|
+
|
39
|
+
def render_graphite_string(**labels)
|
40
|
+
rendered_string = graphite.dup
|
41
|
+
labels.map do |key, value|
|
42
|
+
rendered_string.gsub!(":#{key}", value.to_s)
|
43
|
+
end
|
44
|
+
|
45
|
+
rendered_string.include?(':') and warn("Graphite string #{rendered_string} not fully rendered. Expecting additional label that has not been rendered.")
|
46
|
+
rendered_string
|
47
|
+
end
|
48
|
+
end
|
49
|
+
end
|
50
|
+
end
|
51
|
+
end
|
52
|
+
end
|
@@ -0,0 +1,26 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require_relative 'base'
|
4
|
+
|
5
|
+
module Invoca
|
6
|
+
module Metrics
|
7
|
+
module Prometheus
|
8
|
+
module DeclareMetrics
|
9
|
+
class Counter < Base
|
10
|
+
|
11
|
+
def increment(value = 1, **labels)
|
12
|
+
if metric.present?
|
13
|
+
metric.increment(default_labels.merge(labels), value)
|
14
|
+
else
|
15
|
+
warn("Counter being incremented without metric being present")
|
16
|
+
end
|
17
|
+
|
18
|
+
if graphite
|
19
|
+
Invoca::Metrics::Client.metrics.count(render_graphite_string(**default_labels.merge(labels)), value)
|
20
|
+
end
|
21
|
+
end
|
22
|
+
end
|
23
|
+
end
|
24
|
+
end
|
25
|
+
end
|
26
|
+
end
|
@@ -0,0 +1,61 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require_relative '../configuration'
|
4
|
+
require_relative 'counter'
|
5
|
+
require_relative 'gauge'
|
6
|
+
require_relative 'histogram'
|
7
|
+
|
8
|
+
module Invoca
|
9
|
+
module Metrics
|
10
|
+
module Prometheus
|
11
|
+
module DeclareMetrics
|
12
|
+
class Dsl
|
13
|
+
|
14
|
+
attr_reader :declared_metrics, :source, :prometheus_exporter_config
|
15
|
+
|
16
|
+
def initialize(source)
|
17
|
+
@source = source
|
18
|
+
@declared_metrics = []
|
19
|
+
if Invoca::Metrics::Prometheus.config_present?
|
20
|
+
@prometheus_exporter_config = Invoca::Metrics::Prometheus.config
|
21
|
+
end
|
22
|
+
end
|
23
|
+
|
24
|
+
def counter(name, labels: {}, graphite: nil, help: nil)
|
25
|
+
# TODO: We can drop the 'to_s' when we stop supporting ruby 2.6
|
26
|
+
if !name.to_s.end_with?("_total")
|
27
|
+
warn("#{source} counter #{name.inspect} should end in \"_total\" so that Grafana can tell that it is a counter.")
|
28
|
+
end
|
29
|
+
metric =
|
30
|
+
if Invoca::Metrics::Prometheus.config_present?
|
31
|
+
prometheus_exporter_config.register_metric(:counter, name, help, { labels: labels, graphite: graphite })
|
32
|
+
end
|
33
|
+
add_metric(Counter.new(name, :counter, source, metric, labels: labels, graphite: graphite))
|
34
|
+
end
|
35
|
+
|
36
|
+
def histogram(name, buckets:, labels: {}, graphite: nil, help: nil)
|
37
|
+
metric =
|
38
|
+
if Invoca::Metrics::Prometheus.config_present?
|
39
|
+
prometheus_exporter_config.register_metric(:histogram, name, help, { labels: labels, graphite: graphite, buckets: buckets })
|
40
|
+
end
|
41
|
+
add_metric(Histogram.new(name, :histogram, source, metric, buckets: buckets, labels: labels, graphite: graphite))
|
42
|
+
end
|
43
|
+
|
44
|
+
def gauge(name, labels: {}, graphite: nil, help: nil)
|
45
|
+
metric =
|
46
|
+
if Invoca::Metrics::Prometheus.config_present?
|
47
|
+
prometheus_exporter_config.register_metric(:gauge, name, help, { labels: labels, graphite: graphite })
|
48
|
+
end
|
49
|
+
add_metric(Gauge.new(name, :gauge, source, metric, labels: labels, graphite: graphite))
|
50
|
+
end
|
51
|
+
|
52
|
+
private
|
53
|
+
|
54
|
+
def add_metric(metric)
|
55
|
+
@declared_metrics << metric
|
56
|
+
end
|
57
|
+
end
|
58
|
+
end
|
59
|
+
end
|
60
|
+
end
|
61
|
+
end
|
@@ -0,0 +1,32 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require_relative 'base'
|
4
|
+
require 'active_support/deprecation'
|
5
|
+
|
6
|
+
module Invoca
|
7
|
+
module Metrics
|
8
|
+
module Prometheus
|
9
|
+
module DeclareMetrics
|
10
|
+
class Gauge < Base
|
11
|
+
|
12
|
+
def set(value = :'1', **labels)
|
13
|
+
if value == :'1'
|
14
|
+
value = 1
|
15
|
+
ActiveSupport::Deprecation.warn("gauge default value of 1 is deprecated; please pass an explicit value")
|
16
|
+
end
|
17
|
+
|
18
|
+
if metric.present?
|
19
|
+
metric.observe(value, default_labels.merge(labels))
|
20
|
+
else
|
21
|
+
warn("Gauge being set without metric being present")
|
22
|
+
end
|
23
|
+
|
24
|
+
if graphite
|
25
|
+
Invoca::Metrics::Client.metrics.gauge(render_graphite_string(**default_labels.merge(labels)), value)
|
26
|
+
end
|
27
|
+
end
|
28
|
+
end
|
29
|
+
end
|
30
|
+
end
|
31
|
+
end
|
32
|
+
end
|
@@ -0,0 +1,50 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require_relative 'base'
|
4
|
+
require 'benchmark'
|
5
|
+
require 'active_support/deprecation'
|
6
|
+
|
7
|
+
# TODO: Histogram could override Base#settings to include buckets
|
8
|
+
module Invoca
|
9
|
+
module Metrics
|
10
|
+
module Prometheus
|
11
|
+
module DeclareMetrics
|
12
|
+
class Histogram < Base
|
13
|
+
|
14
|
+
attr_reader :buckets
|
15
|
+
|
16
|
+
def initialize(name, type, source, metric, buckets:, labels:, graphite:)
|
17
|
+
@buckets = buckets
|
18
|
+
super(name, type, source, metric, labels: labels, graphite: graphite)
|
19
|
+
end
|
20
|
+
|
21
|
+
def add(value = :'1', **labels)
|
22
|
+
if value == :'1'
|
23
|
+
value = 1
|
24
|
+
ActiveSupport::Deprecation.warn("histogram default value of 1 is deprecated; please pass an explicit value")
|
25
|
+
end
|
26
|
+
|
27
|
+
if metric.present?
|
28
|
+
metric.observe(value, default_labels.merge(labels)) if metric.present?
|
29
|
+
else
|
30
|
+
warn("Histogram being used without metric being present")
|
31
|
+
end
|
32
|
+
|
33
|
+
if graphite
|
34
|
+
Invoca::Metrics::Client.metrics.timer(render_graphite_string(**default_labels.merge(labels)), value)
|
35
|
+
end
|
36
|
+
end
|
37
|
+
|
38
|
+
def time(return_timing: false, **labels)
|
39
|
+
result = nil
|
40
|
+
elapsed_time = Benchmark.realtime { result = yield }
|
41
|
+
elapsed_time_ms = (elapsed_time * 1000).to_i
|
42
|
+
add(elapsed_time_ms, **labels)
|
43
|
+
|
44
|
+
return_timing ? [result, elapsed_time_ms] : result
|
45
|
+
end
|
46
|
+
end
|
47
|
+
end
|
48
|
+
end
|
49
|
+
end
|
50
|
+
end
|
@@ -0,0 +1,43 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Invoca
|
4
|
+
module Metrics
|
5
|
+
module Prometheus
|
6
|
+
class MetricsRegistry
|
7
|
+
class ContradictoryMetricRegistration < StandardError; end
|
8
|
+
|
9
|
+
attr_reader :metrics
|
10
|
+
|
11
|
+
def initialize
|
12
|
+
@metrics = {}
|
13
|
+
end
|
14
|
+
|
15
|
+
def register(metric)
|
16
|
+
validate_metric_name!(metric)
|
17
|
+
@metrics[metric.name] = metric
|
18
|
+
end
|
19
|
+
|
20
|
+
def method_missing(method_name, *args)
|
21
|
+
metrics[method_name] || super
|
22
|
+
end
|
23
|
+
|
24
|
+
def respond_to?(method_name, *args)
|
25
|
+
@metrics[method_name] || super
|
26
|
+
end
|
27
|
+
|
28
|
+
def respond_to_missing?(method_name, include_private = false)
|
29
|
+
@metrics[method_name] || super
|
30
|
+
end
|
31
|
+
|
32
|
+
private
|
33
|
+
|
34
|
+
def validate_metric_name!(metric)
|
35
|
+
if (existing = @metrics[metric.name])
|
36
|
+
metric.settings == existing.settings or
|
37
|
+
raise ContradictoryMetricRegistration, "metric #{metric.name} already registered by #{existing.source} with contradictory settings #{existing.settings} vs new registered #{metric.source} with setting #{metric.settings}"
|
38
|
+
end
|
39
|
+
end
|
40
|
+
end
|
41
|
+
end
|
42
|
+
end
|
43
|
+
end
|
@@ -0,0 +1,67 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require_relative './prometheus/configuration'
|
4
|
+
require_relative './prometheus/metrics_registry'
|
5
|
+
|
6
|
+
module Invoca
|
7
|
+
module Metrics
|
8
|
+
module Prometheus
|
9
|
+
class NotConfiguredError < StandardError; end
|
10
|
+
|
11
|
+
class << self
|
12
|
+
# This method is used to configure Invoca::Metrics to export metrics to be
|
13
|
+
# pulled into Prometheus.
|
14
|
+
#
|
15
|
+
# @yield [Invoca::Metrics::Prometheus::Configuration]
|
16
|
+
#
|
17
|
+
# @void
|
18
|
+
def configure
|
19
|
+
@config = Invoca::Metrics::Prometheus::Configuration.new.tap { |config| yield config }.freeze
|
20
|
+
|
21
|
+
# If there are any metrics at this point, it's likely that Prometheus metrics
|
22
|
+
# were declared before invoca-metrics was loaded; walk the set of metrics in the
|
23
|
+
# registry, resetting their PrometheusExporter::Metric objects
|
24
|
+
metrics.metrics.values.each do |metric|
|
25
|
+
case metric.type
|
26
|
+
when :counter
|
27
|
+
metric.reset_metric(@config.register_metric(:counter, metric.name, nil, { labels: metric.default_labels, graphite: metric.graphite }))
|
28
|
+
when :gauge
|
29
|
+
metric.reset_metric(@config.register_metric(:gauge, metric.name, nil, { labels: metric.default_labels, graphite: metric.graphite }))
|
30
|
+
when :histogram
|
31
|
+
metric.reset_metric(@config.register_metric(:histogram, metric.name, nil, { labels: metric.default_labels, graphite: metric.graphite, buckets: metric.buckets }))
|
32
|
+
else
|
33
|
+
warn("Metric type #{metric.type} not one of #{Invoca::Metrics::Prometheus::Configuration::AVAILABLE_METRIC_TYPES}.")
|
34
|
+
end
|
35
|
+
end
|
36
|
+
|
37
|
+
@config
|
38
|
+
end
|
39
|
+
|
40
|
+
# Accessor for the existing Prometheus configuration
|
41
|
+
#
|
42
|
+
# @raise [Invoca::Metrics::Prometheus::NotConfiguredError]
|
43
|
+
# @return [Invoca::Metrics::Prometheus::Configuration]
|
44
|
+
def config
|
45
|
+
@config or raise NotConfiguredError, 'Invoca::Metrics::Prometheus is trying to be used without being configured'
|
46
|
+
end
|
47
|
+
|
48
|
+
# Helper method for checking if configuration is present
|
49
|
+
#
|
50
|
+
# @return [Boolean]
|
51
|
+
def config_present?
|
52
|
+
@config.present?
|
53
|
+
end
|
54
|
+
|
55
|
+
# TODO: Rename metrics -> registry so that we don't have things like metrics.metrics
|
56
|
+
# (see the configure method above)
|
57
|
+
|
58
|
+
# This method is used to access the global list of all metrics that were registered
|
59
|
+
#
|
60
|
+
# @return [Invoca::Metrics::Prometheus::MetricsRegistry]
|
61
|
+
def metrics
|
62
|
+
@metrics ||= MetricsRegistry.new
|
63
|
+
end
|
64
|
+
end
|
65
|
+
end
|
66
|
+
end
|
67
|
+
end
|
@@ -0,0 +1,49 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'statsd'
|
4
|
+
|
5
|
+
module Invoca
|
6
|
+
module Metrics
|
7
|
+
class StatsdClient < ::Statsd
|
8
|
+
MILLISECONDS_IN_SECOND = 1000
|
9
|
+
|
10
|
+
@log_send_failures = true
|
11
|
+
|
12
|
+
class << self
|
13
|
+
attr_accessor :log_send_failures
|
14
|
+
end
|
15
|
+
|
16
|
+
def time(stat, sample_rate = 1)
|
17
|
+
start = Time.now
|
18
|
+
result = yield
|
19
|
+
length_of_time = ((Time.now - start) * MILLISECONDS_IN_SECOND).round
|
20
|
+
timing(stat, length_of_time, sample_rate)
|
21
|
+
[result, length_of_time]
|
22
|
+
end
|
23
|
+
|
24
|
+
def send_to_socket(message)
|
25
|
+
self.class.logger&.debug { "Statsd: #{message}" }
|
26
|
+
socket.send(message, 0)
|
27
|
+
rescue => ex
|
28
|
+
if self.class.log_send_failures
|
29
|
+
self.class.logger&.error { "Statsd exception sending: #{ex.class}: #{ex}" }
|
30
|
+
end
|
31
|
+
|
32
|
+
nil
|
33
|
+
end
|
34
|
+
|
35
|
+
private
|
36
|
+
|
37
|
+
# Socket connection should be Thread local and not Fiber local
|
38
|
+
# Can't use `Thread.current[:statsd_client] ||=` because that will be fiber-local as well.
|
39
|
+
def socket
|
40
|
+
Thread.current.thread_variable_get(:statsd_socket) || Thread.current.thread_variable_set(:statsd_socket, new_socket)
|
41
|
+
end
|
42
|
+
|
43
|
+
def new_socket
|
44
|
+
self.class.logger&.info { "Statsd client connection info -- [hostname: #{@host}, port: #{@port}]" }
|
45
|
+
UDPSocket.new.tap { |udp| udp.connect(@host, @port) }
|
46
|
+
end
|
47
|
+
end
|
48
|
+
end
|
49
|
+
end
|
@@ -0,0 +1,117 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'active_support'
|
4
|
+
require 'active_support/core_ext'
|
5
|
+
|
6
|
+
require "invoca/metrics/version"
|
7
|
+
require "invoca/metrics/statsd_client"
|
8
|
+
require "invoca/metrics/client"
|
9
|
+
require "invoca/metrics/direct_metric"
|
10
|
+
require "invoca/metrics/batch"
|
11
|
+
require "invoca/metrics/gauge_cache"
|
12
|
+
require "invoca/metrics/prometheus"
|
13
|
+
require "invoca/metrics/prometheus/declare_metrics/dsl"
|
14
|
+
|
15
|
+
module Invoca
|
16
|
+
module Metrics
|
17
|
+
CONFIG_FIELDS = [:service_name, :server_name, :sub_server_name, :cluster_name, :statsd_host, :statsd_port].freeze
|
18
|
+
|
19
|
+
class << self
|
20
|
+
attr_accessor(*(CONFIG_FIELDS - [:service_name]), :default_config_key)
|
21
|
+
attr_writer :service_name, :graphite_usage_logging_enabled
|
22
|
+
|
23
|
+
def graphite_usage_logging_enabled
|
24
|
+
if @graphite_usage_logging_enabled.nil?
|
25
|
+
@graphite_usage_logging_enabled = true
|
26
|
+
end
|
27
|
+
@graphite_usage_logging_enabled
|
28
|
+
end
|
29
|
+
|
30
|
+
def service_name
|
31
|
+
@service_name or raise ArgumentError, "You must assign a value to Invoca::Metrics.service_name"
|
32
|
+
end
|
33
|
+
|
34
|
+
def initialized?
|
35
|
+
@service_name
|
36
|
+
end
|
37
|
+
|
38
|
+
def config
|
39
|
+
@config ||= {}
|
40
|
+
end
|
41
|
+
|
42
|
+
def config=(config_hash)
|
43
|
+
config_valid?(config_hash) or raise ArgumentError, "Invalid config #{config_hash}. Allowed fields for config key: #{CONFIG_FIELDS}."
|
44
|
+
@config = config_hash
|
45
|
+
end
|
46
|
+
|
47
|
+
def default_client_config
|
48
|
+
{
|
49
|
+
service_name: Invoca::Metrics.service_name,
|
50
|
+
server_name: Invoca::Metrics.server_name,
|
51
|
+
cluster_name: Invoca::Metrics.cluster_name,
|
52
|
+
statsd_host: Invoca::Metrics.statsd_host,
|
53
|
+
statsd_port: Invoca::Metrics.statsd_port,
|
54
|
+
sub_server_name: Invoca::Metrics.sub_server_name
|
55
|
+
}.merge(config[default_config_key] || {})
|
56
|
+
end
|
57
|
+
|
58
|
+
private
|
59
|
+
|
60
|
+
def config_valid?(config_hash)
|
61
|
+
config_hash.nil? || config_hash.all? { |_config_key, config_key_hash| (config_key_hash.keys - CONFIG_FIELDS).empty? }
|
62
|
+
end
|
63
|
+
end
|
64
|
+
|
65
|
+
# mix this module into your classes that need to send metrics
|
66
|
+
#
|
67
|
+
module Source
|
68
|
+
extend ActiveSupport::Concern
|
69
|
+
|
70
|
+
module ClassMethods
|
71
|
+
@metrics_namespace = nil
|
72
|
+
|
73
|
+
def metrics_namespace(namespace)
|
74
|
+
@metrics_namespace = namespace
|
75
|
+
end
|
76
|
+
|
77
|
+
def metrics
|
78
|
+
metrics_for(config_key: Invoca::Metrics.default_config_key)
|
79
|
+
end
|
80
|
+
|
81
|
+
def metrics_for(config_key:, namespace: nil)
|
82
|
+
config_from_key = Invoca::Metrics.config[config_key] || {}
|
83
|
+
metrics_config = if (effective_namespace = namespace || @metrics_namespace)
|
84
|
+
config_from_key.merge(namespace: effective_namespace)
|
85
|
+
else
|
86
|
+
config_from_key
|
87
|
+
end
|
88
|
+
Client.metrics(**Invoca::Metrics.default_client_config.merge(metrics_config))
|
89
|
+
end
|
90
|
+
|
91
|
+
def declare_metrics(&block)
|
92
|
+
dsl = Invoca::Metrics::Prometheus::DeclareMetrics::Dsl.new(self)
|
93
|
+
dsl.instance_eval(&block)
|
94
|
+
dsl.declared_metrics.each do |metric|
|
95
|
+
Invoca::Metrics::Prometheus.metrics.register(metric)
|
96
|
+
end
|
97
|
+
end
|
98
|
+
|
99
|
+
def prometheus_metrics
|
100
|
+
Invoca::Metrics::Prometheus.metrics
|
101
|
+
end
|
102
|
+
end
|
103
|
+
|
104
|
+
def prometheus_metrics
|
105
|
+
self.class.prometheus_metrics
|
106
|
+
end
|
107
|
+
|
108
|
+
def metrics
|
109
|
+
self.class.metrics
|
110
|
+
end
|
111
|
+
|
112
|
+
def metrics_for(config_key:, namespace: nil)
|
113
|
+
self.class.metrics_for(config_key: config_key, namespace: namespace)
|
114
|
+
end
|
115
|
+
end
|
116
|
+
end
|
117
|
+
end
|
metadata
CHANGED
@@ -1,24 +1,105 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: invoca-metrics
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version:
|
4
|
+
version: 1.14.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Invoca development
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
12
|
-
dependencies:
|
13
|
-
|
11
|
+
date: 2022-09-23 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: activesupport
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - ">="
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: '4.2'
|
20
|
+
- - "<"
|
21
|
+
- !ruby/object:Gem::Version
|
22
|
+
version: '7'
|
23
|
+
type: :runtime
|
24
|
+
prerelease: false
|
25
|
+
version_requirements: !ruby/object:Gem::Requirement
|
26
|
+
requirements:
|
27
|
+
- - ">="
|
28
|
+
- !ruby/object:Gem::Version
|
29
|
+
version: '4.2'
|
30
|
+
- - "<"
|
31
|
+
- !ruby/object:Gem::Version
|
32
|
+
version: '7'
|
33
|
+
- !ruby/object:Gem::Dependency
|
34
|
+
name: statsd-ruby
|
35
|
+
requirement: !ruby/object:Gem::Requirement
|
36
|
+
requirements:
|
37
|
+
- - "~>"
|
38
|
+
- !ruby/object:Gem::Version
|
39
|
+
version: 1.2.1
|
40
|
+
type: :runtime
|
41
|
+
prerelease: false
|
42
|
+
version_requirements: !ruby/object:Gem::Requirement
|
43
|
+
requirements:
|
44
|
+
- - "~>"
|
45
|
+
- !ruby/object:Gem::Version
|
46
|
+
version: 1.2.1
|
47
|
+
- !ruby/object:Gem::Dependency
|
48
|
+
name: prometheus_exporter
|
49
|
+
requirement: !ruby/object:Gem::Requirement
|
50
|
+
requirements:
|
51
|
+
- - "~>"
|
52
|
+
- !ruby/object:Gem::Version
|
53
|
+
version: '2.0'
|
54
|
+
type: :runtime
|
55
|
+
prerelease: false
|
56
|
+
version_requirements: !ruby/object:Gem::Requirement
|
57
|
+
requirements:
|
58
|
+
- - "~>"
|
59
|
+
- !ruby/object:Gem::Version
|
60
|
+
version: '2.0'
|
61
|
+
- !ruby/object:Gem::Dependency
|
62
|
+
name: contextual_logger
|
63
|
+
requirement: !ruby/object:Gem::Requirement
|
64
|
+
requirements:
|
65
|
+
- - "~>"
|
66
|
+
- !ruby/object:Gem::Version
|
67
|
+
version: '1.0'
|
68
|
+
type: :runtime
|
69
|
+
prerelease: false
|
70
|
+
version_requirements: !ruby/object:Gem::Requirement
|
71
|
+
requirements:
|
72
|
+
- - "~>"
|
73
|
+
- !ruby/object:Gem::Version
|
74
|
+
version: '1.0'
|
75
|
+
description: Invoca metrics reporting library
|
14
76
|
email:
|
15
77
|
- development@invoca.com
|
16
78
|
executables: []
|
17
79
|
extensions: []
|
18
80
|
extra_rdoc_files: []
|
19
|
-
files:
|
20
|
-
|
21
|
-
|
81
|
+
files:
|
82
|
+
- LICENSE.txt
|
83
|
+
- README.md
|
84
|
+
- invoca-metrics.gemspec
|
85
|
+
- lib/invoca/metrics.rb
|
86
|
+
- lib/invoca/metrics/batch.rb
|
87
|
+
- lib/invoca/metrics/client.rb
|
88
|
+
- lib/invoca/metrics/direct_metric.rb
|
89
|
+
- lib/invoca/metrics/gauge_cache.rb
|
90
|
+
- lib/invoca/metrics/prometheus.rb
|
91
|
+
- lib/invoca/metrics/prometheus/configuration.rb
|
92
|
+
- lib/invoca/metrics/prometheus/declare_metrics/base.rb
|
93
|
+
- lib/invoca/metrics/prometheus/declare_metrics/counter.rb
|
94
|
+
- lib/invoca/metrics/prometheus/declare_metrics/dsl.rb
|
95
|
+
- lib/invoca/metrics/prometheus/declare_metrics/gauge.rb
|
96
|
+
- lib/invoca/metrics/prometheus/declare_metrics/histogram.rb
|
97
|
+
- lib/invoca/metrics/prometheus/metrics_registry.rb
|
98
|
+
- lib/invoca/metrics/statsd_client.rb
|
99
|
+
- lib/invoca/metrics/version.rb
|
100
|
+
homepage: https://github.com/Invoca/invoca-metrics
|
101
|
+
licenses:
|
102
|
+
- MIT
|
22
103
|
metadata:
|
23
104
|
allowed_push_host: https://rubygems.org
|
24
105
|
post_install_message:
|
@@ -39,5 +120,5 @@ requirements: []
|
|
39
120
|
rubygems_version: 3.1.6
|
40
121
|
signing_key:
|
41
122
|
specification_version: 4
|
42
|
-
summary: Invoca metrics
|
123
|
+
summary: Invoca metrics reporting library
|
43
124
|
test_files: []
|