dogstatsd-ruby 5.2.0 → 5.3.3
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +73 -68
- data/lib/datadog/statsd/connection.rb +12 -11
- data/lib/datadog/statsd/connection_cfg.rb +76 -0
- data/lib/datadog/statsd/forwarder.rb +24 -15
- data/lib/datadog/statsd/message_buffer.rb +13 -4
- data/lib/datadog/statsd/sender.rb +73 -13
- data/lib/datadog/statsd/single_thread_sender.rb +40 -5
- data/lib/datadog/statsd/telemetry.rb +22 -1
- data/lib/datadog/statsd/timer.rb +60 -0
- data/lib/datadog/statsd/udp_connection.rb +20 -11
- data/lib/datadog/statsd/uds_connection.rb +16 -5
- data/lib/datadog/statsd/version.rb +1 -1
- data/lib/datadog/statsd.rb +36 -3
- metadata +15 -10
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: aa6ffb63549958aaa8ec14103abbd51082bd945e31d20f0148571ee2c90350b3
|
4
|
+
data.tar.gz: ef1361556ae6de5beb22523187c94ffe7285278918f47d671464b52708ae6973
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: ba586784b291adaf8a2eaa7a5b866f7931f2c887cb372a69a715f51c0c36a34fba1480cb2ecfd8bbf9573052c2839724fb6389cfa81e157a280e0a75ba0b7072
|
7
|
+
data.tar.gz: 7c2210e57a96e50825362dcedeaa686af6497f85941a9b7dab2a8ca67bbb7cec999d4f13015c5aab90b1a21cb824f5d8c3902cf5dd1421b3a54763a501f441b8
|
data/README.md
CHANGED
@@ -22,9 +22,9 @@ To instantiate a DogStatsd client:
|
|
22
22
|
# Import the library
|
23
23
|
require 'datadog/statsd'
|
24
24
|
|
25
|
-
# Create a DogStatsD client instance
|
25
|
+
# Create a DogStatsD client instance
|
26
26
|
statsd = Datadog::Statsd.new('localhost', 8125)
|
27
|
-
...
|
27
|
+
# ...
|
28
28
|
# release resources used by the client instance
|
29
29
|
statsd.close()
|
30
30
|
```
|
@@ -32,85 +32,73 @@ Or if you want to connect over Unix Domain Socket:
|
|
32
32
|
```ruby
|
33
33
|
# Connection over Unix Domain Socket
|
34
34
|
statsd = Datadog::Statsd.new(socket_path: '/path/to/socket/file')
|
35
|
-
...
|
35
|
+
# ...
|
36
36
|
# release resources used by the client instance
|
37
37
|
statsd.close()
|
38
38
|
```
|
39
39
|
|
40
|
-
Find a list of all the available options for your DogStatsD Client in the [DogStatsD-ruby rubydoc](https://www.rubydoc.info/github/DataDog/dogstatsd-ruby/master/Datadog/Statsd) or in the [Datadog public DogStatsD documentation](https://docs.datadoghq.com/developers/dogstatsd/?
|
40
|
+
Find a list of all the available options for your DogStatsD Client in the [DogStatsD-ruby rubydoc](https://www.rubydoc.info/github/DataDog/dogstatsd-ruby/master/Datadog/Statsd) or in the [Datadog public DogStatsD documentation](https://docs.datadoghq.com/developers/dogstatsd/?code-lang=ruby#client-instantiation-parameters).
|
41
41
|
|
42
42
|
### Migrating from v4.x to v5.x
|
43
43
|
|
44
44
|
If you are already using DogStatsD-ruby v4.x and you want to migrate to a version v5.x, the major
|
45
|
-
change concerning you is the new threading model
|
45
|
+
change concerning you is the new [threading model](#threading-model):
|
46
46
|
|
47
47
|
In practice, it means two things:
|
48
48
|
|
49
|
-
1. Now that the client is buffering metrics before sending them, you have to
|
50
|
-
call the method `Datadog::Statsd#flush` if you want to force the sending of metrics. Note that the companion thread will automatically flush the buffered metrics if the buffer gets full or when you are closing the instance.
|
49
|
+
1. Now that the client is buffering metrics before sending them, you have to call `Datadog::Statsd#flush(sync: true)` if you want synchronous behavior. In most cases, this is not needed, as the sender thread will automatically flush the buffered metrics if the buffer gets full or when you are closing the instance.
|
51
50
|
|
52
51
|
2. You have to make sure you are either:
|
53
52
|
|
54
|
-
*
|
55
|
-
* properly
|
53
|
+
* Using a singleton instance of the DogStatsD client instead of creating a new instance whenever you need one; this will let the buffering mechanism flush metrics regularly
|
54
|
+
* Or properly disposing of the DogStatsD client instance when it is not needed anymore using the method `Datadog::Statsd#close`
|
56
55
|
|
57
|
-
If you have issues with the
|
56
|
+
If you have issues with the sender thread or the buffering mode, you can instantiate a client that behaves exactly as in v4.x (i.e. no sender thread and flush on every metric submission):
|
58
57
|
|
59
58
|
```ruby
|
60
|
-
# Import the library
|
61
|
-
require 'datadog/statsd'
|
62
|
-
|
63
59
|
# Create a DogStatsD client instance using UDP
|
64
|
-
statsd = Datadog::Statsd.new('localhost', 8125, single_thread: true,
|
65
|
-
...
|
66
|
-
# to close the instance is not necessary in this case since metrics are flushed on submission
|
67
|
-
# but it is still a good practice and it explicitely closes the socket
|
60
|
+
statsd = Datadog::Statsd.new('localhost', 8125, single_thread: true, buffer_max_pool_size: 1)
|
61
|
+
# ...
|
68
62
|
statsd.close()
|
69
63
|
```
|
70
64
|
|
71
65
|
or
|
72
66
|
|
73
67
|
```ruby
|
74
|
-
# Import the library
|
75
|
-
require 'datadog/statsd'
|
76
|
-
|
77
68
|
# Create a DogStatsD client instance using UDS
|
78
|
-
statsd = Datadog::Statsd.new(socket_path: '/path/to/socket/file', single_thread: true,
|
79
|
-
...
|
80
|
-
# to close the instance is not necessary in this case since metrics are flushed on submission
|
81
|
-
# but it is still a good practice and it explicitely closes the socket
|
69
|
+
statsd = Datadog::Statsd.new(socket_path: '/path/to/socket/file', single_thread: true, buffer_max_pool_size: 1)
|
70
|
+
# ...
|
82
71
|
statsd.close()
|
83
72
|
```
|
84
73
|
|
85
74
|
### v5.x Common Pitfalls
|
86
75
|
|
87
|
-
Version v5.x of `dogstatsd-ruby` is using a
|
76
|
+
Version v5.x of `dogstatsd-ruby` is using a sender thread for flushing. This provides better performance, but you need to consider the following pitfalls:
|
77
|
+
|
78
|
+
1. Applications that use `fork` after having created the dogstatsd instance: the child process will automatically spawn a new sender thread to flush metrics.
|
88
79
|
|
89
|
-
|
90
|
-
* Applications creating a lot of different instances of the client without closing them: it is important to close the instance to free the thread and the socket it is using or it will lead to thread leaks.
|
80
|
+
2. Applications that create multiple instances of the client without closing them: it is important to `#close` all instances to free the thread and the socket they are using otherwise you will leak those resources.
|
91
81
|
|
92
82
|
If you are using [Sidekiq](https://github.com/mperham/sidekiq), please make sure to close the client instances that are instantiated. [See this example on using DogStatsD-ruby v5.x with Sidekiq](https://github.com/DataDog/dogstatsd-ruby/blob/master/examples/sidekiq_example.rb).
|
93
83
|
|
94
84
|
If you are using [Puma](https://github.com/puma/puma) or [Unicorn](https://yhbt.net/unicorn.git), please make sure to create the instance of DogStatsD in the workers, not in the main process before it forks to create its workers. See [this comment for more details](https://github.com/DataDog/dogstatsd-ruby/issues/179#issuecomment-845570345).
|
95
85
|
|
96
|
-
Applications that
|
86
|
+
Applications that run into issues but can't apply these recommendations should use the `single_thread` mode which disables the use of the sender thread.
|
87
|
+
Here is how to instantiate a client in this mode:
|
97
88
|
|
98
89
|
```ruby
|
99
|
-
# Import the library
|
100
|
-
require 'datadog/statsd'
|
101
|
-
|
102
|
-
# Create a DogStatsD client instance.
|
103
90
|
statsd = Datadog::Statsd.new('localhost', 8125, single_thread: true)
|
104
|
-
...
|
91
|
+
# ...
|
105
92
|
# release resources used by the client instance and flush last metrics
|
106
93
|
statsd.close()
|
107
94
|
```
|
108
95
|
|
109
96
|
### Origin detection over UDP
|
110
97
|
|
111
|
-
Origin detection is a method to detect which pod DogStatsD packets are coming from in order to add the pod's tags to the tag list.
|
98
|
+
Origin detection is a method to detect which pod DogStatsD packets are coming from, in order to add the pod's tags to the tag list.
|
99
|
+
|
100
|
+
To enable origin detection over UDP, add the following lines to your application manifest:
|
112
101
|
|
113
|
-
To enable origin detection over UDP, add the following lines to your application manifest
|
114
102
|
```yaml
|
115
103
|
env:
|
116
104
|
- name: DD_ENTITY_ID
|
@@ -118,56 +106,57 @@ env:
|
|
118
106
|
fieldRef:
|
119
107
|
fieldPath: metadata.uid
|
120
108
|
```
|
109
|
+
|
121
110
|
The DogStatsD client attaches an internal tag, `entity_id`. The value of this tag is the content of the `DD_ENTITY_ID` environment variable, which is the pod’s UID.
|
122
111
|
|
123
112
|
## Usage
|
124
113
|
|
125
|
-
In order to use DogStatsD metrics, events, and Service Checks the Agent must be [running and available](https://docs.datadoghq.com/developers/dogstatsd/?tab=ruby).
|
114
|
+
In order to use DogStatsD metrics, events, and Service Checks the Datadog Agent must be [running and available](https://docs.datadoghq.com/developers/dogstatsd/?tab=ruby).
|
126
115
|
|
127
116
|
### Metrics
|
128
117
|
|
129
|
-
After the client is created, you can start sending custom metrics to Datadog. See the dedicated [Metric Submission: DogStatsD documentation](https://docs.datadoghq.com/
|
118
|
+
After the client is created, you can start sending custom metrics to Datadog. See the dedicated [Metric Submission: DogStatsD documentation](https://docs.datadoghq.com/metrics/dogstatsd_metrics_submission/?tab=ruby) to see how to submit all supported metric types to Datadog with working code examples:
|
130
119
|
|
131
|
-
* [Submit a COUNT metric](https://docs.datadoghq.com/
|
132
|
-
* [Submit a GAUGE metric](https://docs.datadoghq.com/
|
133
|
-
* [Submit a SET metric](https://docs.datadoghq.com/
|
134
|
-
* [Submit a HISTOGRAM metric](https://docs.datadoghq.com/
|
135
|
-
* [Submit a DISTRIBUTION metric](https://docs.datadoghq.com/
|
120
|
+
* [Submit a COUNT metric](https://docs.datadoghq.com/metrics/dogstatsd_metrics_submission/?code-lang=ruby#count).
|
121
|
+
* [Submit a GAUGE metric](https://docs.datadoghq.com/metrics/dogstatsd_metrics_submission/?code-lang=ruby#gauge).
|
122
|
+
* [Submit a SET metric](https://docs.datadoghq.com/metrics/dogstatsd_metrics_submission/?code-lang=ruby#set)
|
123
|
+
* [Submit a HISTOGRAM metric](https://docs.datadoghq.com/metrics/dogstatsd_metrics_submission/?code-lang=ruby#histogram)
|
124
|
+
* [Submit a DISTRIBUTION metric](https://docs.datadoghq.com/metrics/dogstatsd_metrics_submission/?code-lang=ruby#distribution)
|
136
125
|
|
137
|
-
Some options are suppported when submitting metrics, like [applying a Sample Rate to your metrics](https://docs.datadoghq.com/
|
126
|
+
Some options are suppported when submitting metrics, like [applying a Sample Rate to your metrics](https://docs.datadoghq.com/metrics/dogstatsd_metrics_submission/?tab=ruby#metric-submission-options) or [tagging your metrics with your custom tags](https://docs.datadoghq.com/metrics/dogstatsd_metrics_submission/?tab=ruby#metric-tagging). Find all the available functions to report metrics in the [DogStatsD-ruby rubydoc](https://www.rubydoc.info/github/DataDog/dogstatsd-ruby/master/Datadog/Statsd).
|
138
127
|
|
139
128
|
### Events
|
140
129
|
|
141
|
-
After the client is created, you can start sending events to your Datadog Event Stream. See the dedicated [Event Submission: DogStatsD documentation](https://docs.datadoghq.com/
|
130
|
+
After the client is created, you can start sending events to your Datadog Event Stream. See the dedicated [Event Submission: DogStatsD documentation](https://docs.datadoghq.com/events/guides/dogstatsd/?code-lang=ruby) to see how to submit an event to Datadog your Event Stream.
|
142
131
|
|
143
132
|
### Service Checks
|
144
133
|
|
145
134
|
After the client is created, you can start sending Service Checks to Datadog. See the dedicated [Service Check Submission: DogStatsD documentation](https://docs.datadoghq.com/developers/service_checks/dogstatsd_service_checks_submission/?tab=ruby) to see how to submit a Service Check to Datadog.
|
146
135
|
|
147
|
-
### Maximum
|
136
|
+
### Maximum packet size in high-throughput scenarios
|
148
137
|
|
149
138
|
In order to have the most efficient use of this library in high-throughput scenarios,
|
150
|
-
|
151
|
-
and UDP (1432 bytes)
|
152
|
-
|
153
|
-
|
139
|
+
recommended values for the maximum packet size have already been set for both UDS (8192 bytes)
|
140
|
+
and UDP (1432 bytes).
|
141
|
+
|
142
|
+
However, if are in control of your network and want to use a different value for the maximum packet
|
143
|
+
size, you can do it by setting the `buffer_max_payload_size` parameter:
|
154
144
|
|
155
145
|
```ruby
|
156
|
-
# Create a DogStatsD client instance.
|
157
146
|
statsd = Datadog::Statsd.new('localhost', 8125, buffer_max_payload_size: 4096)
|
147
|
+
# ...
|
148
|
+
statsd.close()
|
158
149
|
```
|
159
150
|
|
160
151
|
## Threading model
|
161
152
|
|
162
|
-
|
153
|
+
Starting with version 5.0, `dogstatsd-ruby` employs a new threading model where one instance of `Datadog::Statsd` can be shared between threads and where data sending is non-blocking (asynchronous).
|
163
154
|
|
164
|
-
When you instantiate a `Datadog::Statsd`, a
|
155
|
+
When you instantiate a `Datadog::Statsd`, a sender thread is spawned. This thread will be called the Sender thread, as it is modeled by the [Sender](../lib/datadog/statsd/sender.rb) class. You can make use of `single_thread: true` to disable this behavior.
|
165
156
|
|
166
|
-
This thread is stopped when you close the statsd client (`Datadog::Statsd#close`).
|
167
|
-
could lead to a thread leak (even though they will be sleeping, blocked on IO).
|
168
|
-
The communication between the current thread is managed through a standard Ruby Queue.
|
157
|
+
This thread is stopped when you close the statsd client (`Datadog::Statsd#close`). Instantiating a lot of statsd clients without calling `#close` after they are not needed anymore will most likely lead to threads being leaked.
|
169
158
|
|
170
|
-
The sender thread has the following logic (
|
159
|
+
The sender thread has the following logic (from `Datadog::Statsd::Sender#send_loop`):
|
171
160
|
|
172
161
|
```
|
173
162
|
while the sender message queue is not closed do
|
@@ -183,15 +172,22 @@ while the sender message queue is not closed do
|
|
183
172
|
end while
|
184
173
|
```
|
185
174
|
|
186
|
-
|
175
|
+
There are three different kinds of messages:
|
176
|
+
|
177
|
+
1. a control message to flush the buffer in the connection
|
178
|
+
2. a control message to synchronize any thread with the sender thread
|
179
|
+
3. a message to append to the buffer
|
180
|
+
|
181
|
+
There is also an implicit message which closes the queue which will cause the sender thread to finish processing and exit.
|
187
182
|
|
188
|
-
We can see that there is 3 different kind of messages:
|
189
183
|
|
190
|
-
|
191
|
-
|
192
|
-
|
184
|
+
```ruby
|
185
|
+
statsd = Datadog::Statsd.new('localhost', 8125)
|
186
|
+
```
|
187
|
+
|
188
|
+
The message queue's maximum size (in messages) is given by the `sender_queue_size` argument, and has appropriate defaults for UDP (2048) and UDS (512).
|
193
189
|
|
194
|
-
|
190
|
+
The `buffer_flush_interval`, if enabled, is implemented with an additional thread which manages the timing of those flushes. This additional thread is used even if `single_thread: true`.
|
195
191
|
|
196
192
|
### Usual workflow
|
197
193
|
|
@@ -199,20 +195,29 @@ You push metrics to the statsd client which writes them quickly to the sender me
|
|
199
195
|
|
200
196
|
### Flushing
|
201
197
|
|
202
|
-
When calling
|
198
|
+
When calling `Datadog::Statsd#flush`, a specific control message (`:flush`) is sent to the sender thread. When the sender thread receives it, it flushes its internal buffer into the connection.
|
203
199
|
|
204
200
|
### Rendez-vous
|
205
201
|
|
206
|
-
It is possible to ensure a message has been consumed by the sender thread and written to the buffer by simply calling a rendez-vous right after. This is done when you are doing a
|
202
|
+
It is possible to ensure a message has been consumed by the sender thread and written to the buffer by simply calling a rendez-vous right after. This is done when you are doing a synchronous flush using `Datadog::Statsd#flush(sync: true)`.
|
203
|
+
|
204
|
+
Doing so means the caller thread is blocked and waiting until the data has been flushed by the sender thread.
|
205
|
+
|
206
|
+
This is useful when preparing to exit the application or when checking unit tests.
|
207
|
+
|
208
|
+
### Thread-safety
|
209
|
+
|
210
|
+
By default, instances of `Datadog::Statsd` are thread-safe and we recommend that a single instance be reused by all application threads (even in applications that employ forking). The sole exception is the `#close` method — this method is not yet thread safe (work in progress here [#209](https://github.com/DataDog/dogstatsd-ruby/pull/209)).
|
211
|
+
|
212
|
+
When using the `single_thread: true` mode, instances of `Datadog::Statsd` are still thread-safe, but you may run into contention on heavily-threaded applications, so we don’t recommend (for performance reasons) reusing these instances.
|
207
213
|
|
208
|
-
|
214
|
+
## Versioning
|
209
215
|
|
210
|
-
This is
|
216
|
+
This Ruby gem is using [Semantic Versioning](https://guides.rubygems.org/patterns/#semantic-versioning) but please note that supported Ruby versions can change in a minor release of this library. As much as possible, we will add a "future deprecation" message in the minor release preceding the one dropping the support.
|
211
217
|
|
212
218
|
## Credits
|
213
219
|
|
214
|
-
dogstatsd-ruby is forked from Rein Henrichs [original Statsd
|
215
|
-
client](https://github.com/reinh/statsd).
|
220
|
+
dogstatsd-ruby is forked from Rein Henrichs' [original Statsd client](https://github.com/reinh/statsd).
|
216
221
|
|
217
222
|
Copyright (c) 2011 Rein Henrichs. See LICENSE.txt for
|
218
223
|
further details.
|
@@ -8,16 +8,11 @@ module Datadog
|
|
8
8
|
@logger = logger
|
9
9
|
end
|
10
10
|
|
11
|
-
|
12
|
-
|
13
|
-
begin
|
14
|
-
@socket && @socket.close if instance_variable_defined?(:@socket)
|
15
|
-
rescue StandardError => boom
|
16
|
-
logger.error { "Statsd: #{boom.class} #{boom}" } if logger
|
17
|
-
end
|
18
|
-
@socket = nil
|
11
|
+
def reset_telemetry
|
12
|
+
telemetry.reset
|
19
13
|
end
|
20
14
|
|
15
|
+
# not thread safe: `Sender` instances that use this are required to properly synchronize or sequence calls to this method
|
21
16
|
def write(payload)
|
22
17
|
logger.debug { "Statsd: #{payload}" } if logger
|
23
18
|
|
@@ -36,23 +31,29 @@ module Datadog
|
|
36
31
|
retries += 1
|
37
32
|
begin
|
38
33
|
close
|
34
|
+
connect
|
39
35
|
retry
|
40
36
|
rescue StandardError => e
|
41
37
|
boom = e
|
42
38
|
end
|
43
39
|
end
|
44
40
|
|
45
|
-
telemetry.
|
41
|
+
telemetry.dropped_writer(packets: 1, bytes: payload.length) if telemetry
|
46
42
|
logger.error { "Statsd: #{boom.class} #{boom}" } if logger
|
47
43
|
nil
|
48
44
|
end
|
49
45
|
|
50
46
|
private
|
47
|
+
|
51
48
|
attr_reader :telemetry
|
52
49
|
attr_reader :logger
|
53
50
|
|
54
|
-
def
|
55
|
-
|
51
|
+
def connect
|
52
|
+
raise 'Should be implemented by subclass'
|
53
|
+
end
|
54
|
+
|
55
|
+
def close
|
56
|
+
raise 'Should be implemented by subclass'
|
56
57
|
end
|
57
58
|
end
|
58
59
|
end
|
@@ -0,0 +1,76 @@
|
|
1
|
+
module Datadog
|
2
|
+
class Statsd
|
3
|
+
class ConnectionCfg
|
4
|
+
attr_reader :host
|
5
|
+
attr_reader :port
|
6
|
+
attr_reader :socket_path
|
7
|
+
attr_reader :transport_type
|
8
|
+
|
9
|
+
def initialize(host: nil, port: nil, socket_path: nil)
|
10
|
+
initialize_with_constructor_args(host: host, port: port, socket_path: socket_path) ||
|
11
|
+
initialize_with_env_vars ||
|
12
|
+
initialize_with_defaults
|
13
|
+
end
|
14
|
+
|
15
|
+
def make_connection(**params)
|
16
|
+
case @transport_type
|
17
|
+
when :udp
|
18
|
+
UDPConnection.new(@host, @port, **params)
|
19
|
+
when :uds
|
20
|
+
UDSConnection.new(@socket_path, **params)
|
21
|
+
end
|
22
|
+
end
|
23
|
+
|
24
|
+
private
|
25
|
+
|
26
|
+
DEFAULT_HOST = '127.0.0.1'
|
27
|
+
DEFAULT_PORT = 8125
|
28
|
+
|
29
|
+
def initialize_with_constructor_args(host: nil, port: nil, socket_path: nil)
|
30
|
+
try_initialize_with(host: host, port: port, socket_path: socket_path,
|
31
|
+
not_both_error_message:
|
32
|
+
"Both UDP: (host/port #{host}:#{port}) and UDS (socket_path #{socket_path}) " +
|
33
|
+
"constructor arguments were given. Use only one or the other.",
|
34
|
+
)
|
35
|
+
end
|
36
|
+
|
37
|
+
def initialize_with_env_vars()
|
38
|
+
try_initialize_with(
|
39
|
+
host: ENV['DD_AGENT_HOST'],
|
40
|
+
port: ENV['DD_DOGSTATSD_PORT'] && ENV['DD_DOGSTATSD_PORT'].to_i,
|
41
|
+
socket_path: ENV['DD_DOGSTATSD_SOCKET'],
|
42
|
+
not_both_error_message:
|
43
|
+
"Both UDP (DD_AGENT_HOST/DD_DOGSTATSD_PORT #{ENV['DD_AGENT_HOST']}:#{ENV['DD_DOGSTATSD_PORT']}) " +
|
44
|
+
"and UDS (DD_DOGSTATSD_SOCKET #{ENV['DD_DOGSTATSD_SOCKET']}) environment variables are set. " +
|
45
|
+
"Set only one or the other." %
|
46
|
+
[ENV['DD_AGENT_HOST'], ENV['DD_DOGSTATSD_PORT'], ENV['DD_DOGSTATSD_SOCKET']])
|
47
|
+
end
|
48
|
+
|
49
|
+
def initialize_with_defaults()
|
50
|
+
try_initialize_with(host: DEFAULT_HOST, port: DEFAULT_PORT)
|
51
|
+
end
|
52
|
+
|
53
|
+
def try_initialize_with(host: nil, port: nil, socket_path: nil, not_both_error_message: "")
|
54
|
+
if (host || port) && socket_path
|
55
|
+
raise ArgumentError, not_both_error_message
|
56
|
+
end
|
57
|
+
|
58
|
+
if host || port
|
59
|
+
@host = host || DEFAULT_HOST
|
60
|
+
@port = port || DEFAULT_PORT
|
61
|
+
@socket_path = nil
|
62
|
+
@transport_type = :udp
|
63
|
+
return true
|
64
|
+
elsif socket_path
|
65
|
+
@host = nil
|
66
|
+
@port = nil
|
67
|
+
@socket_path = socket_path
|
68
|
+
@transport_type = :uds
|
69
|
+
return true
|
70
|
+
end
|
71
|
+
|
72
|
+
return false
|
73
|
+
end
|
74
|
+
end
|
75
|
+
end
|
76
|
+
end
|
@@ -7,13 +7,14 @@ module Datadog
|
|
7
7
|
attr_reader :transport_type
|
8
8
|
|
9
9
|
def initialize(
|
10
|
-
|
11
|
-
port: nil,
|
12
|
-
socket_path: nil,
|
10
|
+
connection_cfg: nil,
|
13
11
|
|
14
12
|
buffer_max_payload_size: nil,
|
15
13
|
buffer_max_pool_size: nil,
|
16
14
|
buffer_overflowing_stategy: :drop,
|
15
|
+
buffer_flush_interval: nil,
|
16
|
+
|
17
|
+
sender_queue_size: nil,
|
17
18
|
|
18
19
|
telemetry_flush_interval: nil,
|
19
20
|
global_tags: [],
|
@@ -22,24 +23,20 @@ module Datadog
|
|
22
23
|
|
23
24
|
logger: nil
|
24
25
|
)
|
25
|
-
@transport_type =
|
26
|
+
@transport_type = connection_cfg.transport_type
|
26
27
|
|
27
28
|
if telemetry_flush_interval
|
28
29
|
@telemetry = Telemetry.new(telemetry_flush_interval,
|
29
30
|
global_tags: global_tags,
|
30
|
-
transport_type: transport_type
|
31
|
+
transport_type: @transport_type
|
31
32
|
)
|
32
33
|
end
|
33
34
|
|
34
|
-
@connection =
|
35
|
-
when :udp
|
36
|
-
UDPConnection.new(host, port, logger: logger, telemetry: telemetry)
|
37
|
-
when :uds
|
38
|
-
UDSConnection.new(socket_path, logger: logger, telemetry: telemetry)
|
39
|
-
end
|
35
|
+
@connection = connection_cfg.make_connection(logger: logger, telemetry: telemetry)
|
40
36
|
|
41
37
|
# Initialize buffer
|
42
|
-
buffer_max_payload_size ||= (transport_type == :udp ?
|
38
|
+
buffer_max_payload_size ||= (@transport_type == :udp ?
|
39
|
+
UDP_DEFAULT_BUFFER_SIZE : UDS_DEFAULT_BUFFER_SIZE)
|
43
40
|
|
44
41
|
if buffer_max_payload_size <= 0
|
45
42
|
raise ArgumentError, 'buffer_max_payload_size cannot be <= 0'
|
@@ -49,13 +46,26 @@ module Datadog
|
|
49
46
|
raise ArgumentError, "buffer_max_payload_size is not high enough to use telemetry (tags=(#{global_tags.inspect}))"
|
50
47
|
end
|
51
48
|
|
52
|
-
|
49
|
+
buffer = MessageBuffer.new(@connection,
|
53
50
|
max_payload_size: buffer_max_payload_size,
|
54
51
|
max_pool_size: buffer_max_pool_size || DEFAULT_BUFFER_POOL_SIZE,
|
55
52
|
overflowing_stategy: buffer_overflowing_stategy,
|
56
53
|
)
|
57
54
|
|
58
|
-
|
55
|
+
sender_queue_size ||= (@transport_type == :udp ?
|
56
|
+
UDP_DEFAULT_SENDER_QUEUE_SIZE : UDS_DEFAULT_SENDER_QUEUE_SIZE)
|
57
|
+
|
58
|
+
@sender = single_thread ?
|
59
|
+
SingleThreadSender.new(
|
60
|
+
buffer,
|
61
|
+
logger: logger,
|
62
|
+
flush_interval: buffer_flush_interval) :
|
63
|
+
Sender.new(
|
64
|
+
buffer,
|
65
|
+
logger: logger,
|
66
|
+
flush_interval: buffer_flush_interval,
|
67
|
+
telemetry: @telemetry,
|
68
|
+
queue_size: sender_queue_size)
|
59
69
|
@sender.start
|
60
70
|
end
|
61
71
|
|
@@ -99,7 +109,6 @@ module Datadog
|
|
99
109
|
end
|
100
110
|
|
101
111
|
private
|
102
|
-
attr_reader :buffer
|
103
112
|
attr_reader :sender
|
104
113
|
attr_reader :connection
|
105
114
|
|
@@ -19,7 +19,7 @@ module Datadog
|
|
19
19
|
@overflowing_stategy = overflowing_stategy
|
20
20
|
|
21
21
|
@buffer = String.new
|
22
|
-
|
22
|
+
clear_buffer
|
23
23
|
end
|
24
24
|
|
25
25
|
def add(message)
|
@@ -42,16 +42,20 @@ module Datadog
|
|
42
42
|
true
|
43
43
|
end
|
44
44
|
|
45
|
+
def reset
|
46
|
+
clear_buffer
|
47
|
+
connection.reset_telemetry
|
48
|
+
end
|
49
|
+
|
45
50
|
def flush
|
46
51
|
return if buffer.empty?
|
47
52
|
|
48
53
|
connection.write(buffer)
|
49
|
-
|
50
|
-
buffer.clear
|
51
|
-
@message_count = 0
|
54
|
+
clear_buffer
|
52
55
|
end
|
53
56
|
|
54
57
|
private
|
58
|
+
|
55
59
|
attr :max_payload_size
|
56
60
|
attr :max_pool_size
|
57
61
|
|
@@ -66,6 +70,11 @@ module Datadog
|
|
66
70
|
false
|
67
71
|
end
|
68
72
|
|
73
|
+
def clear_buffer
|
74
|
+
buffer.clear
|
75
|
+
@message_count = 0
|
76
|
+
end
|
77
|
+
|
69
78
|
def preemptive_flush?
|
70
79
|
@message_count == max_pool_size || buffer.bytesize > bytesize_threshold
|
71
80
|
end
|
@@ -2,25 +2,54 @@
|
|
2
2
|
|
3
3
|
module Datadog
|
4
4
|
class Statsd
|
5
|
+
# Sender is using a companion thread to flush and pack messages
|
6
|
+
# in a `MessageBuffer`.
|
7
|
+
# The communication with this thread is done using a `Queue`.
|
8
|
+
# If the thread is dead, it is starting a new one to avoid having a blocked
|
9
|
+
# Sender with no companion thread to communicate with (most of the time, having
|
10
|
+
# a dead companion thread means that a fork just happened and that we are
|
11
|
+
# running in the child process).
|
5
12
|
class Sender
|
6
13
|
CLOSEABLE_QUEUES = Queue.instance_methods.include?(:close)
|
7
14
|
|
8
|
-
def initialize(message_buffer)
|
15
|
+
def initialize(message_buffer, telemetry: nil, queue_size: UDP_DEFAULT_BUFFER_SIZE, logger: nil, flush_interval: nil, queue_class: Queue, thread_class: Thread)
|
9
16
|
@message_buffer = message_buffer
|
17
|
+
@telemetry = telemetry
|
18
|
+
@queue_size = queue_size
|
19
|
+
@logger = logger
|
20
|
+
@mx = Mutex.new
|
21
|
+
@queue_class = queue_class
|
22
|
+
@thread_class = thread_class
|
23
|
+
if flush_interval
|
24
|
+
@flush_timer = Datadog::Statsd::Timer.new(flush_interval) { flush(sync: true) }
|
25
|
+
end
|
10
26
|
end
|
11
27
|
|
12
28
|
def flush(sync: false)
|
13
|
-
#
|
14
|
-
|
15
|
-
|
16
|
-
message_queue
|
29
|
+
# keep a copy around in case another thread is calling #stop while this method is running
|
30
|
+
current_message_queue = message_queue
|
31
|
+
|
32
|
+
# don't try to flush if there is no message_queue instantiated or
|
33
|
+
# no companion thread running
|
34
|
+
if !current_message_queue
|
35
|
+
@logger.debug { "Statsd: can't flush: no message queue ready" } if @logger
|
36
|
+
return
|
37
|
+
end
|
38
|
+
if !sender_thread.alive?
|
39
|
+
@logger.debug { "Statsd: can't flush: no sender_thread alive" } if @logger
|
40
|
+
return
|
41
|
+
end
|
17
42
|
|
43
|
+
current_message_queue.push(:flush)
|
18
44
|
rendez_vous if sync
|
19
45
|
end
|
20
46
|
|
21
47
|
def rendez_vous
|
48
|
+
# could happen if #start hasn't be called
|
49
|
+
return unless message_queue
|
50
|
+
|
22
51
|
# Initialize and get the thread's sync queue
|
23
|
-
queue = (
|
52
|
+
queue = (@thread_class.current[:statsd_sync_queue] ||= @queue_class.new)
|
24
53
|
# tell sender-thread to notify us in the current
|
25
54
|
# thread's queue
|
26
55
|
message_queue.push(queue)
|
@@ -32,31 +61,63 @@ module Datadog
|
|
32
61
|
def add(message)
|
33
62
|
raise ArgumentError, 'Start sender first' unless message_queue
|
34
63
|
|
35
|
-
|
64
|
+
# if the thread does not exist, we assume we are running in a forked process,
|
65
|
+
# empty the message queue and message buffers (these messages belong to
|
66
|
+
# the parent process) and spawn a new companion thread.
|
67
|
+
if !sender_thread.alive?
|
68
|
+
@mx.synchronize {
|
69
|
+
# a call from another thread has already re-created
|
70
|
+
# the companion thread before this one acquired the lock
|
71
|
+
break if sender_thread.alive?
|
72
|
+
@logger.debug { "Statsd: companion thread is dead, re-creating one" } if @logger
|
73
|
+
|
74
|
+
message_queue.close if CLOSEABLE_QUEUES
|
75
|
+
@message_queue = nil
|
76
|
+
message_buffer.reset
|
77
|
+
start
|
78
|
+
@flush_timer.start if @flush_timer && @flush_timer.stop?
|
79
|
+
}
|
80
|
+
end
|
81
|
+
|
82
|
+
if message_queue.length <= @queue_size
|
83
|
+
message_queue << message
|
84
|
+
else
|
85
|
+
@telemetry.dropped_queue(packets: 1, bytes: message.bytesize) if @telemetry
|
86
|
+
end
|
36
87
|
end
|
37
88
|
|
38
89
|
def start
|
39
90
|
raise ArgumentError, 'Sender already started' if message_queue
|
40
91
|
|
41
|
-
# initialize message queue for background thread
|
42
|
-
@message_queue =
|
92
|
+
# initialize a new message queue for the background thread
|
93
|
+
@message_queue = @queue_class.new
|
43
94
|
# start background thread
|
44
|
-
@sender_thread =
|
95
|
+
@sender_thread = @thread_class.new(&method(:send_loop))
|
96
|
+
@sender_thread.name = "Statsd Sender" unless Gem::Version.new(RUBY_VERSION) < Gem::Version.new('2.3')
|
97
|
+
@flush_timer.start if @flush_timer
|
45
98
|
end
|
46
99
|
|
47
100
|
if CLOSEABLE_QUEUES
|
101
|
+
# when calling stop, make sure that no other threads is trying
|
102
|
+
# to close the sender nor trying to continue to `#add` more message
|
103
|
+
# into the sender.
|
48
104
|
def stop(join_worker: true)
|
49
105
|
message_queue = @message_queue
|
50
106
|
message_queue.close if message_queue
|
51
107
|
|
108
|
+
@flush_timer.stop if @flush_timer
|
52
109
|
sender_thread = @sender_thread
|
53
110
|
sender_thread.join if sender_thread && join_worker
|
54
111
|
end
|
55
112
|
else
|
113
|
+
# when calling stop, make sure that no other threads is trying
|
114
|
+
# to close the sender nor trying to continue to `#add` more message
|
115
|
+
# into the sender.
|
56
116
|
def stop(join_worker: true)
|
57
117
|
message_queue = @message_queue
|
58
118
|
message_queue << :close if message_queue
|
59
119
|
|
120
|
+
@flush_timer.stop if @flush_timer
|
60
121
|
sender_thread = @sender_thread
|
61
122
|
sender_thread.join if sender_thread && join_worker
|
62
123
|
end
|
@@ -65,7 +126,6 @@ module Datadog
|
|
65
126
|
private
|
66
127
|
|
67
128
|
attr_reader :message_buffer
|
68
|
-
|
69
129
|
attr_reader :message_queue
|
70
130
|
attr_reader :sender_thread
|
71
131
|
|
@@ -79,7 +139,7 @@ module Datadog
|
|
79
139
|
case message
|
80
140
|
when :flush
|
81
141
|
message_buffer.flush
|
82
|
-
when
|
142
|
+
when @queue_class
|
83
143
|
message.push(:go_on)
|
84
144
|
else
|
85
145
|
message_buffer.add(message)
|
@@ -101,7 +161,7 @@ module Datadog
|
|
101
161
|
break
|
102
162
|
when :flush
|
103
163
|
message_buffer.flush
|
104
|
-
when
|
164
|
+
when @queue_class
|
105
165
|
message.push(:go_on)
|
106
166
|
else
|
107
167
|
message_buffer.add(message)
|
@@ -2,30 +2,65 @@
|
|
2
2
|
|
3
3
|
module Datadog
|
4
4
|
class Statsd
|
5
|
+
# The SingleThreadSender is a sender synchronously buffering messages
|
6
|
+
# in a `MessageBuffer`.
|
7
|
+
# It is using current Process.PID to check it is the result of a recent fork
|
8
|
+
# and it is reseting the MessageBuffer if that's the case.
|
5
9
|
class SingleThreadSender
|
6
|
-
def initialize(message_buffer)
|
10
|
+
def initialize(message_buffer, logger: nil, flush_interval: nil)
|
7
11
|
@message_buffer = message_buffer
|
12
|
+
@logger = logger
|
13
|
+
@mx = Mutex.new
|
14
|
+
if flush_interval
|
15
|
+
@flush_timer = Datadog::Statsd::Timer.new(flush_interval) { flush }
|
16
|
+
end
|
17
|
+
# store the pid for which this sender has been created
|
18
|
+
update_fork_pid
|
8
19
|
end
|
9
20
|
|
10
21
|
def add(message)
|
11
|
-
@
|
22
|
+
@mx.synchronize {
|
23
|
+
# we have just forked, meaning we have messages in the buffer that we should
|
24
|
+
# not send, they belong to the parent process, let's clear the buffer.
|
25
|
+
if forked?
|
26
|
+
@message_buffer.reset
|
27
|
+
@flush_timer.start if @flush_timer && @flush_timer.stop?
|
28
|
+
update_fork_pid
|
29
|
+
end
|
30
|
+
@message_buffer.add(message)
|
31
|
+
}
|
12
32
|
end
|
13
33
|
|
14
34
|
def flush(*)
|
15
|
-
@
|
35
|
+
@mx.synchronize {
|
36
|
+
@message_buffer.flush()
|
37
|
+
}
|
16
38
|
end
|
17
39
|
|
18
|
-
# Compatibility with `Sender`
|
19
40
|
def start()
|
41
|
+
@flush_timer.start if @flush_timer
|
20
42
|
end
|
21
43
|
|
22
|
-
# Compatibility with `Sender`
|
23
44
|
def stop()
|
45
|
+
@flush_timer.stop if @flush_timer
|
24
46
|
end
|
25
47
|
|
26
48
|
# Compatibility with `Sender`
|
27
49
|
def rendez_vous()
|
28
50
|
end
|
51
|
+
|
52
|
+
private
|
53
|
+
|
54
|
+
# below are "fork management" methods to be able to clean the MessageBuffer
|
55
|
+
# if it detects that it is running in a unknown PID.
|
56
|
+
|
57
|
+
def forked?
|
58
|
+
Process.pid != @fork_pid
|
59
|
+
end
|
60
|
+
|
61
|
+
def update_fork_pid
|
62
|
+
@fork_pid = Process.pid
|
63
|
+
end
|
29
64
|
end
|
30
65
|
end
|
31
66
|
end
|
@@ -9,8 +9,12 @@ module Datadog
|
|
9
9
|
attr_reader :service_checks
|
10
10
|
attr_reader :bytes_sent
|
11
11
|
attr_reader :bytes_dropped
|
12
|
+
attr_reader :bytes_dropped_queue
|
13
|
+
attr_reader :bytes_dropped_writer
|
12
14
|
attr_reader :packets_sent
|
13
15
|
attr_reader :packets_dropped
|
16
|
+
attr_reader :packets_dropped_queue
|
17
|
+
attr_reader :packets_dropped_writer
|
14
18
|
|
15
19
|
# Rough estimation of maximum telemetry message size without tags
|
16
20
|
MAX_TELEMETRY_MESSAGE_SIZE_WT_TAGS = 50 # bytes
|
@@ -40,8 +44,12 @@ module Datadog
|
|
40
44
|
@service_checks = 0
|
41
45
|
@bytes_sent = 0
|
42
46
|
@bytes_dropped = 0
|
47
|
+
@bytes_dropped_queue = 0
|
48
|
+
@bytes_dropped_writer = 0
|
43
49
|
@packets_sent = 0
|
44
50
|
@packets_dropped = 0
|
51
|
+
@packets_dropped_queue = 0
|
52
|
+
@packets_dropped_writer = 0
|
45
53
|
@next_flush_time = now_in_s + @flush_interval
|
46
54
|
end
|
47
55
|
|
@@ -54,9 +62,18 @@ module Datadog
|
|
54
62
|
@packets_sent += packets
|
55
63
|
end
|
56
64
|
|
57
|
-
def
|
65
|
+
def dropped_queue(bytes: 0, packets: 0)
|
58
66
|
@bytes_dropped += bytes
|
67
|
+
@bytes_dropped_queue += bytes
|
59
68
|
@packets_dropped += packets
|
69
|
+
@packets_dropped_queue += packets
|
70
|
+
end
|
71
|
+
|
72
|
+
def dropped_writer(bytes: 0, packets: 0)
|
73
|
+
@bytes_dropped += bytes
|
74
|
+
@bytes_dropped_writer += bytes
|
75
|
+
@packets_dropped += packets
|
76
|
+
@packets_dropped_writer += packets
|
60
77
|
end
|
61
78
|
|
62
79
|
def should_flush?
|
@@ -70,8 +87,12 @@ module Datadog
|
|
70
87
|
sprintf(pattern, 'service_checks', @service_checks),
|
71
88
|
sprintf(pattern, 'bytes_sent', @bytes_sent),
|
72
89
|
sprintf(pattern, 'bytes_dropped', @bytes_dropped),
|
90
|
+
sprintf(pattern, 'bytes_dropped_queue', @bytes_dropped_queue),
|
91
|
+
sprintf(pattern, 'bytes_dropped_writer', @bytes_dropped_writer),
|
73
92
|
sprintf(pattern, 'packets_sent', @packets_sent),
|
74
93
|
sprintf(pattern, 'packets_dropped', @packets_dropped),
|
94
|
+
sprintf(pattern, 'packets_dropped_queue', @packets_dropped_queue),
|
95
|
+
sprintf(pattern, 'packets_dropped_writer', @packets_dropped_writer),
|
75
96
|
]
|
76
97
|
end
|
77
98
|
|
@@ -0,0 +1,60 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Datadog
|
4
|
+
class Statsd
|
5
|
+
class Timer
|
6
|
+
def initialize(interval, &callback)
|
7
|
+
@mx = Mutex.new
|
8
|
+
@cv = ConditionVariable.new
|
9
|
+
@interval = interval
|
10
|
+
@callback = callback
|
11
|
+
@stop = true
|
12
|
+
end
|
13
|
+
|
14
|
+
def start
|
15
|
+
return unless stop?
|
16
|
+
|
17
|
+
@stop = false
|
18
|
+
@thread = Thread.new do
|
19
|
+
last_execution_time = current_time
|
20
|
+
@mx.synchronize do
|
21
|
+
until @stop
|
22
|
+
timeout = @interval - (current_time - last_execution_time)
|
23
|
+
@cv.wait(@mx, timeout > 0 ? timeout : 0)
|
24
|
+
last_execution_time = current_time
|
25
|
+
@callback.call
|
26
|
+
end
|
27
|
+
end
|
28
|
+
end
|
29
|
+
@thread.name = 'Statsd Timer' unless Gem::Version.new(RUBY_VERSION) < Gem::Version.new('2.3')
|
30
|
+
end
|
31
|
+
|
32
|
+
def stop
|
33
|
+
return if @thread.nil?
|
34
|
+
|
35
|
+
@stop = true
|
36
|
+
@mx.synchronize do
|
37
|
+
@cv.signal
|
38
|
+
end
|
39
|
+
@thread.join
|
40
|
+
@thread = nil
|
41
|
+
end
|
42
|
+
|
43
|
+
def stop?
|
44
|
+
@thread.nil? || @thread.stop?
|
45
|
+
end
|
46
|
+
|
47
|
+
private
|
48
|
+
|
49
|
+
if Process.const_defined?(:CLOCK_MONOTONIC)
|
50
|
+
def current_time
|
51
|
+
Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
52
|
+
end
|
53
|
+
else
|
54
|
+
def current_time
|
55
|
+
Time.now
|
56
|
+
end
|
57
|
+
end
|
58
|
+
end
|
59
|
+
end
|
60
|
+
end
|
@@ -5,32 +5,41 @@ require_relative 'connection'
|
|
5
5
|
module Datadog
|
6
6
|
class Statsd
|
7
7
|
class UDPConnection < Connection
|
8
|
-
|
9
|
-
DEFAULT_PORT = 8125
|
10
|
-
|
11
|
-
# StatsD host. Defaults to 127.0.0.1.
|
8
|
+
# StatsD host.
|
12
9
|
attr_reader :host
|
13
10
|
|
14
|
-
# StatsD port.
|
11
|
+
# StatsD port.
|
15
12
|
attr_reader :port
|
16
13
|
|
17
14
|
def initialize(host, port, **kwargs)
|
18
15
|
super(**kwargs)
|
19
16
|
|
20
|
-
@host = host
|
21
|
-
@port = port
|
17
|
+
@host = host
|
18
|
+
@port = port
|
19
|
+
@socket = nil
|
20
|
+
end
|
21
|
+
|
22
|
+
def close
|
23
|
+
@socket.close if @socket
|
24
|
+
@socket = nil
|
22
25
|
end
|
23
26
|
|
24
27
|
private
|
25
28
|
|
26
29
|
def connect
|
27
|
-
|
28
|
-
|
29
|
-
|
30
|
+
close if @socket
|
31
|
+
|
32
|
+
@socket = UDPSocket.new
|
33
|
+
@socket.connect(host, port)
|
30
34
|
end
|
31
35
|
|
36
|
+
# send_message is writing the message in the socket, it may create the socket if nil
|
37
|
+
# It is not thread-safe but since it is called by either the Sender bg thread or the
|
38
|
+
# SingleThreadSender (which is using a mutex while Flushing), only one thread must call
|
39
|
+
# it at a time.
|
32
40
|
def send_message(message)
|
33
|
-
socket
|
41
|
+
connect unless @socket
|
42
|
+
@socket.send(message, 0)
|
34
43
|
end
|
35
44
|
end
|
36
45
|
end
|
@@ -14,20 +14,31 @@ module Datadog
|
|
14
14
|
super(**kwargs)
|
15
15
|
|
16
16
|
@socket_path = socket_path
|
17
|
+
@socket = nil
|
18
|
+
end
|
19
|
+
|
20
|
+
def close
|
21
|
+
@socket.close if @socket
|
22
|
+
@socket = nil
|
17
23
|
end
|
18
24
|
|
19
25
|
private
|
20
26
|
|
21
27
|
def connect
|
22
|
-
|
23
|
-
|
24
|
-
socket
|
28
|
+
close if @socket
|
29
|
+
|
30
|
+
@socket = Socket.new(Socket::AF_UNIX, Socket::SOCK_DGRAM)
|
31
|
+
@socket.connect(Socket.pack_sockaddr_un(@socket_path))
|
25
32
|
end
|
26
33
|
|
34
|
+
# send_message is writing the message in the socket, it may create the socket if nil
|
35
|
+
# It is not thread-safe but since it is called by either the Sender bg thread or the
|
36
|
+
# SingleThreadSender (which is using a mutex while Flushing), only one thread must call
|
37
|
+
# it at a time.
|
27
38
|
def send_message(message)
|
28
|
-
socket
|
39
|
+
connect unless @socket
|
40
|
+
@socket.sendmsg_nonblock(message)
|
29
41
|
rescue Errno::ECONNREFUSED, Errno::ECONNRESET, Errno::ENOENT => e
|
30
|
-
@socket = nil
|
31
42
|
# TODO: FIXME: This error should be considered as a retryable error in the
|
32
43
|
# Connection class. An even better solution would be to make BadSocketError inherit
|
33
44
|
# from a specific retryable error class in the Connection class.
|
data/lib/datadog/statsd.rb
CHANGED
@@ -5,11 +5,16 @@ require_relative 'statsd/version'
|
|
5
5
|
require_relative 'statsd/telemetry'
|
6
6
|
require_relative 'statsd/udp_connection'
|
7
7
|
require_relative 'statsd/uds_connection'
|
8
|
+
require_relative 'statsd/connection_cfg'
|
8
9
|
require_relative 'statsd/message_buffer'
|
9
10
|
require_relative 'statsd/serialization'
|
10
11
|
require_relative 'statsd/sender'
|
11
12
|
require_relative 'statsd/single_thread_sender'
|
12
13
|
require_relative 'statsd/forwarder'
|
14
|
+
require_relative 'statsd/timer'
|
15
|
+
|
16
|
+
$deprecation_message_mutex = Mutex.new
|
17
|
+
$deprecation_message_done = false
|
13
18
|
|
14
19
|
# = Datadog::Statsd: A DogStatsd client (https://www.datadoghq.com)
|
15
20
|
#
|
@@ -40,7 +45,12 @@ module Datadog
|
|
40
45
|
UDP_DEFAULT_BUFFER_SIZE = 1_432
|
41
46
|
UDS_DEFAULT_BUFFER_SIZE = 8_192
|
42
47
|
DEFAULT_BUFFER_POOL_SIZE = Float::INFINITY
|
48
|
+
|
49
|
+
UDP_DEFAULT_SENDER_QUEUE_SIZE = 2048
|
50
|
+
UDS_DEFAULT_SENDER_QUEUE_SIZE = 512
|
51
|
+
|
43
52
|
MAX_EVENT_SIZE = 8 * 1_024
|
53
|
+
|
44
54
|
# minimum flush interval for the telemetry in seconds
|
45
55
|
DEFAULT_TELEMETRY_FLUSH_INTERVAL = 10
|
46
56
|
|
@@ -69,6 +79,8 @@ module Datadog
|
|
69
79
|
# @option [Logger] logger for debugging
|
70
80
|
# @option [Integer] buffer_max_payload_size max bytes to buffer
|
71
81
|
# @option [Integer] buffer_max_pool_size max messages to buffer
|
82
|
+
# @option [Integer] sender_queue_size size of the sender queue in number of buffers (multi-thread only)
|
83
|
+
# @option [Numeric] buffer_flush_interval interval in second to flush buffer
|
72
84
|
# @option [String] socket_path unix socket path
|
73
85
|
# @option [Float] default sample rate if not overridden
|
74
86
|
# @option [Boolean] single_thread flushes the metrics on the main thread instead of in a companion thread
|
@@ -84,6 +96,9 @@ module Datadog
|
|
84
96
|
buffer_max_payload_size: nil,
|
85
97
|
buffer_max_pool_size: nil,
|
86
98
|
buffer_overflowing_stategy: :drop,
|
99
|
+
buffer_flush_interval: nil,
|
100
|
+
|
101
|
+
sender_queue_size: nil,
|
87
102
|
|
88
103
|
logger: nil,
|
89
104
|
|
@@ -101,10 +116,25 @@ module Datadog
|
|
101
116
|
@serializer = Serialization::Serializer.new(prefix: @prefix, global_tags: tags)
|
102
117
|
@sample_rate = sample_rate
|
103
118
|
|
119
|
+
# deprecation message for ruby < 2.1.0 users as we will drop support for ruby 2.0
|
120
|
+
# in dogstatsd-ruby 5.4.0
|
121
|
+
# TODO(remy): remove this message and the two global vars used in dogstatd-ruby 5.4.0
|
122
|
+
if RUBY_VERSION < '2.1.0' && $deprecation_message_mutex.try_lock && !$deprecation_message_done
|
123
|
+
if logger != nil
|
124
|
+
logger.warn { "deprecation: dogstatsd-ruby will drop support of Ruby < 2.1.0 in a next minor release" }
|
125
|
+
else
|
126
|
+
puts("warning: deprecation: dogstatsd-ruby will drop support of Ruby < 2.1.0 in a next minor release")
|
127
|
+
end
|
128
|
+
$deprecation_message_done = true
|
129
|
+
$deprecation_message_mutex.unlock
|
130
|
+
end
|
131
|
+
|
104
132
|
@forwarder = Forwarder.new(
|
105
|
-
|
106
|
-
|
107
|
-
|
133
|
+
connection_cfg: ConnectionCfg.new(
|
134
|
+
host: host,
|
135
|
+
port: port,
|
136
|
+
socket_path: socket_path,
|
137
|
+
),
|
108
138
|
|
109
139
|
global_tags: tags,
|
110
140
|
logger: logger,
|
@@ -114,6 +144,9 @@ module Datadog
|
|
114
144
|
buffer_max_payload_size: buffer_max_payload_size,
|
115
145
|
buffer_max_pool_size: buffer_max_pool_size,
|
116
146
|
buffer_overflowing_stategy: buffer_overflowing_stategy,
|
147
|
+
buffer_flush_interval: buffer_flush_interval,
|
148
|
+
|
149
|
+
sender_queue_size: sender_queue_size,
|
117
150
|
|
118
151
|
telemetry_flush_interval: telemetry_enable ? telemetry_flush_interval : nil,
|
119
152
|
)
|
metadata
CHANGED
@@ -1,15 +1,15 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: dogstatsd-ruby
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 5.
|
4
|
+
version: 5.3.3
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Rein Henrichs
|
8
8
|
- Karim Bogtob
|
9
|
-
autorequire:
|
9
|
+
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date:
|
12
|
+
date: 2022-02-02 00:00:00.000000000 Z
|
13
13
|
dependencies: []
|
14
14
|
description: A Ruby DogStatsd client
|
15
15
|
email: code@datadoghq.com
|
@@ -23,6 +23,7 @@ files:
|
|
23
23
|
- README.md
|
24
24
|
- lib/datadog/statsd.rb
|
25
25
|
- lib/datadog/statsd/connection.rb
|
26
|
+
- lib/datadog/statsd/connection_cfg.rb
|
26
27
|
- lib/datadog/statsd/forwarder.rb
|
27
28
|
- lib/datadog/statsd/message_buffer.rb
|
28
29
|
- lib/datadog/statsd/sender.rb
|
@@ -34,6 +35,7 @@ files:
|
|
34
35
|
- lib/datadog/statsd/serialization/tag_serializer.rb
|
35
36
|
- lib/datadog/statsd/single_thread_sender.rb
|
36
37
|
- lib/datadog/statsd/telemetry.rb
|
38
|
+
- lib/datadog/statsd/timer.rb
|
37
39
|
- lib/datadog/statsd/udp_connection.rb
|
38
40
|
- lib/datadog/statsd/uds_connection.rb
|
39
41
|
- lib/datadog/statsd/version.rb
|
@@ -42,10 +44,14 @@ licenses:
|
|
42
44
|
- MIT
|
43
45
|
metadata:
|
44
46
|
bug_tracker_uri: https://github.com/DataDog/dogstatsd-ruby/issues
|
45
|
-
changelog_uri: https://github.com/DataDog/dogstatsd-ruby/blob/v5.
|
46
|
-
documentation_uri: https://www.rubydoc.info/gems/dogstatsd-ruby/5.
|
47
|
-
source_code_uri: https://github.com/DataDog/dogstatsd-ruby/tree/v5.
|
48
|
-
post_install_message:
|
47
|
+
changelog_uri: https://github.com/DataDog/dogstatsd-ruby/blob/v5.3.3/CHANGELOG.md
|
48
|
+
documentation_uri: https://www.rubydoc.info/gems/dogstatsd-ruby/5.3.3
|
49
|
+
source_code_uri: https://github.com/DataDog/dogstatsd-ruby/tree/v5.3.3
|
50
|
+
post_install_message: |2+
|
51
|
+
|
52
|
+
If you are upgrading from v4.x of the dogstatsd-ruby library, note the major change to the threading model:
|
53
|
+
https://github.com/DataDog/dogstatsd-ruby#migrating-from-v4x-to-v5x
|
54
|
+
|
49
55
|
rdoc_options: []
|
50
56
|
require_paths:
|
51
57
|
- lib
|
@@ -60,9 +66,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
60
66
|
- !ruby/object:Gem::Version
|
61
67
|
version: '0'
|
62
68
|
requirements: []
|
63
|
-
|
64
|
-
|
65
|
-
signing_key:
|
69
|
+
rubygems_version: 3.2.22
|
70
|
+
signing_key:
|
66
71
|
specification_version: 4
|
67
72
|
summary: A Ruby DogStatsd client
|
68
73
|
test_files: []
|