dogstatsd-ruby 5.3.2 → 5.5.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +16 -3
- data/lib/datadog/statsd/connection.rb +1 -1
- data/lib/datadog/statsd/connection_cfg.rb +76 -0
- data/lib/datadog/statsd/forwarder.rb +28 -15
- data/lib/datadog/statsd/sender.rb +27 -7
- data/lib/datadog/statsd/single_thread_sender.rb +9 -3
- data/lib/datadog/statsd/telemetry.rb +22 -1
- data/lib/datadog/statsd/timer.rb +61 -0
- data/lib/datadog/statsd/udp_connection.rb +4 -7
- data/lib/datadog/statsd/version.rb +1 -1
- data/lib/datadog/statsd.rb +57 -33
- metadata +13 -9
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 92868c3d0ccb9a7847eaec3dd2da46413ab8acd2ce2df0008ee66c3e6500f3f6
|
4
|
+
data.tar.gz: 26064f00534e7df4a132d722041eea6b85a22849ca5c3c7caed05722320bcd9e
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: e9f076b5aa76f4102f66698d5d19a3fbcd84e2b8b3a5fa944dcac07ddf8ee25f48a65d90e5296cbd7327f04dc90a8aba0234606bc50ab7c335120e9e17c3a6d2
|
7
|
+
data.tar.gz: e07f4469b7737fe7d67d32623194c90e09adf3d78ef0c26226bfed103db4067fa01e582f0627d426695dd9a0ff93c4e1e7d79d227bd5ce806d9ca28b2a440382
|
data/README.md
CHANGED
@@ -81,8 +81,6 @@ Version v5.x of `dogstatsd-ruby` is using a sender thread for flushing. This pro
|
|
81
81
|
|
82
82
|
If you are using [Sidekiq](https://github.com/mperham/sidekiq), please make sure to close the client instances that are instantiated. [See this example on using DogStatsD-ruby v5.x with Sidekiq](https://github.com/DataDog/dogstatsd-ruby/blob/master/examples/sidekiq_example.rb).
|
83
83
|
|
84
|
-
If you are using [Puma](https://github.com/puma/puma) or [Unicorn](https://yhbt.net/unicorn.git), please make sure to create the instance of DogStatsD in the workers, not in the main process before it forks to create its workers. See [this comment for more details](https://github.com/DataDog/dogstatsd-ruby/issues/179#issuecomment-845570345).
|
85
|
-
|
86
84
|
Applications that run into issues but can't apply these recommendations should use the `single_thread` mode which disables the use of the sender thread.
|
87
85
|
Here is how to instantiate a client in this mode:
|
88
86
|
|
@@ -180,6 +178,15 @@ There are three different kinds of messages:
|
|
180
178
|
|
181
179
|
There is also an implicit message which closes the queue which will cause the sender thread to finish processing and exit.
|
182
180
|
|
181
|
+
|
182
|
+
```ruby
|
183
|
+
statsd = Datadog::Statsd.new('localhost', 8125)
|
184
|
+
```
|
185
|
+
|
186
|
+
The message queue's maximum size (in messages) is given by the `sender_queue_size` argument, and has appropriate defaults for UDP (2048) and UDS (512).
|
187
|
+
|
188
|
+
The `buffer_flush_interval`, if enabled, is implemented with an additional thread which manages the timing of those flushes. This additional thread is used even if `single_thread: true`.
|
189
|
+
|
183
190
|
### Usual workflow
|
184
191
|
|
185
192
|
You push metrics to the statsd client which writes them quickly to the sender message queue. The sender thread receives those message, buffers them and flushes them to the connection when the buffer limit is reached.
|
@@ -204,7 +211,13 @@ When using the `single_thread: true` mode, instances of `Datadog::Statsd` are st
|
|
204
211
|
|
205
212
|
## Versioning
|
206
213
|
|
207
|
-
This Ruby gem is using [Semantic Versioning](https://guides.rubygems.org/patterns/#semantic-versioning) but please note that supported Ruby versions can change in a minor release of this library.
|
214
|
+
This Ruby gem is using [Semantic Versioning](https://guides.rubygems.org/patterns/#semantic-versioning) but please note that supported Ruby versions can change in a minor release of this library.
|
215
|
+
As much as possible, we will add a "future deprecation" message in the minor release preceding the one dropping the support.
|
216
|
+
|
217
|
+
## Ruby Versions
|
218
|
+
|
219
|
+
This gem supports and is tested on Ruby minor versions 2.1 through 3.1.
|
220
|
+
Support for Ruby 2.0 was dropped in version 5.4.0.
|
208
221
|
|
209
222
|
## Credits
|
210
223
|
|
@@ -0,0 +1,76 @@
|
|
1
|
+
module Datadog
|
2
|
+
class Statsd
|
3
|
+
class ConnectionCfg
|
4
|
+
attr_reader :host
|
5
|
+
attr_reader :port
|
6
|
+
attr_reader :socket_path
|
7
|
+
attr_reader :transport_type
|
8
|
+
|
9
|
+
def initialize(host: nil, port: nil, socket_path: nil)
|
10
|
+
initialize_with_constructor_args(host: host, port: port, socket_path: socket_path) ||
|
11
|
+
initialize_with_env_vars ||
|
12
|
+
initialize_with_defaults
|
13
|
+
end
|
14
|
+
|
15
|
+
def make_connection(**params)
|
16
|
+
case @transport_type
|
17
|
+
when :udp
|
18
|
+
UDPConnection.new(@host, @port, **params)
|
19
|
+
when :uds
|
20
|
+
UDSConnection.new(@socket_path, **params)
|
21
|
+
end
|
22
|
+
end
|
23
|
+
|
24
|
+
private
|
25
|
+
|
26
|
+
DEFAULT_HOST = '127.0.0.1'
|
27
|
+
DEFAULT_PORT = 8125
|
28
|
+
|
29
|
+
def initialize_with_constructor_args(host: nil, port: nil, socket_path: nil)
|
30
|
+
try_initialize_with(host: host, port: port, socket_path: socket_path,
|
31
|
+
not_both_error_message:
|
32
|
+
"Both UDP: (host/port #{host}:#{port}) and UDS (socket_path #{socket_path}) " +
|
33
|
+
"constructor arguments were given. Use only one or the other.",
|
34
|
+
)
|
35
|
+
end
|
36
|
+
|
37
|
+
def initialize_with_env_vars()
|
38
|
+
try_initialize_with(
|
39
|
+
host: ENV['DD_AGENT_HOST'],
|
40
|
+
port: ENV['DD_DOGSTATSD_PORT'] && ENV['DD_DOGSTATSD_PORT'].to_i,
|
41
|
+
socket_path: ENV['DD_DOGSTATSD_SOCKET'],
|
42
|
+
not_both_error_message:
|
43
|
+
"Both UDP (DD_AGENT_HOST/DD_DOGSTATSD_PORT #{ENV['DD_AGENT_HOST']}:#{ENV['DD_DOGSTATSD_PORT']}) " +
|
44
|
+
"and UDS (DD_DOGSTATSD_SOCKET #{ENV['DD_DOGSTATSD_SOCKET']}) environment variables are set. " +
|
45
|
+
"Set only one or the other.",
|
46
|
+
)
|
47
|
+
end
|
48
|
+
|
49
|
+
def initialize_with_defaults()
|
50
|
+
try_initialize_with(host: DEFAULT_HOST, port: DEFAULT_PORT)
|
51
|
+
end
|
52
|
+
|
53
|
+
def try_initialize_with(host: nil, port: nil, socket_path: nil, not_both_error_message: "")
|
54
|
+
if (host || port) && socket_path
|
55
|
+
raise ArgumentError, not_both_error_message
|
56
|
+
end
|
57
|
+
|
58
|
+
if host || port
|
59
|
+
@host = host || DEFAULT_HOST
|
60
|
+
@port = port || DEFAULT_PORT
|
61
|
+
@socket_path = nil
|
62
|
+
@transport_type = :udp
|
63
|
+
return true
|
64
|
+
elsif socket_path
|
65
|
+
@host = nil
|
66
|
+
@port = nil
|
67
|
+
@socket_path = socket_path
|
68
|
+
@transport_type = :uds
|
69
|
+
return true
|
70
|
+
end
|
71
|
+
|
72
|
+
return false
|
73
|
+
end
|
74
|
+
end
|
75
|
+
end
|
76
|
+
end
|
@@ -7,13 +7,14 @@ module Datadog
|
|
7
7
|
attr_reader :transport_type
|
8
8
|
|
9
9
|
def initialize(
|
10
|
-
|
11
|
-
port: nil,
|
12
|
-
socket_path: nil,
|
10
|
+
connection_cfg: nil,
|
13
11
|
|
14
12
|
buffer_max_payload_size: nil,
|
15
13
|
buffer_max_pool_size: nil,
|
16
14
|
buffer_overflowing_stategy: :drop,
|
15
|
+
buffer_flush_interval: nil,
|
16
|
+
|
17
|
+
sender_queue_size: nil,
|
17
18
|
|
18
19
|
telemetry_flush_interval: nil,
|
19
20
|
global_tags: [],
|
@@ -22,24 +23,22 @@ module Datadog
|
|
22
23
|
|
23
24
|
logger: nil
|
24
25
|
)
|
25
|
-
@transport_type =
|
26
|
+
@transport_type = connection_cfg.transport_type
|
26
27
|
|
27
|
-
if telemetry_flush_interval
|
28
|
-
|
28
|
+
@telemetry = if telemetry_flush_interval
|
29
|
+
Telemetry.new(telemetry_flush_interval,
|
29
30
|
global_tags: global_tags,
|
30
|
-
transport_type: transport_type
|
31
|
+
transport_type: @transport_type
|
31
32
|
)
|
33
|
+
else
|
34
|
+
nil
|
32
35
|
end
|
33
36
|
|
34
|
-
@connection =
|
35
|
-
when :udp
|
36
|
-
UDPConnection.new(host, port, logger: logger, telemetry: telemetry)
|
37
|
-
when :uds
|
38
|
-
UDSConnection.new(socket_path, logger: logger, telemetry: telemetry)
|
39
|
-
end
|
37
|
+
@connection = connection_cfg.make_connection(logger: logger, telemetry: telemetry)
|
40
38
|
|
41
39
|
# Initialize buffer
|
42
|
-
buffer_max_payload_size ||= (transport_type == :udp ?
|
40
|
+
buffer_max_payload_size ||= (@transport_type == :udp ?
|
41
|
+
UDP_DEFAULT_BUFFER_SIZE : UDS_DEFAULT_BUFFER_SIZE)
|
43
42
|
|
44
43
|
if buffer_max_payload_size <= 0
|
45
44
|
raise ArgumentError, 'buffer_max_payload_size cannot be <= 0'
|
@@ -54,7 +53,21 @@ module Datadog
|
|
54
53
|
max_pool_size: buffer_max_pool_size || DEFAULT_BUFFER_POOL_SIZE,
|
55
54
|
overflowing_stategy: buffer_overflowing_stategy,
|
56
55
|
)
|
57
|
-
|
56
|
+
|
57
|
+
sender_queue_size ||= (@transport_type == :udp ?
|
58
|
+
UDP_DEFAULT_SENDER_QUEUE_SIZE : UDS_DEFAULT_SENDER_QUEUE_SIZE)
|
59
|
+
|
60
|
+
@sender = single_thread ?
|
61
|
+
SingleThreadSender.new(
|
62
|
+
buffer,
|
63
|
+
logger: logger,
|
64
|
+
flush_interval: buffer_flush_interval) :
|
65
|
+
Sender.new(
|
66
|
+
buffer,
|
67
|
+
logger: logger,
|
68
|
+
flush_interval: buffer_flush_interval,
|
69
|
+
telemetry: @telemetry,
|
70
|
+
queue_size: sender_queue_size)
|
58
71
|
@sender.start
|
59
72
|
end
|
60
73
|
|
@@ -12,10 +12,19 @@ module Datadog
|
|
12
12
|
class Sender
|
13
13
|
CLOSEABLE_QUEUES = Queue.instance_methods.include?(:close)
|
14
14
|
|
15
|
-
def initialize(message_buffer, logger: nil)
|
15
|
+
def initialize(message_buffer, telemetry: nil, queue_size: UDP_DEFAULT_BUFFER_SIZE, logger: nil, flush_interval: nil, queue_class: Queue, thread_class: Thread)
|
16
16
|
@message_buffer = message_buffer
|
17
|
+
@telemetry = telemetry
|
18
|
+
@queue_size = queue_size
|
17
19
|
@logger = logger
|
18
20
|
@mx = Mutex.new
|
21
|
+
@queue_class = queue_class
|
22
|
+
@thread_class = thread_class
|
23
|
+
@flush_timer = if flush_interval
|
24
|
+
Datadog::Statsd::Timer.new(flush_interval) { flush(sync: true) }
|
25
|
+
else
|
26
|
+
nil
|
27
|
+
end
|
19
28
|
end
|
20
29
|
|
21
30
|
def flush(sync: false)
|
@@ -42,7 +51,7 @@ module Datadog
|
|
42
51
|
return unless message_queue
|
43
52
|
|
44
53
|
# Initialize and get the thread's sync queue
|
45
|
-
queue = (
|
54
|
+
queue = (@thread_class.current[:statsd_sync_queue] ||= @queue_class.new)
|
46
55
|
# tell sender-thread to notify us in the current
|
47
56
|
# thread's queue
|
48
57
|
message_queue.push(queue)
|
@@ -68,19 +77,26 @@ module Datadog
|
|
68
77
|
@message_queue = nil
|
69
78
|
message_buffer.reset
|
70
79
|
start
|
80
|
+
@flush_timer.start if @flush_timer && @flush_timer.stop?
|
71
81
|
}
|
72
82
|
end
|
73
83
|
|
74
|
-
message_queue
|
84
|
+
if message_queue.length <= @queue_size
|
85
|
+
message_queue << message
|
86
|
+
else
|
87
|
+
@telemetry.dropped_queue(packets: 1, bytes: message.bytesize) if @telemetry
|
88
|
+
end
|
75
89
|
end
|
76
90
|
|
77
91
|
def start
|
78
92
|
raise ArgumentError, 'Sender already started' if message_queue
|
79
93
|
|
80
94
|
# initialize a new message queue for the background thread
|
81
|
-
@message_queue =
|
95
|
+
@message_queue = @queue_class.new
|
82
96
|
# start background thread
|
83
|
-
@sender_thread =
|
97
|
+
@sender_thread = @thread_class.new(&method(:send_loop))
|
98
|
+
@sender_thread.name = "Statsd Sender" unless Gem::Version.new(RUBY_VERSION) < Gem::Version.new('2.3')
|
99
|
+
@flush_timer.start if @flush_timer
|
84
100
|
end
|
85
101
|
|
86
102
|
if CLOSEABLE_QUEUES
|
@@ -88,6 +104,8 @@ module Datadog
|
|
88
104
|
# to close the sender nor trying to continue to `#add` more message
|
89
105
|
# into the sender.
|
90
106
|
def stop(join_worker: true)
|
107
|
+
@flush_timer.stop if @flush_timer
|
108
|
+
|
91
109
|
message_queue = @message_queue
|
92
110
|
message_queue.close if message_queue
|
93
111
|
|
@@ -99,6 +117,8 @@ module Datadog
|
|
99
117
|
# to close the sender nor trying to continue to `#add` more message
|
100
118
|
# into the sender.
|
101
119
|
def stop(join_worker: true)
|
120
|
+
@flush_timer.stop if @flush_timer
|
121
|
+
|
102
122
|
message_queue = @message_queue
|
103
123
|
message_queue << :close if message_queue
|
104
124
|
|
@@ -123,7 +143,7 @@ module Datadog
|
|
123
143
|
case message
|
124
144
|
when :flush
|
125
145
|
message_buffer.flush
|
126
|
-
when
|
146
|
+
when @queue_class
|
127
147
|
message.push(:go_on)
|
128
148
|
else
|
129
149
|
message_buffer.add(message)
|
@@ -145,7 +165,7 @@ module Datadog
|
|
145
165
|
break
|
146
166
|
when :flush
|
147
167
|
message_buffer.flush
|
148
|
-
when
|
168
|
+
when @queue_class
|
149
169
|
message.push(:go_on)
|
150
170
|
else
|
151
171
|
message_buffer.add(message)
|
@@ -7,10 +7,15 @@ module Datadog
|
|
7
7
|
# It is using current Process.PID to check it is the result of a recent fork
|
8
8
|
# and it is reseting the MessageBuffer if that's the case.
|
9
9
|
class SingleThreadSender
|
10
|
-
def initialize(message_buffer, logger: nil)
|
10
|
+
def initialize(message_buffer, logger: nil, flush_interval: nil)
|
11
11
|
@message_buffer = message_buffer
|
12
12
|
@logger = logger
|
13
13
|
@mx = Mutex.new
|
14
|
+
@flush_timer = if flush_interval
|
15
|
+
Datadog::Statsd::Timer.new(flush_interval) { flush }
|
16
|
+
else
|
17
|
+
nil
|
18
|
+
end
|
14
19
|
# store the pid for which this sender has been created
|
15
20
|
update_fork_pid
|
16
21
|
end
|
@@ -21,6 +26,7 @@ module Datadog
|
|
21
26
|
# not send, they belong to the parent process, let's clear the buffer.
|
22
27
|
if forked?
|
23
28
|
@message_buffer.reset
|
29
|
+
@flush_timer.start if @flush_timer && @flush_timer.stop?
|
24
30
|
update_fork_pid
|
25
31
|
end
|
26
32
|
@message_buffer.add(message)
|
@@ -33,12 +39,12 @@ module Datadog
|
|
33
39
|
}
|
34
40
|
end
|
35
41
|
|
36
|
-
# Compatibility with `Sender`
|
37
42
|
def start()
|
43
|
+
@flush_timer.start if @flush_timer
|
38
44
|
end
|
39
45
|
|
40
|
-
# Compatibility with `Sender`
|
41
46
|
def stop()
|
47
|
+
@flush_timer.stop if @flush_timer
|
42
48
|
end
|
43
49
|
|
44
50
|
# Compatibility with `Sender`
|
@@ -9,8 +9,12 @@ module Datadog
|
|
9
9
|
attr_reader :service_checks
|
10
10
|
attr_reader :bytes_sent
|
11
11
|
attr_reader :bytes_dropped
|
12
|
+
attr_reader :bytes_dropped_queue
|
13
|
+
attr_reader :bytes_dropped_writer
|
12
14
|
attr_reader :packets_sent
|
13
15
|
attr_reader :packets_dropped
|
16
|
+
attr_reader :packets_dropped_queue
|
17
|
+
attr_reader :packets_dropped_writer
|
14
18
|
|
15
19
|
# Rough estimation of maximum telemetry message size without tags
|
16
20
|
MAX_TELEMETRY_MESSAGE_SIZE_WT_TAGS = 50 # bytes
|
@@ -40,8 +44,12 @@ module Datadog
|
|
40
44
|
@service_checks = 0
|
41
45
|
@bytes_sent = 0
|
42
46
|
@bytes_dropped = 0
|
47
|
+
@bytes_dropped_queue = 0
|
48
|
+
@bytes_dropped_writer = 0
|
43
49
|
@packets_sent = 0
|
44
50
|
@packets_dropped = 0
|
51
|
+
@packets_dropped_queue = 0
|
52
|
+
@packets_dropped_writer = 0
|
45
53
|
@next_flush_time = now_in_s + @flush_interval
|
46
54
|
end
|
47
55
|
|
@@ -54,9 +62,18 @@ module Datadog
|
|
54
62
|
@packets_sent += packets
|
55
63
|
end
|
56
64
|
|
57
|
-
def
|
65
|
+
def dropped_queue(bytes: 0, packets: 0)
|
58
66
|
@bytes_dropped += bytes
|
67
|
+
@bytes_dropped_queue += bytes
|
59
68
|
@packets_dropped += packets
|
69
|
+
@packets_dropped_queue += packets
|
70
|
+
end
|
71
|
+
|
72
|
+
def dropped_writer(bytes: 0, packets: 0)
|
73
|
+
@bytes_dropped += bytes
|
74
|
+
@bytes_dropped_writer += bytes
|
75
|
+
@packets_dropped += packets
|
76
|
+
@packets_dropped_writer += packets
|
60
77
|
end
|
61
78
|
|
62
79
|
def should_flush?
|
@@ -70,8 +87,12 @@ module Datadog
|
|
70
87
|
sprintf(pattern, 'service_checks', @service_checks),
|
71
88
|
sprintf(pattern, 'bytes_sent', @bytes_sent),
|
72
89
|
sprintf(pattern, 'bytes_dropped', @bytes_dropped),
|
90
|
+
sprintf(pattern, 'bytes_dropped_queue', @bytes_dropped_queue),
|
91
|
+
sprintf(pattern, 'bytes_dropped_writer', @bytes_dropped_writer),
|
73
92
|
sprintf(pattern, 'packets_sent', @packets_sent),
|
74
93
|
sprintf(pattern, 'packets_dropped', @packets_dropped),
|
94
|
+
sprintf(pattern, 'packets_dropped_queue', @packets_dropped_queue),
|
95
|
+
sprintf(pattern, 'packets_dropped_writer', @packets_dropped_writer),
|
75
96
|
]
|
76
97
|
end
|
77
98
|
|
@@ -0,0 +1,61 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Datadog
|
4
|
+
class Statsd
|
5
|
+
class Timer
|
6
|
+
def initialize(interval, &callback)
|
7
|
+
@mx = Mutex.new
|
8
|
+
@cv = ConditionVariable.new
|
9
|
+
@interval = interval
|
10
|
+
@callback = callback
|
11
|
+
@stop = true
|
12
|
+
@thread = nil
|
13
|
+
end
|
14
|
+
|
15
|
+
def start
|
16
|
+
return unless stop?
|
17
|
+
|
18
|
+
@stop = false
|
19
|
+
@thread = Thread.new do
|
20
|
+
last_execution_time = current_time
|
21
|
+
@mx.synchronize do
|
22
|
+
until @stop
|
23
|
+
timeout = @interval - (current_time - last_execution_time)
|
24
|
+
@cv.wait(@mx, timeout > 0 ? timeout : 0)
|
25
|
+
last_execution_time = current_time
|
26
|
+
@callback.call
|
27
|
+
end
|
28
|
+
end
|
29
|
+
end
|
30
|
+
@thread.name = 'Statsd Timer' unless Gem::Version.new(RUBY_VERSION) < Gem::Version.new('2.3')
|
31
|
+
end
|
32
|
+
|
33
|
+
def stop
|
34
|
+
return if @thread.nil?
|
35
|
+
|
36
|
+
@stop = true
|
37
|
+
@mx.synchronize do
|
38
|
+
@cv.signal
|
39
|
+
end
|
40
|
+
@thread.join
|
41
|
+
@thread = nil
|
42
|
+
end
|
43
|
+
|
44
|
+
def stop?
|
45
|
+
@thread.nil? || @thread.stop?
|
46
|
+
end
|
47
|
+
|
48
|
+
private
|
49
|
+
|
50
|
+
if Process.const_defined?(:CLOCK_MONOTONIC)
|
51
|
+
def current_time
|
52
|
+
Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
53
|
+
end
|
54
|
+
else
|
55
|
+
def current_time
|
56
|
+
Time.now
|
57
|
+
end
|
58
|
+
end
|
59
|
+
end
|
60
|
+
end
|
61
|
+
end
|
@@ -5,20 +5,17 @@ require_relative 'connection'
|
|
5
5
|
module Datadog
|
6
6
|
class Statsd
|
7
7
|
class UDPConnection < Connection
|
8
|
-
|
9
|
-
DEFAULT_PORT = 8125
|
10
|
-
|
11
|
-
# StatsD host. Defaults to 127.0.0.1.
|
8
|
+
# StatsD host.
|
12
9
|
attr_reader :host
|
13
10
|
|
14
|
-
# StatsD port.
|
11
|
+
# StatsD port.
|
15
12
|
attr_reader :port
|
16
13
|
|
17
14
|
def initialize(host, port, **kwargs)
|
18
15
|
super(**kwargs)
|
19
16
|
|
20
|
-
@host = host
|
21
|
-
@port = port
|
17
|
+
@host = host
|
18
|
+
@port = port
|
22
19
|
@socket = nil
|
23
20
|
end
|
24
21
|
|
data/lib/datadog/statsd.rb
CHANGED
@@ -5,14 +5,13 @@ require_relative 'statsd/version'
|
|
5
5
|
require_relative 'statsd/telemetry'
|
6
6
|
require_relative 'statsd/udp_connection'
|
7
7
|
require_relative 'statsd/uds_connection'
|
8
|
+
require_relative 'statsd/connection_cfg'
|
8
9
|
require_relative 'statsd/message_buffer'
|
9
10
|
require_relative 'statsd/serialization'
|
10
11
|
require_relative 'statsd/sender'
|
11
12
|
require_relative 'statsd/single_thread_sender'
|
12
13
|
require_relative 'statsd/forwarder'
|
13
|
-
|
14
|
-
$deprecation_message_mutex = Mutex.new
|
15
|
-
$deprecation_message_done = false
|
14
|
+
require_relative 'statsd/timer'
|
16
15
|
|
17
16
|
# = Datadog::Statsd: A DogStatsd client (https://www.datadoghq.com)
|
18
17
|
#
|
@@ -43,7 +42,12 @@ module Datadog
|
|
43
42
|
UDP_DEFAULT_BUFFER_SIZE = 1_432
|
44
43
|
UDS_DEFAULT_BUFFER_SIZE = 8_192
|
45
44
|
DEFAULT_BUFFER_POOL_SIZE = Float::INFINITY
|
45
|
+
|
46
|
+
UDP_DEFAULT_SENDER_QUEUE_SIZE = 2048
|
47
|
+
UDS_DEFAULT_SENDER_QUEUE_SIZE = 512
|
48
|
+
|
46
49
|
MAX_EVENT_SIZE = 8 * 1_024
|
50
|
+
|
47
51
|
# minimum flush interval for the telemetry in seconds
|
48
52
|
DEFAULT_TELEMETRY_FLUSH_INTERVAL = 10
|
49
53
|
|
@@ -72,6 +76,8 @@ module Datadog
|
|
72
76
|
# @option [Logger] logger for debugging
|
73
77
|
# @option [Integer] buffer_max_payload_size max bytes to buffer
|
74
78
|
# @option [Integer] buffer_max_pool_size max messages to buffer
|
79
|
+
# @option [Integer] sender_queue_size size of the sender queue in number of buffers (multi-thread only)
|
80
|
+
# @option [Numeric] buffer_flush_interval interval in second to flush buffer
|
75
81
|
# @option [String] socket_path unix socket path
|
76
82
|
# @option [Float] default sample rate if not overridden
|
77
83
|
# @option [Boolean] single_thread flushes the metrics on the main thread instead of in a companion thread
|
@@ -87,6 +93,9 @@ module Datadog
|
|
87
93
|
buffer_max_payload_size: nil,
|
88
94
|
buffer_max_pool_size: nil,
|
89
95
|
buffer_overflowing_stategy: :drop,
|
96
|
+
buffer_flush_interval: nil,
|
97
|
+
|
98
|
+
sender_queue_size: nil,
|
90
99
|
|
91
100
|
logger: nil,
|
92
101
|
|
@@ -104,23 +113,12 @@ module Datadog
|
|
104
113
|
@serializer = Serialization::Serializer.new(prefix: @prefix, global_tags: tags)
|
105
114
|
@sample_rate = sample_rate
|
106
115
|
|
107
|
-
# deprecation message for ruby < 2.1.0 users as we will drop support for ruby 2.0
|
108
|
-
# in dogstatsd-ruby 5.4.0
|
109
|
-
# TODO(remy): remove this message and the two global vars used in dogstatd-ruby 5.4.0
|
110
|
-
if RUBY_VERSION < '2.1.0' && $deprecation_message_mutex.try_lock && !$deprecation_message_done
|
111
|
-
if logger != nil
|
112
|
-
logger.warn { "deprecation: dogstatsd-ruby will drop support of Ruby < 2.1.0 in a next minor release" }
|
113
|
-
else
|
114
|
-
puts("warning: deprecation: dogstatsd-ruby will drop support of Ruby < 2.1.0 in a next minor release")
|
115
|
-
end
|
116
|
-
$deprecation_message_done = true
|
117
|
-
$deprecation_message_mutex.unlock
|
118
|
-
end
|
119
|
-
|
120
116
|
@forwarder = Forwarder.new(
|
121
|
-
|
122
|
-
|
123
|
-
|
117
|
+
connection_cfg: ConnectionCfg.new(
|
118
|
+
host: host,
|
119
|
+
port: port,
|
120
|
+
socket_path: socket_path,
|
121
|
+
),
|
124
122
|
|
125
123
|
global_tags: tags,
|
126
124
|
logger: logger,
|
@@ -130,6 +128,9 @@ module Datadog
|
|
130
128
|
buffer_max_payload_size: buffer_max_payload_size,
|
131
129
|
buffer_max_pool_size: buffer_max_pool_size,
|
132
130
|
buffer_overflowing_stategy: buffer_overflowing_stategy,
|
131
|
+
buffer_flush_interval: buffer_flush_interval,
|
132
|
+
|
133
|
+
sender_queue_size: sender_queue_size,
|
133
134
|
|
134
135
|
telemetry_flush_interval: telemetry_enable ? telemetry_flush_interval : nil,
|
135
136
|
)
|
@@ -137,12 +138,13 @@ module Datadog
|
|
137
138
|
|
138
139
|
# yield a new instance to a block and close it when done
|
139
140
|
# for short-term use-cases that don't want to close the socket manually
|
140
|
-
|
141
|
-
|
141
|
+
# TODO: replace with ... once we are on ruby 2.7
|
142
|
+
def self.open(*args, **kwargs)
|
143
|
+
instance = new(*args, **kwargs)
|
142
144
|
|
143
145
|
yield instance
|
144
146
|
ensure
|
145
|
-
instance.close
|
147
|
+
instance.close if instance
|
146
148
|
end
|
147
149
|
|
148
150
|
# Sends an increment (count = 1) for the given stat to the statsd server.
|
@@ -150,6 +152,7 @@ module Datadog
|
|
150
152
|
# @param [String] stat stat name
|
151
153
|
# @param [Hash] opts the options to create the metric with
|
152
154
|
# @option opts [Numeric] :sample_rate sample rate, 1 for always
|
155
|
+
# @option opts [Boolean] :pre_sampled If true, the client assumes the caller has already sampled metrics at :sample_rate, and doesn't perform sampling.
|
153
156
|
# @option opts [Array<String>] :tags An array of tags
|
154
157
|
# @option opts [Numeric] :by increment value, default 1
|
155
158
|
# @see #count
|
@@ -164,6 +167,7 @@ module Datadog
|
|
164
167
|
# @param [String] stat stat name
|
165
168
|
# @param [Hash] opts the options to create the metric with
|
166
169
|
# @option opts [Numeric] :sample_rate sample rate, 1 for always
|
170
|
+
# @option opts [Boolean] :pre_sampled If true, the client assumes the caller has already sampled metrics at :sample_rate, and doesn't perform sampling.
|
167
171
|
# @option opts [Array<String>] :tags An array of tags
|
168
172
|
# @option opts [Numeric] :by decrement value, default 1
|
169
173
|
# @see #count
|
@@ -179,13 +183,14 @@ module Datadog
|
|
179
183
|
# @param [Integer] count count
|
180
184
|
# @param [Hash] opts the options to create the metric with
|
181
185
|
# @option opts [Numeric] :sample_rate sample rate, 1 for always
|
186
|
+
# @option opts [Boolean] :pre_sampled If true, the client assumes the caller has already sampled metrics at :sample_rate, and doesn't perform sampling.
|
182
187
|
# @option opts [Array<String>] :tags An array of tags
|
183
188
|
def count(stat, count, opts = EMPTY_OPTIONS)
|
184
189
|
opts = { sample_rate: opts } if opts.is_a?(Numeric)
|
185
190
|
send_stats(stat, count, COUNTER_TYPE, opts)
|
186
191
|
end
|
187
192
|
|
188
|
-
# Sends an
|
193
|
+
# Sends an arbitrary gauge value for the given stat to the statsd server.
|
189
194
|
#
|
190
195
|
# This is useful for recording things like available disk space,
|
191
196
|
# memory usage, and the like, which have different semantics than
|
@@ -195,6 +200,7 @@ module Datadog
|
|
195
200
|
# @param [Numeric] value gauge value.
|
196
201
|
# @param [Hash] opts the options to create the metric with
|
197
202
|
# @option opts [Numeric] :sample_rate sample rate, 1 for always
|
203
|
+
# @option opts [Boolean] :pre_sampled If true, the client assumes the caller has already sampled metrics at :sample_rate, and doesn't perform sampling.
|
198
204
|
# @option opts [Array<String>] :tags An array of tags
|
199
205
|
# @example Report the current user count:
|
200
206
|
# $statsd.gauge('user.count', User.count)
|
@@ -209,6 +215,7 @@ module Datadog
|
|
209
215
|
# @param [Numeric] value histogram value.
|
210
216
|
# @param [Hash] opts the options to create the metric with
|
211
217
|
# @option opts [Numeric] :sample_rate sample rate, 1 for always
|
218
|
+
# @option opts [Boolean] :pre_sampled If true, the client assumes the caller has already sampled metrics at :sample_rate, and doesn't perform sampling.
|
212
219
|
# @option opts [Array<String>] :tags An array of tags
|
213
220
|
# @example Report the current user count:
|
214
221
|
# $statsd.histogram('user.count', User.count)
|
@@ -222,6 +229,7 @@ module Datadog
|
|
222
229
|
# @param [Numeric] value distribution value.
|
223
230
|
# @param [Hash] opts the options to create the metric with
|
224
231
|
# @option opts [Numeric] :sample_rate sample rate, 1 for always
|
232
|
+
# @option opts [Boolean] :pre_sampled If true, the client assumes the caller has already sampled metrics at :sample_rate, and doesn't perform sampling.
|
225
233
|
# @option opts [Array<String>] :tags An array of tags
|
226
234
|
# @example Report the current user count:
|
227
235
|
# $statsd.distribution('user.count', User.count)
|
@@ -229,6 +237,26 @@ module Datadog
|
|
229
237
|
send_stats(stat, value, DISTRIBUTION_TYPE, opts)
|
230
238
|
end
|
231
239
|
|
240
|
+
# Reports execution time of the provided block as a distribution.
|
241
|
+
#
|
242
|
+
# If the block fails, the stat is still reported, then the error
|
243
|
+
# is reraised
|
244
|
+
#
|
245
|
+
# @param [String] stat stat name.
|
246
|
+
# @param [Numeric] value distribution value.
|
247
|
+
# @param [Hash] opts the options to create the metric with
|
248
|
+
# @option opts [Numeric] :sample_rate sample rate, 1 for always
|
249
|
+
# @option opts [Array<String>] :tags An array of tags
|
250
|
+
# @example Report the time (in ms) taken to activate an account
|
251
|
+
# $statsd.distribution_time('account.activate') { @account.activate! }
|
252
|
+
def distribution_time(stat, opts = EMPTY_OPTIONS)
|
253
|
+
opts = { sample_rate: opts } if opts.is_a?(Numeric)
|
254
|
+
start = now
|
255
|
+
yield
|
256
|
+
ensure
|
257
|
+
distribution(stat, ((now - start) * 1000).round, opts)
|
258
|
+
end
|
259
|
+
|
232
260
|
# Sends a timing (in ms) for the given stat to the statsd server. The
|
233
261
|
# sample_rate determines what percentage of the time this report is sent. The
|
234
262
|
# statsd server then uses the sample_rate to correctly track the average
|
@@ -238,6 +266,7 @@ module Datadog
|
|
238
266
|
# @param [Integer] ms timing in milliseconds
|
239
267
|
# @param [Hash] opts the options to create the metric with
|
240
268
|
# @option opts [Numeric] :sample_rate sample rate, 1 for always
|
269
|
+
# @option opts [Boolean] :pre_sampled If true, the client assumes the caller has already sampled metrics at :sample_rate, and doesn't perform sampling.
|
241
270
|
# @option opts [Array<String>] :tags An array of tags
|
242
271
|
def timing(stat, ms, opts = EMPTY_OPTIONS)
|
243
272
|
opts = { sample_rate: opts } if opts.is_a?(Numeric)
|
@@ -252,6 +281,7 @@ module Datadog
|
|
252
281
|
# @param [String] stat stat name
|
253
282
|
# @param [Hash] opts the options to create the metric with
|
254
283
|
# @option opts [Numeric] :sample_rate sample rate, 1 for always
|
284
|
+
# @option opts [Boolean] :pre_sampled If true, the client assumes the caller has already sampled metrics at :sample_rate, and doesn't perform sampling.
|
255
285
|
# @option opts [Array<String>] :tags An array of tags
|
256
286
|
# @yield The operation to be timed
|
257
287
|
# @see #timing
|
@@ -271,6 +301,7 @@ module Datadog
|
|
271
301
|
# @param [Numeric] value set value.
|
272
302
|
# @param [Hash] opts the options to create the metric with
|
273
303
|
# @option opts [Numeric] :sample_rate sample rate, 1 for always
|
304
|
+
# @option opts [Boolean] :pre_sampled If true, the client assumes the caller has already sampled metrics at :sample_rate, and doesn't perform sampling.
|
274
305
|
# @option opts [Array<String>] :tags An array of tags
|
275
306
|
# @example Record a unique visitory by id:
|
276
307
|
# $statsd.set('visitors.uniques', User.id)
|
@@ -382,17 +413,10 @@ module Datadog
|
|
382
413
|
attr_reader :serializer
|
383
414
|
attr_reader :forwarder
|
384
415
|
|
385
|
-
PROCESS_TIME_SUPPORTED = (RUBY_VERSION >= '2.1.0')
|
386
416
|
EMPTY_OPTIONS = {}.freeze
|
387
417
|
|
388
|
-
|
389
|
-
|
390
|
-
Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
391
|
-
end
|
392
|
-
else
|
393
|
-
def now
|
394
|
-
Time.now.to_f
|
395
|
-
end
|
418
|
+
def now
|
419
|
+
Process.clock_gettime(Process::CLOCK_MONOTONIC)
|
396
420
|
end
|
397
421
|
|
398
422
|
def send_stats(stat, delta, type, opts = EMPTY_OPTIONS)
|
@@ -400,7 +424,7 @@ module Datadog
|
|
400
424
|
|
401
425
|
sample_rate = opts[:sample_rate] || @sample_rate || 1
|
402
426
|
|
403
|
-
if sample_rate == 1 || rand <= sample_rate
|
427
|
+
if sample_rate == 1 || opts[:pre_sampled] || rand <= sample_rate
|
404
428
|
full_stat = serializer.to_stat(stat, delta, type, tags: opts[:tags], sample_rate: sample_rate)
|
405
429
|
|
406
430
|
forwarder.send_message(full_stat)
|
metadata
CHANGED
@@ -1,15 +1,15 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: dogstatsd-ruby
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 5.
|
4
|
+
version: 5.5.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Rein Henrichs
|
8
8
|
- Karim Bogtob
|
9
|
-
autorequire:
|
9
|
+
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date:
|
12
|
+
date: 2022-06-01 00:00:00.000000000 Z
|
13
13
|
dependencies: []
|
14
14
|
description: A Ruby DogStatsd client
|
15
15
|
email: code@datadoghq.com
|
@@ -23,6 +23,7 @@ files:
|
|
23
23
|
- README.md
|
24
24
|
- lib/datadog/statsd.rb
|
25
25
|
- lib/datadog/statsd/connection.rb
|
26
|
+
- lib/datadog/statsd/connection_cfg.rb
|
26
27
|
- lib/datadog/statsd/forwarder.rb
|
27
28
|
- lib/datadog/statsd/message_buffer.rb
|
28
29
|
- lib/datadog/statsd/sender.rb
|
@@ -34,6 +35,7 @@ files:
|
|
34
35
|
- lib/datadog/statsd/serialization/tag_serializer.rb
|
35
36
|
- lib/datadog/statsd/single_thread_sender.rb
|
36
37
|
- lib/datadog/statsd/telemetry.rb
|
38
|
+
- lib/datadog/statsd/timer.rb
|
37
39
|
- lib/datadog/statsd/udp_connection.rb
|
38
40
|
- lib/datadog/statsd/uds_connection.rb
|
39
41
|
- lib/datadog/statsd/version.rb
|
@@ -42,9 +44,9 @@ licenses:
|
|
42
44
|
- MIT
|
43
45
|
metadata:
|
44
46
|
bug_tracker_uri: https://github.com/DataDog/dogstatsd-ruby/issues
|
45
|
-
changelog_uri: https://github.com/DataDog/dogstatsd-ruby/blob/v5.
|
46
|
-
documentation_uri: https://www.rubydoc.info/gems/dogstatsd-ruby/5.
|
47
|
-
source_code_uri: https://github.com/DataDog/dogstatsd-ruby/tree/v5.
|
47
|
+
changelog_uri: https://github.com/DataDog/dogstatsd-ruby/blob/v5.5.0/CHANGELOG.md
|
48
|
+
documentation_uri: https://www.rubydoc.info/gems/dogstatsd-ruby/5.5.0
|
49
|
+
source_code_uri: https://github.com/DataDog/dogstatsd-ruby/tree/v5.5.0
|
48
50
|
post_install_message: |2+
|
49
51
|
|
50
52
|
If you are upgrading from v4.x of the dogstatsd-ruby library, note the major change to the threading model:
|
@@ -57,15 +59,17 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
57
59
|
requirements:
|
58
60
|
- - ">="
|
59
61
|
- !ruby/object:Gem::Version
|
60
|
-
version: 2.
|
62
|
+
version: 2.1.0
|
61
63
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
62
64
|
requirements:
|
63
65
|
- - ">="
|
64
66
|
- !ruby/object:Gem::Version
|
65
67
|
version: '0'
|
66
68
|
requirements: []
|
67
|
-
|
68
|
-
|
69
|
+
rubyforge_project:
|
70
|
+
rubygems_version: 2.7.10
|
71
|
+
signing_key:
|
69
72
|
specification_version: 4
|
70
73
|
summary: A Ruby DogStatsd client
|
71
74
|
test_files: []
|
75
|
+
...
|