dogstatsd-ruby 5.6.5 → 5.7.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +41 -3
- data/lib/datadog/statsd/forwarder.rb +6 -0
- data/lib/datadog/statsd/message_buffer.rb +2 -2
- data/lib/datadog/statsd/origin_detection.rb +155 -0
- data/lib/datadog/statsd/sender.rb +14 -4
- data/lib/datadog/statsd/serialization/event_serializer.rb +7 -1
- data/lib/datadog/statsd/serialization/field_serializer.rb +36 -0
- data/lib/datadog/statsd/serialization/serializer.rb +6 -6
- data/lib/datadog/statsd/serialization/service_check_serializer.rb +7 -1
- data/lib/datadog/statsd/serialization/stat_serializer.rb +9 -6
- data/lib/datadog/statsd/serialization.rb +1 -0
- data/lib/datadog/statsd/telemetry.rb +9 -3
- data/lib/datadog/statsd/version.rb +1 -1
- data/lib/datadog/statsd.rb +66 -4
- metadata +7 -5
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: ef3f9f930d3c5f43019052d225eaaff5d45e88d47cedac035ba5ce9fa9ab956d
|
|
4
|
+
data.tar.gz: f3d38f39151c8300ec560662c29771a741c0099d75826768e20d2a97fb49ce88
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 366d0bb434390d69b2f018efe35b0c0b149d3cc8853c730a69456ff4c51a5f1838db187794dc6db49a4825d5dcaa26db01c2630136b8dea151f3c338d7d4bb6b
|
|
7
|
+
data.tar.gz: f468d710abbc858d333ee2dd43c8df9806e0ba56c3149075fe4e94330b3c14d0b48a4e78fc7db6b9d6941ec20267bab39ce75eae51a8625fec2c0edc65b03f92
|
data/README.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
A client for DogStatsD, an extension of the StatsD metric server for Datadog. Full API documentation is available in [DogStatsD-ruby rubydoc](https://www.rubydoc.info/github/DataDog/dogstatsd-ruby/master/Datadog/Statsd).
|
|
4
4
|
|
|
5
|
-
[](https://github.com/DataDog/dogstatsd-ruby/actions/workflows/test.yml)
|
|
6
6
|
|
|
7
7
|
See [CHANGELOG.md](CHANGELOG.md) for changes. To suggest a feature, report a bug, or general discussion, [open an issue](http://github.com/DataDog/dogstatsd-ruby/issues/).
|
|
8
8
|
|
|
@@ -91,7 +91,29 @@ statsd = Datadog::Statsd.new('localhost', 8125, single_thread: true)
|
|
|
91
91
|
statsd.close()
|
|
92
92
|
```
|
|
93
93
|
|
|
94
|
-
### Origin detection
|
|
94
|
+
### Origin detection in Kubernetes
|
|
95
|
+
|
|
96
|
+
Origin detection is a method to detect the pod that DogStatsD packets are coming from and add the pod's tags to the tag list.
|
|
97
|
+
|
|
98
|
+
#### Tag cardinality
|
|
99
|
+
|
|
100
|
+
The tags that can be added to metrics can be found [here][tags]. The cardinality can be specified globally by setting the `DD_CARDINALITY`
|
|
101
|
+
environment or by passing a `'cardinality'` field to the constructor. Cardinality can also be specified per metric by passing the value
|
|
102
|
+
in the `cardinality:` option. Valid values for this parameter are `:none`, `:low`, `:orchestrator` or `:high`. If an invalid
|
|
103
|
+
value is passed, an `ArgumentError` is raised.
|
|
104
|
+
|
|
105
|
+
Origin detection is achieved in a number of ways:
|
|
106
|
+
|
|
107
|
+
#### CGroups
|
|
108
|
+
|
|
109
|
+
On Linux the container ID can be extracted from `procfs` entries related to `cgroups`. The client reads from `/proc/self/cgroup` or
|
|
110
|
+
`/proc/self/mountinfo` to attempt to parse the container id.
|
|
111
|
+
|
|
112
|
+
In cgroup v2, the container ID can be inferred by resolving the cgroup path from `/proc/self/cgroup`, combining it with the cgroup
|
|
113
|
+
mount point from `/proc/self/mountinfo]`. The resulting directory's inode is sent to the agent. Provided the agent is on the same
|
|
114
|
+
node as the client, this can be used to identify the pod's UID.
|
|
115
|
+
|
|
116
|
+
### Over UDP
|
|
95
117
|
|
|
96
118
|
Origin detection is a method to detect which pod DogStatsD packets are coming from, in order to add the pod's tags to the tag list.
|
|
97
119
|
|
|
@@ -105,7 +127,20 @@ env:
|
|
|
105
127
|
fieldPath: metadata.uid
|
|
106
128
|
```
|
|
107
129
|
|
|
108
|
-
The DogStatsD client attaches an internal tag, `entity_id`. The value of this tag is the content of the `DD_ENTITY_ID` environment
|
|
130
|
+
The DogStatsD client attaches an internal tag, `entity_id`. The value of this tag is the content of the `DD_ENTITY_ID` environment
|
|
131
|
+
variable, which is the pod’s UID.
|
|
132
|
+
|
|
133
|
+
#### DD_EXTERNAL_ENV
|
|
134
|
+
|
|
135
|
+
If the pod is annotated with the label:
|
|
136
|
+
|
|
137
|
+
```
|
|
138
|
+
admission.datadoghq.com/enabled: "true"
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
The [admissions controller] injects an environment variable `DD_EXTERNAL_ENV`. The value of this is sent in a field with the
|
|
142
|
+
metric which can be used by the agent to determine the metrics origin.
|
|
143
|
+
|
|
109
144
|
|
|
110
145
|
## Usage
|
|
111
146
|
|
|
@@ -235,3 +270,6 @@ dogstatsd-ruby is forked from Rein Henrichs' [original Statsd client](https://gi
|
|
|
235
270
|
|
|
236
271
|
Copyright (c) 2011 Rein Henrichs. See LICENSE.txt for
|
|
237
272
|
further details.
|
|
273
|
+
|
|
274
|
+
[admissions controller]: https://docs.datadoghq.com/containers/cluster_agent/admission_controller/?tab=datadogoperator
|
|
275
|
+
[tags]: https://docs.datadoghq.com/containers/kubernetes/tag/?tab=datadogoperator
|
|
@@ -22,6 +22,9 @@ module Datadog
|
|
|
22
22
|
single_thread: false,
|
|
23
23
|
|
|
24
24
|
logger: nil,
|
|
25
|
+
container_id: nil,
|
|
26
|
+
external_data: nil,
|
|
27
|
+
cardinality: nil,
|
|
25
28
|
|
|
26
29
|
serializer:
|
|
27
30
|
)
|
|
@@ -29,6 +32,9 @@ module Datadog
|
|
|
29
32
|
|
|
30
33
|
@telemetry = if telemetry_flush_interval
|
|
31
34
|
Telemetry.new(telemetry_flush_interval,
|
|
35
|
+
container_id,
|
|
36
|
+
external_data,
|
|
37
|
+
cardinality,
|
|
32
38
|
global_tags: global_tags,
|
|
33
39
|
transport_type: @transport_type
|
|
34
40
|
)
|
|
@@ -28,8 +28,8 @@ module Datadog
|
|
|
28
28
|
# Serializes the message if it hasn't been already. Part of the
|
|
29
29
|
# delay_serialization feature.
|
|
30
30
|
if message.is_a?(Array)
|
|
31
|
-
stat, delta, type, tags, sample_rate = message
|
|
32
|
-
message = @serializer.to_stat(stat, delta, type, tags: tags, sample_rate: sample_rate)
|
|
31
|
+
stat, delta, type, tags, sample_rate, cardinality = message
|
|
32
|
+
message = @serializer.to_stat(stat, delta, type, tags: tags, sample_rate: sample_rate, cardinality: cardinality)
|
|
33
33
|
end
|
|
34
34
|
|
|
35
35
|
message_size = message.bytesize
|
|
@@ -0,0 +1,155 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
module Datadog
|
|
3
|
+
class Statsd
|
|
4
|
+
private
|
|
5
|
+
|
|
6
|
+
CGROUPV1BASECONTROLLER = "memory"
|
|
7
|
+
HOSTCGROUPNAMESPACEINODE = 0xEFFFFFFB
|
|
8
|
+
|
|
9
|
+
def host_cgroup_namespace?
|
|
10
|
+
stat = File.stat("/proc/self/ns/cgroup") rescue nil
|
|
11
|
+
return false unless stat
|
|
12
|
+
stat.ino == HOSTCGROUPNAMESPACEINODE
|
|
13
|
+
end
|
|
14
|
+
|
|
15
|
+
def parse_cgroup_node_path(lines)
|
|
16
|
+
res = {}
|
|
17
|
+
lines.split("\n").each do |line|
|
|
18
|
+
tokens = line.split(':')
|
|
19
|
+
next unless tokens.length == 3
|
|
20
|
+
|
|
21
|
+
controller = tokens[1]
|
|
22
|
+
path = tokens[2]
|
|
23
|
+
|
|
24
|
+
if controller == CGROUPV1BASECONTROLLER || controller == ''
|
|
25
|
+
res[controller] = path
|
|
26
|
+
end
|
|
27
|
+
end
|
|
28
|
+
|
|
29
|
+
res
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def get_cgroup_inode(cgroup_mount_path, proc_self_cgroup_path)
|
|
33
|
+
content = File.read(proc_self_cgroup_path) rescue nil
|
|
34
|
+
return nil unless content
|
|
35
|
+
|
|
36
|
+
controllers = parse_cgroup_node_path(content)
|
|
37
|
+
|
|
38
|
+
[CGROUPV1BASECONTROLLER, ''].each do |controller|
|
|
39
|
+
next unless controllers[controller]
|
|
40
|
+
|
|
41
|
+
segments = [
|
|
42
|
+
cgroup_mount_path.chomp('/'),
|
|
43
|
+
controller.strip,
|
|
44
|
+
controllers[controller].sub(/^\//, '')
|
|
45
|
+
]
|
|
46
|
+
path = segments.reject(&:empty?).join("/")
|
|
47
|
+
inode = inode_for_path(path)
|
|
48
|
+
return inode unless inode.nil?
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
nil
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
def inode_for_path(path)
|
|
55
|
+
stat = File.stat(path) rescue nil
|
|
56
|
+
return nil unless stat
|
|
57
|
+
"in-#{stat.ino}"
|
|
58
|
+
end
|
|
59
|
+
|
|
60
|
+
def parse_container_id(handle)
|
|
61
|
+
exp_line = /^\d+:[^:]*:(.+)$/
|
|
62
|
+
uuid = /[0-9a-f]{8}[-_][0-9a-f]{4}[-_][0-9a-f]{4}[-_][0-9a-f]{4}[-_][0-9a-f]{12}/
|
|
63
|
+
container = /[0-9a-f]{64}/
|
|
64
|
+
task = /[0-9a-f]{32}-\d+/
|
|
65
|
+
exp_container_id = /(#{uuid}|#{container}|#{task})(?:\.scope)?$/
|
|
66
|
+
|
|
67
|
+
handle.each_line do |line|
|
|
68
|
+
match = line.match(exp_line)
|
|
69
|
+
next unless match && match[1]
|
|
70
|
+
id_match = match[1].match(exp_container_id)
|
|
71
|
+
|
|
72
|
+
return id_match[1] if id_match && id_match[1]
|
|
73
|
+
end
|
|
74
|
+
|
|
75
|
+
nil
|
|
76
|
+
end
|
|
77
|
+
|
|
78
|
+
def read_container_id(fpath)
|
|
79
|
+
handle = File.open(fpath, 'r') rescue nil
|
|
80
|
+
return nil unless handle
|
|
81
|
+
|
|
82
|
+
id = parse_container_id(handle)
|
|
83
|
+
handle.close
|
|
84
|
+
id
|
|
85
|
+
end
|
|
86
|
+
|
|
87
|
+
# Extracts the final container info from a line in mount info
|
|
88
|
+
def extract_container_info(line)
|
|
89
|
+
parts = line.strip.split("/")
|
|
90
|
+
return nil unless parts.last == "hostname"
|
|
91
|
+
|
|
92
|
+
# Expected structure: [..., <group>, <container_id>, ..., "hostname"]
|
|
93
|
+
container_id = nil
|
|
94
|
+
group = nil
|
|
95
|
+
|
|
96
|
+
parts.each_with_index do |part, idx|
|
|
97
|
+
# Match the container id and include the section prior to it.
|
|
98
|
+
if part.length == 64 && !!(part =~ /\A[0-9a-f]{64}\z/)
|
|
99
|
+
group = parts[idx - 1] if idx >= 1
|
|
100
|
+
container_id = part
|
|
101
|
+
elsif part.length > 32 && !!(part =~ /\A[0-9a-f]{32}-\d+\z/)
|
|
102
|
+
group = parts[idx - 1] if idx >= 1
|
|
103
|
+
container_id = part
|
|
104
|
+
elsif !!(part =~ /\A[0-9a-f]{8}(-[0-9a-f]{4}){4}\z/)
|
|
105
|
+
group = parts[idx - 1] if idx >= 1
|
|
106
|
+
container_id = part
|
|
107
|
+
end
|
|
108
|
+
end
|
|
109
|
+
|
|
110
|
+
return container_id unless group == "sandboxes"
|
|
111
|
+
end
|
|
112
|
+
|
|
113
|
+
# Parse /proc/self/mountinfo to extract the container id.
|
|
114
|
+
# Often container runtimes embed the container id in the mount paths.
|
|
115
|
+
# We parse the mount with a final `hostname` component, which is part of
|
|
116
|
+
# the containers `etc/hostname` bind mount.
|
|
117
|
+
def parse_mount_info(handle)
|
|
118
|
+
handle.each_line do |line|
|
|
119
|
+
split = line.split(" ")
|
|
120
|
+
mnt1 = split[3]
|
|
121
|
+
mnt2 = split[4]
|
|
122
|
+
[mnt1, mnt2].each do |line|
|
|
123
|
+
container_id = extract_container_info(line)
|
|
124
|
+
return container_id unless container_id.nil?
|
|
125
|
+
end
|
|
126
|
+
end
|
|
127
|
+
|
|
128
|
+
nil
|
|
129
|
+
end
|
|
130
|
+
|
|
131
|
+
def read_mount_info(path)
|
|
132
|
+
handle = File.open(path, 'r') rescue nil
|
|
133
|
+
return nil unless handle
|
|
134
|
+
|
|
135
|
+
info = parse_mount_info(handle)
|
|
136
|
+
handle.close
|
|
137
|
+
info
|
|
138
|
+
end
|
|
139
|
+
|
|
140
|
+
def get_container_id(user_provided_id, cgroup_fallback)
|
|
141
|
+
return user_provided_id unless user_provided_id.nil?
|
|
142
|
+
return nil unless cgroup_fallback
|
|
143
|
+
|
|
144
|
+
container_id = read_container_id("/proc/self/cgroup")
|
|
145
|
+
return container_id unless container_id.nil?
|
|
146
|
+
|
|
147
|
+
container_id = read_mount_info("/proc/self/mountinfo")
|
|
148
|
+
return container_id unless container_id.nil?
|
|
149
|
+
|
|
150
|
+
return nil if host_cgroup_namespace?
|
|
151
|
+
|
|
152
|
+
get_cgroup_inode("/sys/fs/cgroup", "/proc/self/cgroup")
|
|
153
|
+
end
|
|
154
|
+
end
|
|
155
|
+
end
|
|
@@ -20,6 +20,7 @@ module Datadog
|
|
|
20
20
|
@mx = Mutex.new
|
|
21
21
|
@queue_class = queue_class
|
|
22
22
|
@thread_class = thread_class
|
|
23
|
+
@done = false
|
|
23
24
|
@flush_timer = if flush_interval
|
|
24
25
|
Datadog::Statsd::Timer.new(flush_interval) { flush(sync: true) }
|
|
25
26
|
else
|
|
@@ -66,8 +67,11 @@ module Datadog
|
|
|
66
67
|
# if the thread does not exist, we assume we are running in a forked process,
|
|
67
68
|
# empty the message queue and message buffers (these messages belong to
|
|
68
69
|
# the parent process) and spawn a new companion thread.
|
|
69
|
-
if !sender_thread.alive?
|
|
70
|
+
if sender_thread.nil? || !sender_thread.alive?
|
|
70
71
|
@mx.synchronize {
|
|
72
|
+
# an attempt was previously made to start the sender thread but failed.
|
|
73
|
+
# skipping re-start
|
|
74
|
+
return if @done
|
|
71
75
|
# a call from another thread has already re-created
|
|
72
76
|
# the companion thread before this one acquired the lock
|
|
73
77
|
break if sender_thread.alive?
|
|
@@ -96,9 +100,15 @@ module Datadog
|
|
|
96
100
|
|
|
97
101
|
# initialize a new message queue for the background thread
|
|
98
102
|
@message_queue = @queue_class.new
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
103
|
+
begin
|
|
104
|
+
# start background thread
|
|
105
|
+
@sender_thread = @thread_class.new(&method(:send_loop))
|
|
106
|
+
@sender_thread.name = "Statsd Sender" unless Gem::Version.new(RUBY_VERSION) < Gem::Version.new('2.3')
|
|
107
|
+
rescue ThreadError => e
|
|
108
|
+
@logger.debug { "Statsd: Failed to start sender thread: #{e.message}" } if @logger
|
|
109
|
+
@mx.synchronize { @done = true }
|
|
110
|
+
end
|
|
111
|
+
|
|
102
112
|
@flush_timer.start if @flush_timer
|
|
103
113
|
end
|
|
104
114
|
|
|
@@ -13,8 +13,9 @@ module Datadog
|
|
|
13
13
|
alert_type: 't:',
|
|
14
14
|
}.freeze
|
|
15
15
|
|
|
16
|
-
def initialize(global_tags: [])
|
|
16
|
+
def initialize(container_id, external_data, global_tags: [])
|
|
17
17
|
@tag_serializer = TagSerializer.new(global_tags)
|
|
18
|
+
@field_serializer = FieldSerializer.new(container_id, external_data)
|
|
18
19
|
end
|
|
19
20
|
|
|
20
21
|
def format(title, text, options = EMPTY_OPTIONS)
|
|
@@ -47,6 +48,10 @@ module Datadog
|
|
|
47
48
|
event << tags
|
|
48
49
|
end
|
|
49
50
|
|
|
51
|
+
if fields = field_serializer.format(options[:cardinality])
|
|
52
|
+
event << fields
|
|
53
|
+
end
|
|
54
|
+
|
|
50
55
|
if event.bytesize > MAX_EVENT_SIZE
|
|
51
56
|
if options[:truncate_if_too_long]
|
|
52
57
|
event.slice!(MAX_EVENT_SIZE..event.length)
|
|
@@ -59,6 +64,7 @@ module Datadog
|
|
|
59
64
|
|
|
60
65
|
protected
|
|
61
66
|
attr_reader :tag_serializer
|
|
67
|
+
attr_reader :field_serializer
|
|
62
68
|
|
|
63
69
|
def escape(text)
|
|
64
70
|
text.delete('|').tap do |t|
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Datadog
|
|
4
|
+
class Statsd
|
|
5
|
+
module Serialization
|
|
6
|
+
class FieldSerializer
|
|
7
|
+
VALID_CARDINALITY = [:none, :low, :orchestrator, :high]
|
|
8
|
+
|
|
9
|
+
def initialize(container_id, external_data)
|
|
10
|
+
@container_id = container_id
|
|
11
|
+
@external_data = external_data
|
|
12
|
+
end
|
|
13
|
+
|
|
14
|
+
def format(cardinality)
|
|
15
|
+
if @container_id.nil? && @external_data.nil? && cardinality.nil?
|
|
16
|
+
return ""
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
field = String.new
|
|
20
|
+
field << "|c:#{@container_id}" unless @container_id.nil?
|
|
21
|
+
field << "|e:#{@external_data}" unless @external_data.nil?
|
|
22
|
+
|
|
23
|
+
unless cardinality.nil?
|
|
24
|
+
unless VALID_CARDINALITY.include?(cardinality.to_sym)
|
|
25
|
+
raise ArgumentError, "Invalid cardinality #{cardinality}. Valid options are #{VALID_CARDINALITY.join(', ')}."
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
field << "|card:#{cardinality}"
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
field
|
|
32
|
+
end
|
|
33
|
+
end
|
|
34
|
+
end
|
|
35
|
+
end
|
|
36
|
+
end
|
|
@@ -6,15 +6,15 @@ module Datadog
|
|
|
6
6
|
class Statsd
|
|
7
7
|
module Serialization
|
|
8
8
|
class Serializer
|
|
9
|
-
def initialize(prefix: nil, global_tags: [])
|
|
10
|
-
@stat_serializer = StatSerializer.new(prefix, global_tags: global_tags)
|
|
11
|
-
@service_check_serializer = ServiceCheckSerializer.new(global_tags: global_tags)
|
|
12
|
-
@event_serializer = EventSerializer.new(global_tags: global_tags)
|
|
9
|
+
def initialize(prefix: nil, container_id: nil, external_data: nil, global_tags: [])
|
|
10
|
+
@stat_serializer = StatSerializer.new(prefix, container_id, external_data, global_tags: global_tags)
|
|
11
|
+
@service_check_serializer = ServiceCheckSerializer.new(container_id, external_data, global_tags: global_tags)
|
|
12
|
+
@event_serializer = EventSerializer.new(container_id, external_data, global_tags: global_tags)
|
|
13
13
|
end
|
|
14
14
|
|
|
15
15
|
# using *args would make new allocations
|
|
16
|
-
def to_stat(name, delta, type, tags: [], sample_rate: 1)
|
|
17
|
-
stat_serializer.format(name, delta, type, tags: tags, sample_rate: sample_rate)
|
|
16
|
+
def to_stat(name, delta, type, tags: [], sample_rate: 1, cardinality: nil)
|
|
17
|
+
stat_serializer.format(name, delta, type, tags: tags, sample_rate: sample_rate, cardinality: cardinality)
|
|
18
18
|
end
|
|
19
19
|
|
|
20
20
|
# using *args would make new allocations
|
|
@@ -9,8 +9,9 @@ module Datadog
|
|
|
9
9
|
hostname: 'h:',
|
|
10
10
|
}.freeze
|
|
11
11
|
|
|
12
|
-
def initialize(global_tags: [])
|
|
12
|
+
def initialize(container_id, external_data, global_tags: [])
|
|
13
13
|
@tag_serializer = TagSerializer.new(global_tags)
|
|
14
|
+
@field_serializer = FieldSerializer.new(container_id, external_data)
|
|
14
15
|
end
|
|
15
16
|
|
|
16
17
|
def format(name, status, options = EMPTY_OPTIONS)
|
|
@@ -42,11 +43,16 @@ module Datadog
|
|
|
42
43
|
service_check << '|#'
|
|
43
44
|
service_check << tags
|
|
44
45
|
end
|
|
46
|
+
|
|
47
|
+
if fields = field_serializer.format(options[:cardinality])
|
|
48
|
+
service_check << fields
|
|
49
|
+
end
|
|
45
50
|
end
|
|
46
51
|
end
|
|
47
52
|
|
|
48
53
|
protected
|
|
49
54
|
attr_reader :tag_serializer
|
|
55
|
+
attr_reader :field_serializer
|
|
50
56
|
|
|
51
57
|
def escape_message(message)
|
|
52
58
|
message.delete('|').tap do |m|
|
|
@@ -4,26 +4,28 @@ module Datadog
|
|
|
4
4
|
class Statsd
|
|
5
5
|
module Serialization
|
|
6
6
|
class StatSerializer
|
|
7
|
-
def initialize(prefix, global_tags: [])
|
|
7
|
+
def initialize(prefix, container_id, external_data, global_tags: [])
|
|
8
8
|
@prefix = prefix
|
|
9
9
|
@prefix_str = prefix.to_s
|
|
10
10
|
@tag_serializer = TagSerializer.new(global_tags)
|
|
11
|
+
@field_serializer = FieldSerializer.new(container_id, external_data)
|
|
11
12
|
end
|
|
12
13
|
|
|
13
|
-
def format(metric_name, delta, type, tags: [], sample_rate: 1)
|
|
14
|
+
def format(metric_name, delta, type, tags: [], sample_rate: 1, cardinality: nil)
|
|
14
15
|
metric_name = formatted_metric_name(metric_name)
|
|
16
|
+
fields = field_serializer.format(cardinality)
|
|
15
17
|
|
|
16
18
|
if sample_rate != 1
|
|
17
19
|
if tags_list = tag_serializer.format(tags)
|
|
18
|
-
"#{@prefix_str}#{metric_name}:#{delta}|#{type}|@#{sample_rate}|##{tags_list}"
|
|
20
|
+
"#{@prefix_str}#{metric_name}:#{delta}|#{type}|@#{sample_rate}|##{tags_list}#{fields}"
|
|
19
21
|
else
|
|
20
|
-
"#{@prefix_str}#{metric_name}:#{delta}|#{type}|@#{sample_rate}"
|
|
22
|
+
"#{@prefix_str}#{metric_name}:#{delta}|#{type}|@#{sample_rate}#{fields}"
|
|
21
23
|
end
|
|
22
24
|
else
|
|
23
25
|
if tags_list = tag_serializer.format(tags)
|
|
24
|
-
"#{@prefix_str}#{metric_name}:#{delta}|#{type}|##{tags_list}"
|
|
26
|
+
"#{@prefix_str}#{metric_name}:#{delta}|#{type}|##{tags_list}#{fields}"
|
|
25
27
|
else
|
|
26
|
-
"#{@prefix_str}#{metric_name}:#{delta}|#{type}"
|
|
28
|
+
"#{@prefix_str}#{metric_name}:#{delta}|#{type}#{fields}"
|
|
27
29
|
end
|
|
28
30
|
end
|
|
29
31
|
end
|
|
@@ -36,6 +38,7 @@ module Datadog
|
|
|
36
38
|
|
|
37
39
|
attr_reader :prefix
|
|
38
40
|
attr_reader :tag_serializer
|
|
41
|
+
attr_reader :field_serializer
|
|
39
42
|
|
|
40
43
|
if RUBY_VERSION < '3'
|
|
41
44
|
def metric_name_to_string(metric_name)
|
|
@@ -10,6 +10,7 @@ end
|
|
|
10
10
|
require_relative 'serialization/tag_serializer'
|
|
11
11
|
require_relative 'serialization/service_check_serializer'
|
|
12
12
|
require_relative 'serialization/event_serializer'
|
|
13
|
+
require_relative 'serialization/field_serializer'
|
|
13
14
|
require_relative 'serialization/stat_serializer'
|
|
14
15
|
|
|
15
16
|
require_relative 'serialization/serializer'
|
|
@@ -19,7 +19,7 @@ module Datadog
|
|
|
19
19
|
# Rough estimation of maximum telemetry message size without tags
|
|
20
20
|
MAX_TELEMETRY_MESSAGE_SIZE_WT_TAGS = 50 # bytes
|
|
21
21
|
|
|
22
|
-
def initialize(flush_interval, global_tags: [], transport_type: :udp)
|
|
22
|
+
def initialize(flush_interval, container_id, external_data, cardinality, global_tags: [], transport_type: :udp)
|
|
23
23
|
@flush_interval = flush_interval
|
|
24
24
|
@global_tags = global_tags
|
|
25
25
|
@transport_type = transport_type
|
|
@@ -32,10 +32,15 @@ module Datadog
|
|
|
32
32
|
client_version: VERSION,
|
|
33
33
|
client_transport: transport_type,
|
|
34
34
|
).format(global_tags)
|
|
35
|
+
|
|
36
|
+
@serialized_fields = Serialization::FieldSerializer.new(
|
|
37
|
+
container_id,
|
|
38
|
+
external_data
|
|
39
|
+
).format(cardinality)
|
|
35
40
|
end
|
|
36
41
|
|
|
37
42
|
def would_fit_in?(max_buffer_payload_size)
|
|
38
|
-
MAX_TELEMETRY_MESSAGE_SIZE_WT_TAGS + serialized_tags.size < max_buffer_payload_size
|
|
43
|
+
MAX_TELEMETRY_MESSAGE_SIZE_WT_TAGS + serialized_tags.size + serialized_fields.size < max_buffer_payload_size
|
|
39
44
|
end
|
|
40
45
|
|
|
41
46
|
def reset
|
|
@@ -98,9 +103,10 @@ module Datadog
|
|
|
98
103
|
|
|
99
104
|
private
|
|
100
105
|
attr_reader :serialized_tags
|
|
106
|
+
attr_reader :serialized_fields
|
|
101
107
|
|
|
102
108
|
def pattern
|
|
103
|
-
@pattern ||= "datadog.dogstatsd.client.%s:%d|#{COUNTER_TYPE}|##{serialized_tags}"
|
|
109
|
+
@pattern ||= "datadog.dogstatsd.client.%s:%d|#{COUNTER_TYPE}|##{serialized_tags}#{serialized_fields}"
|
|
104
110
|
end
|
|
105
111
|
|
|
106
112
|
if Kernel.const_defined?('Process') && Process.respond_to?(:clock_gettime)
|
data/lib/datadog/statsd.rb
CHANGED
|
@@ -6,6 +6,7 @@ require_relative 'statsd/telemetry'
|
|
|
6
6
|
require_relative 'statsd/udp_connection'
|
|
7
7
|
require_relative 'statsd/uds_connection'
|
|
8
8
|
require_relative 'statsd/connection_cfg'
|
|
9
|
+
require_relative 'statsd/origin_detection'
|
|
9
10
|
require_relative 'statsd/message_buffer'
|
|
10
11
|
require_relative 'statsd/serialization'
|
|
11
12
|
require_relative 'statsd/sender'
|
|
@@ -82,6 +83,9 @@ module Datadog
|
|
|
82
83
|
# @option [Float] default sample rate if not overridden
|
|
83
84
|
# @option [Boolean] single_thread flushes the metrics on the main thread instead of in a companion thread
|
|
84
85
|
# @option [Boolean] delay_serialization delays stat serialization
|
|
86
|
+
# @option [Boolean] origin_detection is origin detection enabled
|
|
87
|
+
# @option [String] container_id the container ID field, used for origin detection
|
|
88
|
+
# @option [String] cardinality the default tag cardinality to use
|
|
85
89
|
def initialize(
|
|
86
90
|
host = nil,
|
|
87
91
|
port = nil,
|
|
@@ -104,7 +108,11 @@ module Datadog
|
|
|
104
108
|
delay_serialization: false,
|
|
105
109
|
|
|
106
110
|
telemetry_enable: true,
|
|
107
|
-
telemetry_flush_interval: DEFAULT_TELEMETRY_FLUSH_INTERVAL
|
|
111
|
+
telemetry_flush_interval: DEFAULT_TELEMETRY_FLUSH_INTERVAL,
|
|
112
|
+
|
|
113
|
+
origin_detection: true,
|
|
114
|
+
container_id: nil,
|
|
115
|
+
cardinality: nil
|
|
108
116
|
)
|
|
109
117
|
unless tags.nil? || tags.is_a?(Array) || tags.is_a?(Hash)
|
|
110
118
|
raise ArgumentError, 'tags must be an array of string tags or a Hash'
|
|
@@ -112,7 +120,20 @@ module Datadog
|
|
|
112
120
|
|
|
113
121
|
@namespace = namespace
|
|
114
122
|
@prefix = @namespace ? "#{@namespace}.".freeze : nil
|
|
115
|
-
|
|
123
|
+
|
|
124
|
+
origin_detection_enabled = origin_detection_enabled?(origin_detection)
|
|
125
|
+
container_id = get_container_id(container_id, origin_detection_enabled)
|
|
126
|
+
|
|
127
|
+
external_data = sanitize(ENV['DD_EXTERNAL_ENV']) if origin_detection_enabled
|
|
128
|
+
|
|
129
|
+
@serializer = Serialization::Serializer.new(prefix: @prefix,
|
|
130
|
+
container_id: container_id,
|
|
131
|
+
external_data: external_data,
|
|
132
|
+
global_tags: tags,
|
|
133
|
+
)
|
|
134
|
+
|
|
135
|
+
@cardinality = cardinality || ENV['DD_CARDINALITY'] || ENV['DATADOG_CARDINALITY']
|
|
136
|
+
|
|
116
137
|
@sample_rate = sample_rate
|
|
117
138
|
@delay_serialization = delay_serialization
|
|
118
139
|
|
|
@@ -136,6 +157,10 @@ module Datadog
|
|
|
136
157
|
sender_queue_size: sender_queue_size,
|
|
137
158
|
|
|
138
159
|
telemetry_flush_interval: telemetry_enable ? telemetry_flush_interval : nil,
|
|
160
|
+
container_id: container_id,
|
|
161
|
+
external_data: external_data,
|
|
162
|
+
cardinality: @cardinality,
|
|
163
|
+
|
|
139
164
|
serializer: serializer
|
|
140
165
|
)
|
|
141
166
|
end
|
|
@@ -159,6 +184,7 @@ module Datadog
|
|
|
159
184
|
# @option opts [Boolean] :pre_sampled If true, the client assumes the caller has already sampled metrics at :sample_rate, and doesn't perform sampling.
|
|
160
185
|
# @option opts [Array<String>] :tags An array of tags
|
|
161
186
|
# @option opts [Numeric] :by increment value, default 1
|
|
187
|
+
# @option opts [String] :cardinality The tag cardinality to use
|
|
162
188
|
# @see #count
|
|
163
189
|
def increment(stat, opts = EMPTY_OPTIONS)
|
|
164
190
|
opts = { sample_rate: opts } if opts.is_a?(Numeric)
|
|
@@ -174,6 +200,7 @@ module Datadog
|
|
|
174
200
|
# @option opts [Boolean] :pre_sampled If true, the client assumes the caller has already sampled metrics at :sample_rate, and doesn't perform sampling.
|
|
175
201
|
# @option opts [Array<String>] :tags An array of tags
|
|
176
202
|
# @option opts [Numeric] :by decrement value, default 1
|
|
203
|
+
# @option opts [String] :cardinality The tag cardinality to use
|
|
177
204
|
# @see #count
|
|
178
205
|
def decrement(stat, opts = EMPTY_OPTIONS)
|
|
179
206
|
opts = { sample_rate: opts } if opts.is_a?(Numeric)
|
|
@@ -189,6 +216,7 @@ module Datadog
|
|
|
189
216
|
# @option opts [Numeric] :sample_rate sample rate, 1 for always
|
|
190
217
|
# @option opts [Boolean] :pre_sampled If true, the client assumes the caller has already sampled metrics at :sample_rate, and doesn't perform sampling.
|
|
191
218
|
# @option opts [Array<String>] :tags An array of tags
|
|
219
|
+
# @option opts [String] :cardinality The tag cardinality to use
|
|
192
220
|
def count(stat, count, opts = EMPTY_OPTIONS)
|
|
193
221
|
opts = { sample_rate: opts } if opts.is_a?(Numeric)
|
|
194
222
|
send_stats(stat, count, COUNTER_TYPE, opts)
|
|
@@ -206,6 +234,7 @@ module Datadog
|
|
|
206
234
|
# @option opts [Numeric] :sample_rate sample rate, 1 for always
|
|
207
235
|
# @option opts [Boolean] :pre_sampled If true, the client assumes the caller has already sampled metrics at :sample_rate, and doesn't perform sampling.
|
|
208
236
|
# @option opts [Array<String>] :tags An array of tags
|
|
237
|
+
# @option opts [String] :cardinality The tag cardinality to use
|
|
209
238
|
# @example Report the current user count:
|
|
210
239
|
# $statsd.gauge('user.count', User.count)
|
|
211
240
|
def gauge(stat, value, opts = EMPTY_OPTIONS)
|
|
@@ -221,6 +250,7 @@ module Datadog
|
|
|
221
250
|
# @option opts [Numeric] :sample_rate sample rate, 1 for always
|
|
222
251
|
# @option opts [Boolean] :pre_sampled If true, the client assumes the caller has already sampled metrics at :sample_rate, and doesn't perform sampling.
|
|
223
252
|
# @option opts [Array<String>] :tags An array of tags
|
|
253
|
+
# @option opts [String] :cardinality The tag cardinality to use
|
|
224
254
|
# @example Report the current user count:
|
|
225
255
|
# $statsd.histogram('user.count', User.count)
|
|
226
256
|
def histogram(stat, value, opts = EMPTY_OPTIONS)
|
|
@@ -235,6 +265,7 @@ module Datadog
|
|
|
235
265
|
# @option opts [Numeric] :sample_rate sample rate, 1 for always
|
|
236
266
|
# @option opts [Boolean] :pre_sampled If true, the client assumes the caller has already sampled metrics at :sample_rate, and doesn't perform sampling.
|
|
237
267
|
# @option opts [Array<String>] :tags An array of tags
|
|
268
|
+
# @option opts [String] :cardinality The tag cardinality to use
|
|
238
269
|
# @example Report the current user count:
|
|
239
270
|
# $statsd.distribution('user.count', User.count)
|
|
240
271
|
def distribution(stat, value, opts = EMPTY_OPTIONS)
|
|
@@ -251,6 +282,7 @@ module Datadog
|
|
|
251
282
|
# @param [Hash] opts the options to create the metric with
|
|
252
283
|
# @option opts [Numeric] :sample_rate sample rate, 1 for always
|
|
253
284
|
# @option opts [Array<String>] :tags An array of tags
|
|
285
|
+
# @option opts [String] :cardinality The tag cardinality to use
|
|
254
286
|
# @example Report the time (in ms) taken to activate an account
|
|
255
287
|
# $statsd.distribution_time('account.activate') { @account.activate! }
|
|
256
288
|
def distribution_time(stat, opts = EMPTY_OPTIONS)
|
|
@@ -272,6 +304,7 @@ module Datadog
|
|
|
272
304
|
# @option opts [Numeric] :sample_rate sample rate, 1 for always
|
|
273
305
|
# @option opts [Boolean] :pre_sampled If true, the client assumes the caller has already sampled metrics at :sample_rate, and doesn't perform sampling.
|
|
274
306
|
# @option opts [Array<String>] :tags An array of tags
|
|
307
|
+
# @option opts [String] :cardinality The tag cardinality to use
|
|
275
308
|
def timing(stat, ms, opts = EMPTY_OPTIONS)
|
|
276
309
|
opts = { sample_rate: opts } if opts.is_a?(Numeric)
|
|
277
310
|
send_stats(stat, ms, TIMING_TYPE, opts)
|
|
@@ -287,6 +320,7 @@ module Datadog
|
|
|
287
320
|
# @option opts [Numeric] :sample_rate sample rate, 1 for always
|
|
288
321
|
# @option opts [Boolean] :pre_sampled If true, the client assumes the caller has already sampled metrics at :sample_rate, and doesn't perform sampling.
|
|
289
322
|
# @option opts [Array<String>] :tags An array of tags
|
|
323
|
+
# @option opts [String] :cardinality The tag cardinality to use
|
|
290
324
|
# @yield The operation to be timed
|
|
291
325
|
# @see #timing
|
|
292
326
|
# @example Report the time (in ms) taken to activate an account
|
|
@@ -307,6 +341,7 @@ module Datadog
|
|
|
307
341
|
# @option opts [Numeric] :sample_rate sample rate, 1 for always
|
|
308
342
|
# @option opts [Boolean] :pre_sampled If true, the client assumes the caller has already sampled metrics at :sample_rate, and doesn't perform sampling.
|
|
309
343
|
# @option opts [Array<String>] :tags An array of tags
|
|
344
|
+
# @option opts [String] :cardinality The tag cardinality to use
|
|
310
345
|
# @example Record a unique visitory by id:
|
|
311
346
|
# $statsd.set('visitors.uniques', User.id)
|
|
312
347
|
def set(stat, value, opts = EMPTY_OPTIONS)
|
|
@@ -348,6 +383,7 @@ module Datadog
|
|
|
348
383
|
# @option opts [String, nil] :alert_type ('info') Can be "error", "warning", "info" or "success".
|
|
349
384
|
# @option opts [Boolean, false] :truncate_if_too_long (false) Truncate the event if it is too long
|
|
350
385
|
# @option opts [Array<String>] :tags tags to be added to every metric
|
|
386
|
+
# @option opts [String] :cardinality The tag cardinality to use
|
|
351
387
|
# @example Report an awful event:
|
|
352
388
|
# $statsd.event('Something terrible happened', 'The end is near if we do nothing', :alert_type=>'warning', :tags=>['end_of_times','urgent'])
|
|
353
389
|
def event(title, text, opts = EMPTY_OPTIONS)
|
|
@@ -427,17 +463,43 @@ module Datadog
|
|
|
427
463
|
telemetry.sent(metrics: 1) if telemetry
|
|
428
464
|
|
|
429
465
|
sample_rate = opts[:sample_rate] || @sample_rate || 1
|
|
466
|
+
cardinality = opts[:cardinality] || @cardinality
|
|
430
467
|
|
|
431
468
|
if sample_rate == 1 || opts[:pre_sampled] || rand <= sample_rate
|
|
432
469
|
full_stat =
|
|
433
470
|
if @delay_serialization
|
|
434
|
-
[stat, delta, type, opts[:tags], sample_rate]
|
|
471
|
+
[stat, delta, type, opts[:tags], sample_rate, cardinality]
|
|
435
472
|
else
|
|
436
|
-
serializer.to_stat(stat, delta, type, tags: opts[:tags], sample_rate: sample_rate)
|
|
473
|
+
serializer.to_stat(stat, delta, type, tags: opts[:tags], sample_rate: sample_rate, cardinality: cardinality)
|
|
437
474
|
end
|
|
438
475
|
|
|
439
476
|
forwarder.send_message(full_stat)
|
|
440
477
|
end
|
|
441
478
|
end
|
|
479
|
+
|
|
480
|
+
def origin_detection_enabled?(origin_detection)
|
|
481
|
+
if !origin_detection.nil? && !origin_detection
|
|
482
|
+
return false
|
|
483
|
+
end
|
|
484
|
+
|
|
485
|
+
if ENV['DD_ORIGIN_DETECTION_ENABLED']
|
|
486
|
+
return ![
|
|
487
|
+
'0',
|
|
488
|
+
'f',
|
|
489
|
+
'false'
|
|
490
|
+
].include?(
|
|
491
|
+
ENV['DD_ORIGIN_DETECTION_ENABLED'].downcase
|
|
492
|
+
)
|
|
493
|
+
end
|
|
494
|
+
|
|
495
|
+
return true
|
|
496
|
+
end
|
|
497
|
+
|
|
498
|
+
# Sanitize the DD_EXTERNAL_ENV input to ensure it doesn't contain invalid characters
|
|
499
|
+
# that may break the protocol.
|
|
500
|
+
# Removing any non-printable characters and `|`.
|
|
501
|
+
def sanitize(external_data)
|
|
502
|
+
external_data.gsub(/[^[:print:]]|`\|/, '') unless external_data.nil?
|
|
503
|
+
end
|
|
442
504
|
end
|
|
443
505
|
end
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: dogstatsd-ruby
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 5.
|
|
4
|
+
version: 5.7.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Rein Henrichs
|
|
@@ -9,7 +9,7 @@ authors:
|
|
|
9
9
|
autorequire:
|
|
10
10
|
bindir: bin
|
|
11
11
|
cert_chain: []
|
|
12
|
-
date: 2025-
|
|
12
|
+
date: 2025-08-20 00:00:00.000000000 Z
|
|
13
13
|
dependencies: []
|
|
14
14
|
description: A Ruby DogStatsd client
|
|
15
15
|
email: code@datadoghq.com
|
|
@@ -26,9 +26,11 @@ files:
|
|
|
26
26
|
- lib/datadog/statsd/connection_cfg.rb
|
|
27
27
|
- lib/datadog/statsd/forwarder.rb
|
|
28
28
|
- lib/datadog/statsd/message_buffer.rb
|
|
29
|
+
- lib/datadog/statsd/origin_detection.rb
|
|
29
30
|
- lib/datadog/statsd/sender.rb
|
|
30
31
|
- lib/datadog/statsd/serialization.rb
|
|
31
32
|
- lib/datadog/statsd/serialization/event_serializer.rb
|
|
33
|
+
- lib/datadog/statsd/serialization/field_serializer.rb
|
|
32
34
|
- lib/datadog/statsd/serialization/serializer.rb
|
|
33
35
|
- lib/datadog/statsd/serialization/service_check_serializer.rb
|
|
34
36
|
- lib/datadog/statsd/serialization/stat_serializer.rb
|
|
@@ -44,9 +46,9 @@ licenses:
|
|
|
44
46
|
- MIT
|
|
45
47
|
metadata:
|
|
46
48
|
bug_tracker_uri: https://github.com/DataDog/dogstatsd-ruby/issues
|
|
47
|
-
changelog_uri: https://github.com/DataDog/dogstatsd-ruby/blob/v5.
|
|
48
|
-
documentation_uri: https://www.rubydoc.info/gems/dogstatsd-ruby/5.
|
|
49
|
-
source_code_uri: https://github.com/DataDog/dogstatsd-ruby/tree/v5.
|
|
49
|
+
changelog_uri: https://github.com/DataDog/dogstatsd-ruby/blob/v5.7.1/CHANGELOG.md
|
|
50
|
+
documentation_uri: https://www.rubydoc.info/gems/dogstatsd-ruby/5.7.1
|
|
51
|
+
source_code_uri: https://github.com/DataDog/dogstatsd-ruby/tree/v5.7.1
|
|
50
52
|
post_install_message: |2+
|
|
51
53
|
|
|
52
54
|
If you are upgrading from v4.x of the dogstatsd-ruby library, note the major change to the threading model:
|