dogstatsd-ruby 5.3.0 → 5.3.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 828679b3ff6ef3a2acce023e8b3cc5d362fe8c41ac17acee8bd2d5662ae3b2f3
-  data.tar.gz: 0c836007b9b82eb7e0cf056f86d9426e8f43b1e1543876c53819b8e0f46c8235
+  metadata.gz: cfde2027b6fb73eee85ed0c612db51df707c2b01bec2266a3f2049acc34990f1
+  data.tar.gz: 5b1bb3263af1cbdde2bb84cf4d92f2e84ad7e0a0b3eaa5ad67f048750c062c46
 SHA512:
-  metadata.gz: dc31df8c212fa4ab0854075528a03729d8600fabbc2e387ec4bf4c3c712a5106600c68065d5204d082f3d9fbaeafa74c8dc6cee15c8dea03fe7de192bcfb654e
-  data.tar.gz: 552b383c99f0071662d83991dc5e3472b0cc73125ef9fa3c558de1a59b8bf48fed3dc0d7b96e62808cf0bb044e65c4e332e569e7c9add6dcaf72a0300bfb4ca3
+  metadata.gz: 2b563e322f2eaff18eeb07f786d6277290c032e94859d89d856a7bc2512a278424ab5ec9f94f8bf477a222e9ef00075ad2e0293b29dbb0021a5b6551f836b3c9
+  data.tar.gz: 787b04cc62a139289784a9b1992a152ca74fafa3d97afa68944dea95121ff6d3af80164b551b5430d50b263f0d0c8f702d22c1450faaaebdbc4bc69f09c331fe

data/README.md

@@ -46,14 +46,14 @@ change concerning you is the new [threading model](#threading-model):
 
 In practice, it means two things:
 
-1. Now that the client is buffering metrics before sending them, you have to call `Datadog::Statsd#flush(sync: true)` if you want synchronous behavior. In most cases, this is not needed, as the companion thread will automatically flush the buffered metrics if the buffer gets full or when you are closing the instance.
+1. Now that the client is buffering metrics before sending them, you have to call `Datadog::Statsd#flush(sync: true)` if you want synchronous behavior. In most cases, this is not needed, as the sender thread will automatically flush the buffered metrics if the buffer gets full or when you are closing the instance.
 
 2. You have to make sure you are either:
 
   * Using a singleton instance of the DogStatsD client instead of creating a new instance whenever you need one; this will let the buffering mechanism flush metrics regularly
   * Or properly disposing of the DogStatsD client instance when it is not needed anymore using the method `Datadog::Statsd#close`
 
-If you have issues with the companion thread or the buffering mode, you can instantiate a client that behaves exactly as in v4.x (i.e. no companion thread and flush on every metric submission):
+If you have issues with the sender thread or the buffering mode, you can instantiate a client that behaves exactly as in v4.x (i.e. no sender thread and flush on every metric submission):
 
 ```ruby
 # Create a DogStatsD client instance using UDP
@@ -73,9 +73,9 @@ statsd.close()
 
 ### v5.x Common Pitfalls
 
-Version v5.x of `dogstatsd-ruby` is using a companion thread for flushing. This provides better performance, but you need to consider the following pitfalls:
+Version v5.x of `dogstatsd-ruby` is using a sender thread for flushing. This provides better performance, but you need to consider the following pitfalls:
 
-1. Applications that use `fork` after having created the dogstatsd instance: the child process will automatically spawn a new companion thread to flush metrics.
+1. Applications that use `fork` after having created the dogstatsd instance: the child process will automatically spawn a new sender thread to flush metrics.
 
 2. Applications that create multiple instances of the client without closing them: it is important to `#close` all instances to free the thread and the socket they are using otherwise you will leak those resources.
 
@@ -83,7 +83,7 @@ If you are using [Sidekiq](https://github.com/mperham/sidekiq), please make sure
 
 If you are using [Puma](https://github.com/puma/puma) or [Unicorn](https://yhbt.net/unicorn.git), please make sure to create the instance of DogStatsD in the workers, not in the main process before it forks to create its workers. See [this comment for more details](https://github.com/DataDog/dogstatsd-ruby/issues/179#issuecomment-845570345).
 
-Applications that run into issues but can't apply these recommendations should use the `single_thread` mode which disables the use of the companion thread.
+Applications that run into issues but can't apply these recommendations should use the `single_thread` mode which disables the use of the sender thread.
 Here is how to instantiate a client in this mode:
 
 ```ruby
@@ -152,7 +152,7 @@ statsd.close()
 
 Starting with version 5.0, `dogstatsd-ruby` employs a new threading model where one instance of `Datadog::Statsd` can be shared between threads and where data sending is non-blocking (asynchronous).
 
-When you instantiate a `Datadog::Statsd`, a companion thread is spawned. This thread will be called the Sender thread, as it is modeled by the [Sender](../lib/datadog/statsd/sender.rb) class. You can make use of `single_thread: true` to disable this behavior.
+When you instantiate a `Datadog::Statsd`, a sender thread is spawned. This thread will be called the Sender thread, as it is modeled by the [Sender](../lib/datadog/statsd/sender.rb) class. You can make use of `single_thread: true` to disable this behavior.
 
 This thread is stopped when you close the statsd client (`Datadog::Statsd#close`). Instantiating a lot of statsd clients without calling `#close` after they are not needed anymore will most likely lead to threads being leaked.
 
@@ -196,6 +196,12 @@ Doing so means the caller thread is blocked and waiting until the data has been
 
 This is useful when preparing to exit the application or when checking unit tests.
 
+### Thread-safety
+
+By default, instances of `Datadog::Statsd` are thread-safe and we recommend that a single instance be reused by all application threads (even in applications that employ forking). The sole exception is the `#close` method — this method is not yet thread safe (work in progress here [#209](https://github.com/DataDog/dogstatsd-ruby/pull/209)).
+
+When using the `single_thread: true` mode, instances of `Datadog::Statsd` are still thread-safe, but you may run into contention on heavily-threaded applications, so we don’t recommend (for performance reasons) reusing these instances.
+
 ## Versioning
 
 This Ruby gem is using [Semantic Versioning](https://guides.rubygems.org/patterns/#semantic-versioning) but please note that supported Ruby versions can change in a minor release of this library. As much as possible, we will add a "future deprecation" message in the minor release preceding the one dropping the support.

data/lib/datadog/statsd/connection.rb

@@ -12,6 +12,7 @@ module Datadog
         telemetry.reset
       end
 
+      # not thread safe: `Sender` instances that use this are required to properly synchronize or sequence calls to this method
       def write(payload)
         logger.debug { "Statsd: #{payload}" } if logger
 

data/lib/datadog/statsd/udp_connection.rb

@@ -20,7 +20,6 @@ module Datadog
         @host = host || ENV.fetch('DD_AGENT_HOST', DEFAULT_HOST)
         @port = port || ENV.fetch('DD_DOGSTATSD_PORT', DEFAULT_PORT).to_i
         @socket = nil
-        connect
       end
 
       def close
@@ -37,7 +36,12 @@ module Datadog
         @socket.connect(host, port)
       end
 
+      # send_message is writing the message in the socket, it may create the socket if nil
+      # It is not thread-safe but since it is called by either the Sender bg thread or the
+      # SingleThreadSender (which is using a mutex while Flushing), only one thread must call
+      # it at a time.
       def send_message(message)
+        connect unless @socket
         @socket.send(message, 0)
       end
     end

data/lib/datadog/statsd/uds_connection.rb

@@ -15,7 +15,6 @@ module Datadog
 
         @socket_path = socket_path
         @socket = nil
-        connect
       end
 
       def close
@@ -32,7 +31,12 @@ module Datadog
         @socket.connect(Socket.pack_sockaddr_un(@socket_path))
       end
 
+      # send_message is writing the message in the socket, it may create the socket if nil
+      # It is not thread-safe but since it is called by either the Sender bg thread or the
+      # SingleThreadSender (which is using a mutex while Flushing), only one thread must call
+      # it at a time.
       def send_message(message)
+        connect unless @socket
         @socket.sendmsg_nonblock(message)
       rescue Errno::ECONNREFUSED, Errno::ECONNRESET, Errno::ENOENT => e
         # TODO: FIXME: This error should be considered as a retryable error in the

data/lib/datadog/statsd/version.rb

@@ -4,6 +4,6 @@ require_relative 'connection'
 
 module Datadog
   class Statsd
-    VERSION = '5.3.0'
+    VERSION = '5.3.2'
   end
 end

metadata

@@ -1,15 +1,15 @@
 --- !ruby/object:Gem::Specification
 name: dogstatsd-ruby
 version: !ruby/object:Gem::Version
-  version: 5.3.0
+  version: 5.3.2
 platform: ruby
 authors:
 - Rein Henrichs
 - Karim Bogtob
-autorequire:
+autorequire: 
 bindir: bin
 cert_chain: []
-date: 2021-10-06 00:00:00.000000000 Z
+date: 2021-11-03 00:00:00.000000000 Z
 dependencies: []
 description: A Ruby DogStatsd client
 email: code@datadoghq.com
@@ -34,7 +34,6 @@ files:
 - lib/datadog/statsd/serialization/tag_serializer.rb
 - lib/datadog/statsd/single_thread_sender.rb
 - lib/datadog/statsd/telemetry.rb
-- lib/datadog/statsd/threaded_sender.rb
 - lib/datadog/statsd/udp_connection.rb
 - lib/datadog/statsd/uds_connection.rb
 - lib/datadog/statsd/version.rb
@@ -43,10 +42,14 @@ licenses:
 - MIT
 metadata:
   bug_tracker_uri: https://github.com/DataDog/dogstatsd-ruby/issues
-  changelog_uri: https://github.com/DataDog/dogstatsd-ruby/blob/v5.3.0/CHANGELOG.md
-  documentation_uri: https://www.rubydoc.info/gems/dogstatsd-ruby/5.3.0
-  source_code_uri: https://github.com/DataDog/dogstatsd-ruby/tree/v5.3.0
-post_install_message:
+  changelog_uri: https://github.com/DataDog/dogstatsd-ruby/blob/v5.3.2/CHANGELOG.md
+  documentation_uri: https://www.rubydoc.info/gems/dogstatsd-ruby/5.3.2
+  source_code_uri: https://github.com/DataDog/dogstatsd-ruby/tree/v5.3.2
+post_install_message: |2+
+
+  If you are upgrading from v4.x of the dogstatsd-ruby library, note the major change to the threading model:
+  https://github.com/DataDog/dogstatsd-ruby#migrating-from-v4x-to-v5x
+
 rdoc_options: []
 require_paths:
 - lib
@@ -61,9 +64,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubyforge_project:
-rubygems_version: 2.7.10
-signing_key:
+rubygems_version: 3.2.22
+signing_key: 
 specification_version: 4
 summary: A Ruby DogStatsd client
 test_files: []

data/lib/datadog/statsd/threaded_sender.rb

@@ -1,132 +0,0 @@
-# frozen_string_literal: true
-
-module Datadog
-  class FlushQueue < Queue
-  end
-  class CloseQueue < Queue
-  end
-  class Statsd
-    # Sender is using a background thread to flush and pack messages
-    # in a `MessageBuffer`.
-    # The communication with this thread is done using a `Queue`.
-    # If the thread is dead, it is starting a new one to avoid having a blocked
-    # Sender with no background thread to communicate with (most of the time,
-    # having a dead background thread means that a fork just happened and that we
-    # are running in the child process).
-    class Sender
-      CLOSEABLE_QUEUES = Queue.instance_methods.include?(:close)
-
-      def initialize(message_buffer, logger: nil)
-        @message_buffer = message_buffer
-        @logger = logger
-
-        # communication and synchronization with the background thread
-        # @mux is also used to not having multiple threads fighting for
-        # closing the Sender or creating a new background thread
-        @channel = Queue.new
-        @mux = Mutex.new
-
-        @is_closed = false
-
-        # start background thread immediately
-        @sender_thread = Thread.new(&method(:send_loop))
-      end
-
-      def flush(sync: false)
-        @mux.synchronize {
-          # we don't want to send a flush action to the bg thread if:
-          # - there is no bg thread running
-          # - the sender has been closed
-          return if !sender_thread.alive? || @is_closed
-
-          if sync
-             # blocking flush
-             blocking_queue = FlushQueue.new
-             channel << blocking_queue
-             blocking_queue.pop # wait for the bg thread to finish its work
-             blocking_queue.close if CLOSEABLE_QUEUES
-           else
-             # asynchronous flush
-             channel << :flush
-           end
-         }
-      end
-
-      def add(message)
-        return if @is_closed # don't send a message to the bg thread if the sender has been closed
-
-        # the bg thread is not running anymore, this is happening if the main process has forked and
-        # we are running in the child, we will spawn a bg thread and reset buffers (containing parents' messages)
-        if !sender_thread.alive?
-          @mux.synchronize {
-            return if @is_closed
-            # test if a call from another thread has already re-created
-            # the background thread before this one acquired the lock
-            break if sender_thread.alive?
-
-            # re-create the channel of communication since we will spawn a new bg thread
-            channel.close if CLOSEABLE_QUEUES
-            @channel = Queue.new
-            message_buffer.reset # don't use messages appended by another fork
-            @sender_thread = Thread.new(&method(:send_loop))
-          }
-        end
-
-        channel << message
-      end
-
-      # Compatibility with `Sender`
-      def start()
-      end
-
-      def stop()
-        return if @is_closed
-        # use this lock to both: not having another thread stopping this instance nor
-        # having a #add call creating a new thread
-        @mux.synchronize {
-          @is_closed = true
-          if sender_thread.alive? # no reasons to stop the bg thread is none is running already
-            blocking_queue = CloseQueue.new
-            channel << blocking_queue
-            blocking_queue.pop # wait for the bg thread to finish its work
-            blocking_queue.close if CLOSEABLE_QUEUES
-            sender_thread.join(3) # wait for completion, timeout after 3 seconds
-            # TODO(remy): should I close `channel` here?
-          end
-        }
-      end
-
-      private
-
-      attr_reader :message_buffer
-      attr_reader :channel
-      attr_reader :mux
-      attr_reader :sender_thread
-
-      def send_loop
-        until (message = channel.pop).nil? && (CLOSEABLE_QUEUES && channel.closed?)
-          # skip if message is nil, e.g. when the channel is empty and closed
-          next unless message
-
-          case message
-          # if a FlushQueue is received, the background thread has to flush the message
-          # buffer and to send an :unblock to let the caller know that it has finished
-          when FlushQueue
-            message_buffer.flush
-            message << :unblock
-          # if a :flush is received, the background thread has to flush asynchronously
-          when :flush
-            message_buffer.flush
-          # if a CloseQueue is received, the background thread has to do a last flush
-          # and to send an :unblock to let the caller know that it has finished
-          when CloseQueue
-            message << :unblock
-            return
-          else
-            message_buffer.add(message)
-          end
-        end
-      end
-    end
-  end
-end