statsd-instrument 2.1.4 → 2.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 4fc69c05df312c43b484dde1e38e7d1ea4258e38
-  data.tar.gz: 74480d14bbbad40d9e089144735bcd982258119f
+  metadata.gz: 1888f7964454431b2167c0c253ea7c53de3c1afb
+  data.tar.gz: 74c6d0f9f44eb989bcfdc44cfea912fb07c1447a
 SHA512:
-  metadata.gz: 248787123782dfebb2e8a24be83aa793ae2b7490fb9859373999446441425305c5a69c306f2c7fb58b19ee73bf2f90272f53b8e28615c64f3983c40a9f34b875
-  data.tar.gz: 690c614ac56b4937d6d7afec393a037491e41bf4c128efccdec92c4c2fbc62cc5f58f8b911417de171408c63f066b7b44047d33050323ad75ffb61ae0354c65c
+  metadata.gz: b2faeaa7145f89905c966f08b044bb90b544f53f9bc5de0b1aa29bcf7731e3807f70ad114692a31481e19693d761f74253702791c24212f173c3254dc466f9dc
+  data.tar.gz: 8f7d60c2a4144afb033adbb18435373d7bba35702782e835642923a09dff7ec8028fbf7de655c7031aa6102a8df698685af0b71b6bca4195ca17e86cb20d01fa

data/CHANGELOG.md

@@ -5,6 +5,10 @@ please at an entry to the "unreleased changes" section below.
 
 ### Unreleased changes
 
+### Version 2.2.0
+
+- Add support for two new datadog specific metric types: events and service checks.
+
 ### Version 2.1.3
 
 - The `assert_statsd_calls` test helper will now raise an exception whenever a block isn't passed.

data/CONTRIBUTING.md

@@ -1,6 +1,11 @@
 # Contributing
 
-This project is MIT licensed and welcomes outside contributions.
+This project is MIT licensed.
+
+> **Note**: this project is currently not actively maintained, but is heavily used in production.
+> As a result, pull requests and issues may not be responded to. Also, due to the limited time we have
+> avaibale to work on this library, we cannot accept PRs that do not maintain backwards compatibility,
+> or PRs that would affect the performance of the hot code paths.
 
 ## Reporting issues
 
@@ -17,7 +22,6 @@ When reporting issues, please incldue the following information:
 1. Fork the repository, and create a branch.
 2. Implement the feature or bugfix, and add tests that cover the changed functionality.
 3. Create a pull request. Make sure that you get Travis CI passes.
-4. Ping **@jstorimer** and/or **@wvanbergen** for a code review.
 
 Some notes:
 
@@ -26,6 +30,9 @@ Some notes:
 - Add an entry to the "unreleased changes" section of [CHANGELOG.md](./CHANGELOG.md).
 - **Do not** update `StatsD::Instrument::VERSION`. This will be done during the release prodecure.
 
+> **Important:** if you change anything in the hot code path (sending a StatsD metric), please
+> include benchmarks to show the performance impact of your changes.
+
 ## Release procedure
 
 1. Update the version number in `lib/statsd/instrument/version.rb`.

data/README.md

@@ -1,6 +1,6 @@
 # StatsD client for Ruby apps
 
-[![Built on Travis](https://secure.travis-ci.org/Shopify/statsd-instrument.png?branch=master)](https://secure.travis-ci.org/Shopify/statsd-instrument)
+[![Built on Travis](https://secure.travis-ci.org/Shopify/statsd-instrument.svg?branch=master)](https://secure.travis-ci.org/Shopify/statsd-instrument)
 
 This is a ruby client for statsd (http://github.com/etsy/statsd). It provides a lightweight way to track and measure metrics in your application.
 
@@ -106,6 +106,32 @@ StatsD.set('GoogleBase.customers', "12345", sample_rate: 1.0)
 
 Because you are counting unique values, the results of using a sampling value less than 1.0 can lead to unexpected, hard to interpret results.
 
+#### StatsD.event
+
+An event is a (title, text) tuple that can be used to correlate metrics with something that occured within the system.
+This is a good fit for instance to correlate response time variation with a deploy of the new code.
+
+```ruby
+StatsD.event('shipit.deploy', 'started', sample_rate: 1.0)
+```
+
+*Note: This is only supported by the [datadog implementatation](https://docs.datadoghq.com/guides/dogstatsd/#events).*
+
+Events support additional metadata such as `date_happened`, `hostname`, `aggregation_key`, `priority`, `source_type_name`, `alert_type`.
+
+#### StatsD.service_check
+
+An event is a (check_name, status) tuple that can be used to monitor the status of services your application depends on.
+
+```ruby
+StatsD.service_check('shipit.redis_connection', 'ok')
+```
+
+*Note: This is only supported by the [datadog implementatation](https://docs.datadoghq.com/guides/dogstatsd/#service-checks).*
+
+Service checks support additional metadata such as `timestamp`, `hostname`, `message`.
+
+
 ### Metaprogramming Methods
 
 As mentioned, it's most common to use the provided metaprogramming methods. This lets you define all of your instrumentation in one file and not litter your code with instrumentation details. You should enable a class for instrumentation by extending it with the `StatsD::Instrument` class.
@@ -191,7 +217,7 @@ GoogleBase.statsd_count :insert, lambda{|object, args| object.class.to_s.downcas
 
 ### Tags
 
-The Datadog implementation support tags, which you can use to slice and dice metrics in their UI. You can specify a list of tags as an option, either standalone tag (e.g. `"mytag"`), or key value based, separated by a colon: `"env:production"`.
+The Datadog implementation supports tags, which you can use to slice and dice metrics in their UI. You can specify a list of tags as an option, either standalone tag (e.g. `"mytag"`), or key value based, separated by a colon: `"env:production"`.
 
 ``` ruby
 StatsD.increment('my.counter', tags: ['env:production', 'unicorn'])

data/lib/statsd/instrument/backends/udp_backend.rb

@@ -3,12 +3,86 @@ require 'monitor'
 module StatsD::Instrument::Backends
   class UDPBackend < StatsD::Instrument::Backend
 
+    class DogStatsDProtocol
+      EVENT_OPTIONS = {
+        date_happened: 'd',
+        hostname: 'h',
+        aggregation_key: 'k',
+        priority: 'p',
+        source_type_name: 's',
+        alert_type: 't',
+      }
+
+      SERVICE_CHECK_OPTIONS = {
+        timestamp: 'd',
+        hostname: 'h',
+        message: 'm',
+      }
+
+      def supported?(metric)
+        [:c, :ms, :g, :h, :s, :_e, :_sc].include?(metric.type)
+      end
+
+      def generate_packet(metric)
+        packet = ""
+
+        if metric.type == :_e
+          escaped_title = metric.name.tr('\n', '\\n')
+          escaped_text = metric.value.tr('\n', '\\n')
+
+          packet << "_e{#{escaped_title.size},#{escaped_text.size}}:#{escaped_title}|#{escaped_text}"
+          packet << generate_metadata(metric, EVENT_OPTIONS)
+        elsif metric.type == :_sc
+          packet << "_sc|#{metric.name}|#{metric.value}"
+          packet << generate_metadata(metric, SERVICE_CHECK_OPTIONS)
+        else
+          packet << "#{metric.name}:#{metric.value}|#{metric.type}"
+        end
+
+        packet << "|@#{metric.sample_rate}" if metric.sample_rate < 1
+        packet << "|##{metric.tags.join(',')}" if metric.tags
+        packet
+      end
+
+      private
+
+      def generate_metadata(metric, options)
+        (metric.metadata.keys & options.keys).map do |key|
+          "|#{options[key]}:#{metric.metadata[key]}"
+        end.join
+      end
+    end
+
+    class StatsiteStatsDProtocol
+      def supported?(metric)
+        [:c, :ms, :g, :s, :kv].include?(metric.type)
+      end
+
+      def generate_packet(metric)
+        packet = "#{metric.name}:#{metric.value}|#{metric.type}"
+        packet << "|@#{metric.sample_rate}" unless metric.sample_rate == 1
+        packet << "\n"
+        packet
+      end
+    end
+
+    class StatsDProtocol
+      def supported?(metric)
+        [:c, :ms, :g, :s].include?(metric.type)
+      end
+
+      def generate_packet(metric)
+        packet = "#{metric.name}:#{metric.value}|#{metric.type}"
+        packet << "|@#{metric.sample_rate}" if metric.sample_rate < 1
+        packet
+      end
+    end
+
     DEFAULT_IMPLEMENTATION = :statsd
 
     include MonitorMixin
 
-    attr_reader :host, :port
-    attr_accessor :implementation
+    attr_reader :host, :port, :implementation
 
     def initialize(server = nil, implementation = nil)
       super()
@@ -16,8 +90,20 @@ module StatsD::Instrument::Backends
       self.implementation = (implementation || DEFAULT_IMPLEMENTATION).to_sym
     end
 
+    def implementation=(value)
+      @packet_factory = case value
+        when :datadog
+          DogStatsDProtocol.new
+        when :statsite
+          StatsiteStatsDProtocol.new
+        else
+          StatsDProtocol.new
+        end
+      @implementation = value
+    end
+
     def collect_metric(metric)
-      unless implementation_supports_metric_type?(metric.type)
+      unless @packet_factory.supported?(metric)
         StatsD.logger.warn("[StatsD] Metric type #{metric.type.inspect} not supported on #{implementation} implementation.")
         return false
       end
@@ -26,20 +112,12 @@ module StatsD::Instrument::Backends
         return false
       end
 
-      write_packet(generate_packet(metric))
-    end
-
-    def implementation_supports_metric_type?(type)
-      case type
-        when :h;  implementation == :datadog
-        when :kv; implementation == :statsite
-        else true
-      end
+      write_packet(@packet_factory.generate_packet(metric))
     end
 
     def server=(connection_string)
-      self.host, port = connection_string.split(':', 2)
-      self.port = port.to_i
+      @host, @port = connection_string.split(':', 2)
+      @port = @port.to_i
       invalidate_socket
     end
 
@@ -61,25 +139,6 @@ module StatsD::Instrument::Backends
       @socket
     end
 
-    def generate_packet(metric)
-      command = "#{metric.name}:#{metric.value}|#{metric.type}"
-      command << "|@#{metric.sample_rate}" if metric.sample_rate < 1 || (implementation == :statsite && metric.sample_rate > 1)
-      if metric.tags
-        if tags_supported?
-          command << "|##{metric.tags.join(',')}"
-        else
-          StatsD.logger.warn("[StatsD] Tags are only supported on Datadog implementation.")
-        end
-      end
-
-      command << "\n" if implementation == :statsite
-      command
-    end
-
-    def tags_supported?
-      implementation == :datadog
-    end
-
     def write_packet(command)
       synchronize do
         socket.send(command, 0) > 0

data/lib/statsd/instrument/metric.rb

@@ -28,7 +28,7 @@
 #
 class StatsD::Instrument::Metric
 
-  attr_accessor :type, :name, :value, :sample_rate, :tags
+  attr_accessor :type, :name, :value, :sample_rate, :tags, :metadata
 
   # Initializes a new metric instance.
   # Normally, you don't want to call this method directly, but use one of the metric collection
@@ -46,11 +46,13 @@ class StatsD::Instrument::Metric
   def initialize(options = {})
     @type = options[:type] or raise ArgumentError, "Metric :type is required."
     @name = options[:name] or raise ArgumentError, "Metric :name is required."
+    @name = normalize_name(@name)
     @name = StatsD.prefix ? "#{StatsD.prefix}.#{@name}" : @name unless options[:no_prefix]
 
     @value       = options[:value] || default_value
     @sample_rate = options[:sample_rate] || StatsD.default_sample_rate
     @tags        = StatsD::Instrument::Metric.normalize_tags(options[:tags])
+    @metadata    = options.reject { |k, _| [:type, :name, :value, :sample_rate, :tags].include?(k) }
   end
 
   # The default value for this metric, which will be used if it is not set.
@@ -93,6 +95,14 @@ class StatsD::Instrument::Metric
     s:  'set',
   }
 
+  # Strip metric names of special characters used by StatsD line protocol, replace with underscore
+  #
+  # @param name [String]
+  # @return [String]
+  def normalize_name(name)
+    name.tr(':|@'.freeze, '_')
+  end
+
   # Utility function to convert tags to the canonical form.
   #
   # - Tags specified as key value pairs will be converted into an array

data/lib/statsd/instrument/version.rb

@@ -1,5 +1,5 @@
 module StatsD
   module Instrument
-    VERSION = "2.1.4"
+    VERSION = "2.2.0"
   end
 end

data/lib/statsd/instrument.rb

@@ -369,6 +369,36 @@ module StatsD
     collect_metric(hash_argument(metric_options).merge(type: :s, name: key, value: value))
   end
 
+  # Emits an event metric.
+  # @param title [String] Title of the event.
+  # @param text [String] Body of the event.
+  # @param metric_options [Hash] (default: {}) Metric options
+  # @return (see #collect_metric)
+  # @note Supported by the datadog implementation only.
+  def event(title, text, *metric_options)
+    if text.is_a?(Hash) && metric_options.empty?
+      metric_options = [text]
+      text = text.fetch(:text, nil)
+    end
+
+    collect_metric(hash_argument(metric_options).merge(type: :_e, name: title, value: text))
+  end
+
+  # Emits a service check metric.
+  # @param title [String] Title of the event.
+  # @param text [String] Body of the event.
+  # @param metric_options [Hash] (default: {}) Metric options
+  # @return (see #collect_metric)
+  # @note Supported by the datadog implementation only.
+  def service_check(name, status, *metric_options)
+    if status.is_a?(Hash) && metric_options.empty?
+      metric_options = [status]
+      status = status.fetch(:status, nil)
+    end
+
+    collect_metric(hash_argument(metric_options).merge(type: :_sc, name: name, value: status))
+  end
+
   private
 
   # Converts old-style ordered arguments in an argument hash for backwards compatibility.

data/test/metric_test.rb

@@ -24,6 +24,15 @@ class MetricTest < Minitest::Test
     assert_equal 'counter', m.name
   end
 
+  def test_bad_metric_name
+    m = StatsD::Instrument::Metric.new(type: :c, name: 'my:metric', no_prefix: true)
+    assert_equal 'my_metric', m.name
+    m = StatsD::Instrument::Metric.new(type: :c, name: 'my|metric', no_prefix: true)
+    assert_equal 'my_metric', m.name
+    m = StatsD::Instrument::Metric.new(type: :c, name: 'my@metric', no_prefix: true)
+    assert_equal 'my_metric', m.name
+  end
+
   def test_handle_bad_tags
     assert_equal ['ignored'], StatsD::Instrument::Metric.normalize_tags(['igno|red'])
     assert_equal ['lol::class:omg::lol'], StatsD::Instrument::Metric.normalize_tags({ :"lol::class" => "omg::lol" })

data/test/udp_backend_test.rb

@@ -76,6 +76,50 @@ class UDPBackendTest < Minitest::Test
     StatsD.histogram('fooh', 42.4)
   end
 
+  def test_event_on_datadog
+    @backend.implementation = :datadog
+    @backend.expects(:write_packet).with('_e{4,3}:fooh|baz|h:localhost:3000|@0.01|#foo')
+    StatsD.event('fooh', 'baz', hostname: 'localhost:3000', sample_rate: 0.01, tags: ["foo"])
+  end
+
+  def test_event_on_datadog_escapes_newlines
+    @backend.implementation = :datadog
+    @backend.expects(:write_packet).with('_e{8,5}:fooh\\n\\n|baz\\n')
+    StatsD.event('fooh\n\n', 'baz\n')
+  end
+
+  def test_event_on_datadog_ignores_invalid_metadata
+    @backend.implementation = :datadog
+    @backend.expects(:write_packet).with('_e{4,3}:fooh|baz')
+    StatsD.event('fooh', 'baz', i_am_not_supported: 'not-supported')
+  end
+
+  def test_event_warns_when_not_using_datadog
+    @backend.implementation = :other
+    @backend.expects(:write_packet).never
+    @logger.expects(:warn)
+    StatsD.event('fooh', 'bar')
+  end
+
+  def test_service_check_on_datadog
+    @backend.implementation = :datadog
+    @backend.expects(:write_packet).with('_sc|fooh|baz|h:localhost:3000|@0.01|#foo')
+    StatsD.service_check('fooh', 'baz', hostname: 'localhost:3000', sample_rate: 0.01, tags: ["foo"])
+  end
+
+  def test_service_check_on_datadog_ignores_invalid_metadata
+    @backend.implementation = :datadog
+    @backend.expects(:write_packet).with('_sc|fooh|baz')
+    StatsD.service_check('fooh', 'baz', i_am_not_supported: 'not-supported')
+  end
+
+  def test_service_check_warns_when_not_using_datadog
+    @backend.implementation = :other
+    @backend.expects(:write_packet).never
+    @logger.expects(:warn)
+    StatsD.service_check('fooh', 'bar')
+  end
+
   def test_histogram_warns_if_not_using_datadog
     @backend.implementation = :other
     @backend.expects(:write_packet).never
@@ -108,13 +152,6 @@ class UDPBackendTest < Minitest::Test
     StatsD.increment('fooc', 3, tags: ['topic:foo', 'bar'])
   end
 
-  def test_warn_when_using_tags_and_not_on_datadog
-    @backend.implementation = :other
-    @backend.expects(:write_packet).with("fooc:1|c")
-    @logger.expects(:warn)
-    StatsD.increment('fooc', tags: ['ignored'])
-  end
-
   def test_socket_error_should_not_raise_but_log
     @socket.stubs(:connect).raises(SocketError)
     @logger.expects(:error)

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: statsd-instrument
 version: !ruby/object:Gem::Version
-  version: 2.1.4
+  version: 2.2.0
 platform: ruby
 authors:
 - Jesse Storimer
@@ -10,7 +10,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2017-07-26 00:00:00.000000000 Z
+date: 2018-05-18 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rake
@@ -162,7 +162,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.5.2
+rubygems_version: 2.6.14
 signing_key: 
 specification_version: 4
 summary: A StatsD client for Ruby apps