statsd-instrument 1.0.0

data/.gitignore

@@ -0,0 +1,4 @@
+*.gem
+.bundle
+Gemfile.lock
+pkg/*

data/Gemfile

@@ -0,0 +1,4 @@
+source "http://rubygems.org"
+
+# Specify your gem's dependencies in statsd-instrument.gemspec
+gemspec

data/README.md

@@ -0,0 +1,125 @@
+# StatsD client for Ruby apps
+
+This is a ruby client for statsd (http://github.com/etsy/statsd). It provides a lightweight way to track and measure metrics in your application. 
+
+We call out to statsd by sending data over a UDP socket. UDP sockets are fast, but unreliable, there is no guarantee that your data will ever arrive at it's location. In other words, fire and forget. This is perfect for this use case because it means your code doesn't get bogged down trying to log statistics. We send data to statsd several times per request and haven't noticed a performance hit.
+
+The fact that all of your stats data may not make it into statsd is no issue. Graphite (the graph database that statsd is built on) will only show you trends in your data. Internally it only keeps enough data to satisfy the levels of granularity we specify. As well as satisfying it's requirement as a fixed size database. We can throw as much data at it as we want it and it will do it's best to show us the trends over time and get rid of the fluff.
+
+For Shopify, our retention periods are:
+
+1. 10 seconds of granularity for the last 6 hours
+2. 60 seconds of granularity for the last week
+3. 10 minutes of granularity for the last 5 years
+
+This is the same as what Etsy uses (mentioned in the README for [http://github.com/etsy/statd](http://github.com/etsy/statd])).
+
+## Configuration
+
+``` ruby
+StatsD.server = 'statsd.myservice.com:8125'
+StatsD.logger = Rails.logger
+StatsD.mode = :production
+```
+
+If you set the mode to anything besides production then the library will print its calls to the logger, rather than sending them over the wire.
+
+## StatsD keys
+
+StatsD keys look like 'admin.logins.api.success'. Each dot in the key represents a 'folder' in the graphite interface. You can include any data you want in the keys.
+
+## Usage
+
+### StatsD.measure
+
+Lets you benchmark how long the execution of a specific method takes.
+
+``` ruby
+# You can pass a key and a ms value
+StatsD.measure('GoogleBase.insert', 2.55)
+
+# or more commonly pass a block that calls your code
+StatsD.measure('GoogleBase.insert') do
+  GoogleBase.insert(product)
+end
+```
+
+Rather than using this method directly it's more common to use the metaprogramming methods made available.
+
+``` ruby
+GoogleBase.extend StatsD::Instrument
+GoogleBase.statsd_measure :insert, 'GoogleBase.insert'
+```
+		
+### StatsD.increment
+
+Lets you increment a key in statsd to keep a count of something. If the specified key doesn't exist it will create it for you.
+
+``` ruby
+# increments default to +1
+StatsD.increment('GoogleBase.insert')
+# you can also specify how much to increment the key by
+StatsD.increment('GoogleBase.insert', 10)
+# you can also specify a sample rate, so only 1/10 of events
+# actually get to statsd. Useful for very high volume data
+StatsD.increment('GoogleBase.insert', 1, 0.1)
+```
+
+Again it's more common to use the metaprogramming methods.
+
+## Metaprogramming Methods
+
+As mentioned, it's most common to use the provided metaprogramming methods. This lets you define all of your instrumentation in one file and not litter your code with instrumentation details. You should enable a class for instrumentation by extending it with the `StatsD::Instrument` class.
+
+``` ruby
+GoogleBase.extend StatsD::Instrument
+```
+
+Then use the methods provided below to instrument methods in your class.
+
+### statsd\_count
+
+This will increment the given key even if the method doesn't finish (ie. raises).
+
+``` ruby
+GoogleBase.statsd_count :insert, 'GoogleBase.insert'
+```
+
+Note how I used the 'GoogleBase.insert' key above when measuring this method, and I reused here when counting the method calls. StatsD automatically separates these two kinds of stats into namespaces so there won't be a key collision here.
+
+### statsd\_count\_if
+
+This will only increment the given key if the method executes successfully.
+
+``` ruby
+GoogleBase.statsd_count_if :insert, 'GoogleBase.insert'
+```
+
+So now, if GoogleBase#insert raises an exception or returns false (ie. result == false), we won't increment the key. If you want to define what success means for a given method you can pass a block that takes the result of the method.
+
+``` ruby
+GoogleBase.statsd_count_if :insert, 'GoogleBase.insert' do |response|
+  result.code == 200
+end
+```
+
+In the above example we will only increment the key in statsd if the result of the block returns true. So the method is returning a Net::HTTP response and we're checking the status code.
+
+### statsd\_count\_success
+
+Similar to statsd_count_if, except this will increment one key in the case of success and another key in the case of failure.
+
+``` ruby
+GoogleBase.statsd_count_success :insert, 'GoogleBase.insert'
+```
+
+So if this method fails execution (raises or returns false) we'll increment the failure key ('GoogleBase.insert.failure'), otherwise we'll increment the success key ('GoogleBase.insert.success'). Notice that we're modifying the given key before sending it to statsd.
+
+Again you can pass a block to define what success means.
+
+``` ruby
+GoogleBase.statsd_count_if :insert, 'GoogleBase.insert' do |response|
+  result.code == 200
+end
+```
+

data/Rakefile

@@ -0,0 +1,8 @@
+require 'bundler/gem_tasks'
+require 'rake/testtask'
+
+Rake::TestTask.new('test') do |t|
+  t.ruby_opts << '-rubygems'
+  t.libs << 'lib'
+  t.test_files = FileList['test/*.rb']
+end

data/lib/statsd-instrument.rb

@@ -0,0 +1 @@
+require 'statsd/instrument'

data/lib/statsd/instrument.rb

@@ -0,0 +1,146 @@
+require 'socket'
+
+module StatsD
+  class << self
+    attr_accessor :host, :port, :mode, :logger, :enabled
+  end
+  self.enabled = true
+
+  trap("TTOU") { self.enabled = false }
+  trap("TTIN") { self.enabled = true }
+
+  # StatsD.server = 'localhost:1234'
+  def self.server=(conn)
+    self.host, port = conn.split(':')
+    self.port = port.to_i
+  end
+
+  module Instrument
+    def statsd_measure(method, name)
+      add_to_method(method, name, :measure) do |old_method, new_method, metric_name, *args|
+        define_method(new_method) do |*args|
+          StatsD.measure(send(metric_name)) { send(old_method, *args) }
+        end
+      end
+    end
+
+    def statsd_count_success(method, name)
+      add_to_method(method, name, :count_success) do |old_method, new_method, metric_name|
+        define_method(new_method) do |*args|
+          begin
+            truthiness = result = send(old_method, *args)
+          rescue
+            truthiness = false
+            raise
+          else
+            truthiness = (yield(result) rescue false) if block_given?
+            result
+          ensure
+            StatsD.increment("#{send(metric_name)}." + (truthiness == false ? 'failure' : 'success'))
+          end
+        end
+      end
+    end
+
+    def statsd_count_if(method, name)
+      add_to_method(method, name, :count_if) do |old_method, new_method, metric_name|
+        define_method(new_method) do |*args|
+          begin
+            truthiness = result = send(old_method, *args)
+          rescue
+            truthiness = false
+            raise
+          else
+            truthiness = (yield(result) rescue false) if block_given?
+            result
+          ensure
+            StatsD.increment(send(metric_name)) if truthiness
+          end
+        end
+      end
+    end
+
+    def statsd_count(method, name)
+      add_to_method(method, name, :count) do |old_method, new_method, metric_name|
+        define_method(new_method) do |*args|
+          StatsD.increment(send(metric_name))
+          send(old_method, *args)
+        end
+      end
+    end
+
+    private
+    def statsd_memoize(metric_name, name)
+      define_method(metric_name) do
+        name = eval("\"#{name}\"", binding)
+
+        self.class.send(:define_method, metric_name) do
+          name
+        end
+        send(metric_name)
+      end
+    end
+
+    def add_to_method(method, name, action, &block)
+      metric_name = :"#{method}_#{name}_metric_name"
+      statsd_memoize(metric_name, name)
+
+      method_name_without_statsd = :"#{method}_for_#{action}_on_#{self.name}_without_#{name}"
+      # raw_ssl_request_for_measure_on_FedEx_without_ActiveMerchant.Shipping.#{self.class.name}.ssl_request
+
+      method_name_with_statsd = :"#{method}_for_#{action}_on_#{self.name}_with_#{name}"
+      # raw_ssl_request_for_measure_on_FedEx_with_ActiveMerchant.Shipping.#{self.class.name}.ssl_request
+
+      raise ArgumentError, "already instrumented #{method} for #{self.name}" if method_defined? method_name_without_statsd
+      raise ArgumentError, "could not find method #{method} for #{self.name}" unless method_defined?(method) || private_method_defined?(method)
+
+      alias_method method_name_without_statsd, method
+      yield method_name_without_statsd, method_name_with_statsd, metric_name
+      alias_method method, method_name_with_statsd
+    end
+  end
+
+  # glork:320|ms
+  def self.measure(key, milli = nil)
+    result = nil
+    ms = Benchmark.ms do
+      result = yield 
+    end if milli.nil?
+
+    write(key, ms, :ms)
+    result
+  end
+
+  # gorets:1|c
+  def self.increment(key, delta = 1, sample_rate = 1)
+    write(key, delta, :incr, sample_rate)
+  end
+
+  private
+
+  def self.socket
+    @socket ||= UDPSocket.new
+  end
+
+  def self.write(k,v,op, sample_rate = 1)
+    return unless enabled
+    return if sample_rate < 1 && rand > sample_rate
+
+    command = "#{k}:#{v}"
+    case op
+    when :incr
+      command << '|c'
+    when :ms
+      command << '|ms'
+    end
+
+    command << "|@#{sample_rate}" if sample_rate < 1
+
+    if mode == :production
+      socket.send(command, 0, host, port)
+    else
+      logger.info "[StatsD] #{command}"
+    end
+  end
+end
+

data/statsd-instrument.gemspec

@@ -0,0 +1,15 @@
+Gem::Specification.new do |s|
+  s.name        = "statsd-instrument"
+  s.version     = '1.0.0'
+  s.authors     = ["Jesse Storimer"]
+  s.email       = ["jesse@shopify.com"]
+  s.homepage    = "http://github.com/shopify/statsd-instrument"
+
+  s.summary     = %q{A StatsD client for Ruby apps}
+  s.description = %q{A StatsD client for Ruby apps. Provides metaprogramming methods to inject StatsD instrumentation into your code.}
+
+  s.files         = `git ls-files`.split("\n")
+  s.test_files    = `git ls-files -- {test,spec,features}/*`.split("\n")
+
+  s.add_development_dependency 'mocha'
+end

data/test/statsd-instrument_test.rb

@@ -0,0 +1,148 @@
+require 'statsd-instrument'
+require 'test/unit'
+require 'mocha'
+require 'logger'
+
+StatsD.logger = Logger.new('/dev/null')
+
+module ActiveMerchant; end
+class ActiveMerchant::Base
+  def ssl_post(arg)
+    if arg
+      'OK'
+    else
+      raise 'Not OK'
+    end
+  end
+end
+
+class ActiveMerchant::Gateway < ActiveMerchant::Base
+  def purchase(arg)
+    ssl_post(arg)
+    true
+  rescue
+    false
+  end
+end
+
+class ActiveMerchant::UniqueGateway < ActiveMerchant::Base
+  def ssl_post(arg)
+    {:success => arg}
+  end
+
+  def purchase(arg)
+    ssl_post(arg)
+  end
+end
+
+ActiveMerchant::Base.extend StatsD::Instrument
+
+class StatsDTest < Test::Unit::TestCase
+  def setup
+    StatsD.stubs(:increment)
+  end
+
+  def test_statsd_count_if
+    ActiveMerchant::Gateway.statsd_count_if :ssl_post, 'ActiveMerchant.Billing.#{self.class.name}.if'
+
+    StatsD.expects(:increment).with(includes('if')).once
+    ActiveMerchant::Gateway.new.purchase(true)
+    ActiveMerchant::Gateway.new.purchase(false)
+  end
+
+  def test_statsd_count_if_with_block
+    ActiveMerchant::UniqueGateway.statsd_count_if :ssl_post, 'ActiveMerchant.Billing.#{self.class.name}.block' do |result|
+      result[:success]
+    end
+
+    StatsD.expects(:increment).with(includes('block')).once
+    ActiveMerchant::UniqueGateway.new.purchase(true)
+    ActiveMerchant::UniqueGateway.new.purchase(false)
+  end
+
+  def test_statsd_count_success
+    ActiveMerchant::Gateway.statsd_count_success :ssl_post, 'ActiveMerchant.Billing.#{self.class.name}'
+
+    StatsD.expects(:increment).with(includes('success'))
+    ActiveMerchant::Gateway.new.purchase(true)
+
+    StatsD.expects(:increment).with(includes('failure'))
+    ActiveMerchant::Gateway.new.purchase(false)
+  end
+
+  def test_statsd_count_success_with_block
+    ActiveMerchant::UniqueGateway.statsd_count_success :ssl_post, 'ActiveMerchant.Billing.#{self.class.name}' do |result|
+      result[:success]
+    end
+
+    StatsD.expects(:increment).with(includes('success'))
+    ActiveMerchant::UniqueGateway.new.purchase(true)
+
+    StatsD.expects(:increment).with(includes('failure'))
+    ActiveMerchant::UniqueGateway.new.purchase(false)
+  end
+
+  def test_statsd_count
+    ActiveMerchant::Gateway.statsd_count :ssl_post, 'ActiveMerchant.Billing.#{self.class.name}.ssl_post'
+
+    StatsD.expects(:increment).with(includes('ssl_post'))
+    ActiveMerchant::Gateway.new.purchase(true)
+  end
+
+  def test_statsd_measure
+    ActiveMerchant::UniqueGateway.statsd_measure :ssl_post, 'ActiveMerchant.Billing.#{self.class.name}.ssl_post'
+
+    StatsD.expects(:measure).with(includes('ssl_post')).returns({:success => true})
+    ActiveMerchant::UniqueGateway.new.purchase(true)
+  end
+
+  def test_count_with_sampling
+    StatsD.unstub(:increment)
+    StatsD.stubs(:rand).returns(0.6)
+    StatsD.logger.expects(:info).never
+
+    StatsD.increment('sampling.foo.bar', 1, 0.1)
+  end
+
+  def test_count_with_successful_sample
+    StatsD.unstub(:increment)
+    StatsD.stubs(:rand).returns(0.01)
+    StatsD.logger.expects(:info).once.with do |string|
+      string.include?('@0.1')
+    end
+
+    StatsD.increment('sampling.foo.bar', 1, 0.1)
+  end
+
+  def test_production_mode_should_use_udp_socket
+    StatsD.unstub(:increment)
+
+    StatsD.mode = :production
+    StatsD.server = 'localhost:123'
+    UDPSocket.any_instance.expects(:send)
+
+    StatsD.increment('fooz')
+    StatsD.mode = :test
+  end
+
+  def test_should_not_write_when_disabled
+    StatsD.enabled = false
+    StatsD.expects(:logger).never
+    StatsD.increment('fooz')
+    StatsD.enabled = true
+  end
+
+  def test_receiving_TTOU_should_disable
+    Process.kill("TTOU", $$)
+    sleep 0.5
+    assert !StatsD.enabled
+  end
+
+  def test_receiving_TTIN_should_disable
+    StatsD.enabled = false
+
+    Process.kill("TTIN", $$)
+    sleep 0.5
+    assert StatsD.enabled
+  end
+end

metadata

@@ -0,0 +1,88 @@
+--- !ruby/object:Gem::Specification 
+name: statsd-instrument
+version: !ruby/object:Gem::Version 
+  hash: 23
+  prerelease: 
+  segments: 
+  - 1
+  - 0
+  - 0
+  version: 1.0.0
+platform: ruby
+authors: 
+- Jesse Storimer
+autorequire: 
+bindir: bin
+cert_chain: []
+
+date: 2011-07-28 00:00:00 -04:00
+default_executable: 
+dependencies: 
+- !ruby/object:Gem::Dependency 
+  name: mocha
+  prerelease: false
+  requirement: &id001 !ruby/object:Gem::Requirement 
+    none: false
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        hash: 3
+        segments: 
+        - 0
+        version: "0"
+  type: :development
+  version_requirements: *id001
+description: A StatsD client for Ruby apps. Provides metaprogramming methods to inject StatsD instrumentation into your code.
+email: 
+- jesse@shopify.com
+executables: []
+
+extensions: []
+
+extra_rdoc_files: []
+
+files: 
+- .gitignore
+- Gemfile
+- README.md
+- Rakefile
+- lib/statsd-instrument.rb
+- lib/statsd/instrument.rb
+- statsd-instrument.gemspec
+- test/statsd-instrument_test.rb
+has_rdoc: true
+homepage: http://github.com/shopify/statsd-instrument
+licenses: []
+
+post_install_message: 
+rdoc_options: []
+
+require_paths: 
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement 
+  none: false
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      hash: 3
+      segments: 
+      - 0
+      version: "0"
+required_rubygems_version: !ruby/object:Gem::Requirement 
+  none: false
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      hash: 3
+      segments: 
+      - 0
+      version: "0"
+requirements: []
+
+rubyforge_project: 
+rubygems_version: 1.3.9.2
+signing_key: 
+specification_version: 3
+summary: A StatsD client for Ruby apps
+test_files: 
+- test/statsd-instrument_test.rb