harness 0.0.1 → 0.1.0

data/README.md

@@ -1,6 +1,12 @@
 # Harness
 
-TODO: Write a gem description
+Harness connects measurements coming from `ActiveSupport::Notifications`
+to external metric tracking services. Counters are stored locally with
+redis before being sent to the service.
+
+Currently Supported Services:
+
+* Librato
 
 ## Installation
 
@@ -18,7 +24,151 @@ Or install it yourself as:
 
 ## Usage
 
-TODO: Write usage instructions here
+In the metrics world there are two types of things: Gauges and Counters.
+Gauges are time senstive and represent something at a specific point in
+time. Counters keep track of things and should be increasing. Counters
+can be reset back to zero. You can combine counters and/or gauges to
+correlate data about your application.
+
+Harness makes this process easily. Harness' primary goal it make it dead
+simple to start measuring different parts of your application.
+`ActiveSupport::Notifications` makes this very easy because it provides
+measurements and implements the observer pattern.
+
+## Tracking Things
+
+I guess you read the `ActiveSupport::Notifications`
+[documentation](http://api.rubyonrails.org/classes/ActiveSupport/Notifications.html)
+before going any further or this will seems like php to you. Harness
+hooks into your notifications and looks for `:gauge` or `:counter`
+options. If either is present, it will be sent to the external service.
+For example, you can track how long it's taking to do a specific thing:
+
+```ruby
+class MyClass
+  def important_method(stuff)
+    ActiveSupport::Notifications.instrument "important_method.my_class", :gauge => true do
+      do_important_stuff
+    end
+  end
+end
+```
+
+You can do the same with a counter. Counter values are automatically
+stored in redis and incremented. This means you can simply pass
+`:counter => true` in instrumentations if you'd like to count it. You
+may also pass `:counter => 5` if you'd like to provide your own value.
+This value is stored in redis so the next time `:counter => true` will
+work correctly. You can reset all the counters back to zero by calling:
+`Harness.reset_counters!`.
+
+```ruby
+class MyClass
+  def important_method(stuff)
+    ActiveSupport::Notifications.instrument "important_method.my_class", :counter => true do
+      do_important_stuff
+    end
+  end
+end
+```
+
+The instuments name will be sent as the name (`important_method.my_class`) 
+for that gauge or counter.
+
+Harness will do all the extra work in sending these metrics to whatever
+service you're using.
+
+## Customzing
+
+You can pash a hash to `:counter` or `:gauge` to initialize the
+measurement your own way.
+
+```ruby
+class MyClass
+  def important_method(stuff)
+    ActiveSupport::Notifications.instrument "important_method.my_class", :gauge => { :id => 'custom-id', :name => "My Measurement" } do
+      do_important_stuff
+    end
+  end
+end
+```
+
+## One Off Gauges and Counters
+
+You can instantiate `Harness::Counter` and `Harness::Guage` wherever you
+want. Events from `ActiveSupport` are just converted to these classes
+under the covers anyways. You can use these class if you want to take
+peridocial measurements or tracking something that happens outside the
+application.
+
+```ruby
+gauge = Harness::Gauge.new
+gauge.id = "foo.bar"
+gauge.name = "Foo's Bar"
+gauge.time # defaults to Time.now
+gauge.value = readings_from_my_server
+gauge.units = 'bytes'
+gauge.log
+
+counter = Harness::Counter.new
+counter.id = "foo.bar"
+counter.name = "# of Foo bars"
+counter.time # defaults to Time.now
+counter.value = read_total_users_in_database
+counter.log
+
+# Both class take an option hash
+
+gauge = Harness::Guage.new :time => Time.now, :id => 'foo.bar'
+counter = Harness::Counter.new :time => Time.now, :id => 'foo.bar'
+```
+
+## Configuration
+
+```ruby
+Harness.config.adapter = :librato
+
+Harness.config.librato.email = 'example@example.com'
+Harness.config.librato.token = 'your-api-key'
+
+Harness.redis = Redis.new
+```
+
+## Rails Integration
+
+Harness will automatically log metrics coming from `ActionPack`,
+`ActiveRecord`, `ActiveSupport` and `ActionMailer`.
+
+You can configure Harness from `application.rb`
+
+```ruby
+config.harness.adapter = :librato
+config.librato.email = 'example@example.com'
+config.librato.token = 'your-api-key'
+```
+
+Redis will be automatically configured if you `REDISTOGO_URL` or
+`REDIS_URL` environment variables at set. They are wrapped in a
+namespace so there will be no conflicts. If they are not present, the
+default values are used. You can customize this in an initializer:
+
+```ruby
+# config/initializers/harness.rb
+require 'erb'
+
+file = Rails.root.join 'config', 'resque.yml'
+config = YAML.load(ERB.new(File.read(Rails.root.join('config', 'resque.yml'))).result)
+
+Harness.redis = Redis.new(:url => config[Rails.env])
+```
+
+`rake harness:reset_counters` is also added.
+
+### Rails Environments
+
+Measurements are completely ignored in the test env. They are processed
+in development mode, but not sent to the external service. Everything is
+logged in production.
 
 ## Contributing
 

data/Rakefile

@@ -1,2 +1,10 @@
 #!/usr/bin/env rake
 require "bundler/gem_tasks"
+
+require 'rake/testtask'
+Rake::TestTask.new(:test) do |test|
+  test.libs << 'test'
+  test.pattern = 'test/**/*_test.rb'
+end
+
+task :default => :test

data/harness.gemspec

@@ -14,4 +14,11 @@ Gem::Specification.new do |gem|
   gem.name          = "harness"
   gem.require_paths = ["lib"]
   gem.version       = Harness::VERSION
+
+  gem.add_dependency "activesupport", "~> 3"
+  gem.add_dependency "redis"
+  gem.add_dependency "redis-namespace"
+
+  gem.add_development_dependency "simplecov"
+  gem.add_development_dependency "webmock"
 end

data/lib/harness/adapters/librato_adapter.rb

@@ -0,0 +1,68 @@
+require 'json'
+require 'uri'
+require 'net/http'
+
+module Harness
+  class LibratoAdapter
+    class Config 
+      attr_accessor :email, :token
+    end
+
+    def self.config
+      @config ||= Config.new
+    end
+
+    def self.log_gauge(gauge)
+      raise Harness::LoggingError if gauge.id.length > 63
+
+      post({:gauges => [{
+        :name => sanitize(gauge.id),
+        :display_name => gauge.name,
+        :value => gauge.value,
+        :measure_time => gauge.time.to_i,
+        :source => gauge.source,
+        :attributes => { :display_units_short => gauge.units }
+      }]})
+    end
+
+    def self.log_counter(counter)
+      raise Harness::LoggingError if counter.id.length > 63
+
+      post({:counters => [{
+        :name => sanitize(counter.id),
+        :display_name => counter.name,
+        :value => counter.value,
+        :measure_time => counter.time.to_i,
+        :source => counter.source,
+        :attributes => { :display_units_short => counter.units }
+      }]})
+    end
+
+    private
+    def self.post(params)
+      unless config.email && config.token
+        raise "Adapter not configured. Ensure email and token are set."
+      end
+
+      uri = URI.parse('https://metrics-api.librato.com/v1/metrics')
+
+      Net::HTTP.start(uri.host, uri.port, :use_ssl => uri.scheme == 'https') do |http|
+        request = Net::HTTP::Post.new uri.request_uri
+
+        request.basic_auth config.email, config.token
+        request['Content-Type'] = 'application/json'
+        request.body = params.to_json
+
+        response = http.request request
+
+        raise Harness::LoggingError, response.body unless response.code.to_i == 200
+      end
+
+      true
+    end
+
+    def self.sanitize(name)
+      name.gsub(/\./, '-')
+    end
+  end
+end

data/lib/harness/adapters/memory_adapter.rb

@@ -0,0 +1,19 @@
+module Harness
+  class MemoryAdapter
+    def self.gauges
+      @gauges ||= []
+    end
+
+    def self.counters
+      @counters ||= []
+    end
+
+    def self.log_gauge(gauge)
+      gauges << gauge
+    end
+
+    def self.log_counter(counter)
+      counters << counter
+    end
+  end
+end

data/lib/harness/adapters/null_adapter.rb

@@ -0,0 +1,11 @@
+module Harness
+  class NullAdapter
+    def self.log_gauge(gauge)
+
+    end
+
+    def self.log_counter(counter)
+
+    end
+  end
+end

data/lib/harness/consumer.rb

@@ -0,0 +1,47 @@
+module Harness
+  class Consumer
+    def consume
+      Thread.new do
+        while measurement = queue.pop
+          begin
+            logger.debug "[Harness] Processing Measurement: #{measurement.inspect}"
+
+            case measurement.class.to_s.demodulize.underscore.to_sym
+            when :gauge
+              adapter.log_gauge measurement
+            when :counter
+              adapter.log_counter measurement
+            end
+          rescue LoggingError => ex
+            logger.debug "[Harness] Logging measurement failed! Server Said: #{ex}"
+            logger.debug ex.backtrace.join("\n")
+            logger.warn "[Harness] Could not post measurement! Enable debug logging to see full errors"
+          ensure
+            mutex.synchronize { @finished = queue.empty? }
+          end
+        end
+      end
+    end
+
+    def finished?
+      @finished
+    end
+
+    private
+    def queue
+      Harness.queue
+    end
+
+    def adapter
+      Harness.config.adapter
+    end
+
+    def mutex
+      Harness.mutex
+    end
+
+    def logger
+      Harness.logger
+    end
+  end
+end

data/lib/harness/counter.rb

@@ -0,0 +1,27 @@
+module Harness
+  class Counter < Measurement
+    def self.from_event(event)
+      if event.payload[:counter].is_a? Hash
+        counter = new event.payload[:counter]
+      else
+        counter = new
+      end
+
+      counter.id ||= event.name
+
+      Harness.redis.sadd 'counters', counter.id
+
+      if event.payload[:counter].is_a? Fixnum
+        counter.value = event.payload[:counter]
+      end
+
+      if counter.value
+        Harness.redis.set counter.id, counter.value
+      else
+        counter.value = Harness.redis.incr(counter.id).to_i
+      end
+
+      counter
+    end
+  end
+end

data/lib/harness/gauge.rb

@@ -0,0 +1,21 @@
+module Harness
+  class Gauge < Measurement
+    def initialize(attributes = {})
+      super
+      self.units ||= :ms
+    end
+
+    def self.from_event(event)
+      if event.payload[:gauge].is_a? Hash
+        gauge = new event.payload[:gauge]
+      else
+        gauge = new
+      end
+
+      gauge.id ||= event.name
+      gauge.value = event.duration
+
+      gauge
+    end
+  end
+end

data/lib/harness/instrumentation.rb

@@ -0,0 +1,6 @@
+ActiveSupport::Notifications.subscribe %r{.+} do |*args|
+  event = ActiveSupport::Notifications::Event.new(*args)
+
+  Harness::Gauge.from_event(event).log if event.payload[:gauge]
+  Harness::Counter.from_event(event).log if event.payload[:counter]
+end

data/lib/harness/integration/action_controller.rb

@@ -0,0 +1,9 @@
+events = %w(write_fragment read_fragment expire_fragment write_page expire_page process_action send_file)
+
+regex = %r{#{events.join("|")}.action_controller}
+
+ActiveSupport::Notifications.subscribe regex do |*args|
+  event = ActiveSupport::Notifications::Event.new(*args)
+  gauge = Harness::Gauge.from_event event
+  gauge.log
+end

data/lib/harness/integration/action_mailer.rb

@@ -0,0 +1,10 @@
+events = %w(receive deliver)
+
+regex = %r{#{events.join("|")}.action_mailer}
+
+ActiveSupport::Notifications.subscribe regex do |*args|
+  event = ActiveSupport::Notifications::Event.new(*args)
+  gauge = Harness::Gauge.from_event event
+  gauge.log
+end
+

data/lib/harness/integration/action_view.rb

@@ -0,0 +1,10 @@
+events = %w(render_template render_partial)
+
+regex = %r{#{events.join("|")}.action_view}
+
+ActiveSupport::Notifications.subscribe regex do |*args|
+  event = ActiveSupport::Notifications::Event.new(*args)
+  gauge = Harness::Gauge.from_event event
+  gauge.log
+end
+

data/lib/harness/integration/active_support.rb

@@ -0,0 +1,9 @@
+events = %w(cache_read cache_generate cache_fetch_hit cache_write cache_delete)
+
+regex = %r{#{events.join("|")}.active_support}
+
+ActiveSupport::Notifications.subscribe regex do |*args|
+  event = ActiveSupport::Notifications::Event.new(*args)
+  gauge = Harness::Gauge.from_event event
+  gauge.log
+end

data/lib/harness/measurement.rb

@@ -0,0 +1,17 @@
+module Harness
+  class Measurement
+    attr_accessor :id, :name, :source, :time, :value, :units
+
+    def initialize(attributes = {})
+      attributes.each_pair do |name, value|
+        send "#{name}=", value
+      end
+
+      self.time ||= Time.now
+    end
+
+    def log
+      Harness.log self
+    end
+  end
+end

data/lib/harness/railtie.rb

@@ -0,0 +1,36 @@
+module Harness
+  class Railtie < ::Rails::Railtie
+    config.harness = Harness.config
+
+    rake_tasks do
+      load "harness/tasks.rake"
+    end
+
+    initializer "harness.thread" do
+      Thread.abort_on_exception = Rails.env.development? || Rails.env.test?
+    end
+
+    initializer "harness.adapter" do |app|
+      case Rails.env
+      when 'development'
+        app.config.harness.adapter = :null
+      when 'test'
+        app.config.harness.adapter = :null
+      else
+        app.config.harness.adapter = :librato
+      end
+    end
+
+    initializer "harness.logger" do |app|
+      Harness.logger = Rails.logger
+    end
+
+    initializer "harness.redis" do 
+      if existing_url = ENV['REDISTOGO_URL'] || ENV['REDIS_URL']
+        Harness.redis ||= Redis::Namespace.new('harness', :redis => Redis.connect(:url => existing_url))
+      else
+        Harness.redis ||= Redis::Namespace.new('harness', :redis => Redis.connect(:host => 'localhost', :port => '6379'))
+      end
+    end
+  end
+end

data/lib/harness/tasks.rake

@@ -0,0 +1,6 @@
+namespace :harness do
+  desc "Reset all counters back to zero"
+  task :reset_counters => :environment do
+    Harness.reset_counters!
+  end
+end

data/lib/harness/version.rb

@@ -1,3 +1,3 @@
 module Harness
-  VERSION = "0.0.1"
+  VERSION = "0.1.0"
 end

data/lib/harness.rb

@@ -1,5 +1,108 @@
 require "harness/version"
 
+require 'thread'
+
+require 'securerandom'
+
+require 'redis'
+require 'redis/namespace'
+
+require 'active_support/notifications'
+require 'active_support/core_ext/string'
+
 module Harness
-  # Your code goes here...
+  class LoggingError < RuntimeError ; end
+
+  class Config
+    attr_reader :adapter
+    attr_accessor :test_mode
+
+    def adapter=(val)
+      if val.is_a? Symbol
+        @adapter = "Harness::#{val.to_s.classify}Adapter".constantize
+      else
+        @adapter = val
+      end
+    end
+
+    def method_missing(name, *args, &block)
+      begin
+        "Harness::#{name.to_s.classify}Adapter".constantize.config
+      rescue NameError
+        super
+      end
+    end
+  end
+
+  def self.config
+    @config ||= Config.new
+  end
+
+  def self.queue
+    @queue ||= Queue.new
+  end
+
+  def self.consumer
+    Thread.current["#{object_id}_harness_consumer_"] ||= Consumer.new
+  end
+
+  def self.log(measurement)
+    queue << measurement
+    wait if config.test_mode
+  end
+
+  def self.mutex
+    @mutex ||= Mutex.new
+  end
+
+  def self.wait
+    sleep 0.01 until consumer.finished? && queue.empty?
+  end
+
+  def self.logger
+    @logger
+  end
+
+  def self.logger=(logger)
+    @logger = logger
+  end
+
+  def self.redis=(redis)
+    @redis = redis
+  end
+
+  def self.redis
+    @redis
+  end
+
+  def self.reset_counters!
+    redis.smembers('counters').each do |counter|
+      redis.set counter, -1
+    end
+  end
 end
+
+require 'harness/measurement'
+require 'harness/counter'
+require 'harness/gauge'
+
+require 'harness/instrumentation'
+
+require 'harness/consumer'
+
+require 'harness/adapters/librato_adapter'
+require 'harness/adapters/memory_adapter'
+require 'harness/adapters/null_adapter'
+
+require 'harness/integration/action_controller'
+require 'harness/integration/action_view'
+require 'harness/integration/action_mailer'
+require 'harness/integration/active_support'
+
+require 'harness/railtie' if defined?(Rails)
+
+Harness.consumer.consume
+
+require 'logger'
+
+Harness.logger = Logger.new $stdout

data/test/integration/counters_with_redis_test.rb

@@ -0,0 +1,69 @@
+require 'test_helper'
+
+class CountersWithRedis < IntegrationTest
+  def test_stores_name_in_redis
+    instrument "event-counter", :counter => true
+
+    assert_includes redis.smembers('counters'), 'event-counter'
+    assert_equal 1, redis.get('event-counter').to_i
+
+    assert_counter_logged "event-counter"
+  end
+
+  def test_increments_counter_each_instrument
+    instrument "event-counter", :counter => true
+    assert_counter_logged "event-counter"
+    counters.clear
+
+    assert_empty counters
+    instrument "event-counter", :counter => true
+    assert_counter_logged "event-counter"
+
+    assert_equal 2, counters.first.value
+  end
+
+  def test_sets_given_value_in_redis_with_shortform
+    instrument "event-counter", :counter => 10
+
+    assert_equal 10, redis.get("event-counter").to_i
+
+    counters.clear
+
+    instrument "event-counter", :counter => true
+    assert_counter_logged 'event-counter'
+
+    assert_equal 11, counters.first.value
+  end
+
+  def test_sets_given_value_in_redis_with_longform
+    instrument "event-counter", :counter => { :value => 10 }
+
+    assert_equal 10, redis.get("event-counter").to_i
+
+    counters.clear
+
+    instrument "event-counter", :counter => true
+    assert_counter_logged 'event-counter'
+
+    assert_equal 11, counters.first.value
+  end
+
+  def test_resets_counters
+    instrument "event-counter", :counter => true
+    instrument "event-counter2", :counter => true
+
+    assert_equal 1, redis.get("event-counter").to_i
+    assert_equal 1, redis.get("event-counter2").to_i
+
+    Harness.reset_counters!
+
+    assert_equal -1, redis.get("event-counter").to_i
+    assert_equal -1, redis.get("event-counter2").to_i
+
+    counters.clear
+    instrument "event-counter", :counter => true
+    assert_counter_logged 'event-counter'
+
+    assert_equal 0, counters.first.value
+  end
+end