librato-rails 0.3.0

data/LICENSE

@@ -0,0 +1,24 @@
+Copyright (c) 2012. Librato, Inc.
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+    * Redistributions of source code must retain the above copyright
+      notice, this list of conditions and the following disclaimer.
+    * Redistributions in binary form must reproduce the above copyright
+      notice, this list of conditions and the following disclaimer in the
+      documentation and/or other materials provided with the distribution.
+    * Neither the name of the <organization> nor the
+      names of its contributors may be used to endorse or promote products
+      derived from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL LIBRATO, INC. BE LIABLE FOR ANY
+DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

data/README.md

@@ -0,0 +1,105 @@
+librato-rails
+=======
+
+[![Build Status](https://secure.travis-ci.org/librato/librato-rails.png?branch=master)](http://travis-ci.org/librato/librato-rails)
+
+Report key statistics for your Rails app to [Librato Metrics](https://metrics.librato.com/), easily track your own custom metrics. Currently supports Rails 3.0+.
+
+**NOTE: This is currently in alpha development and is not yet officially supported**
+
+## Installation
+
+In your `Gemfile` add:
+
+    gem 'librato-rails'
+    
+Then run `bundle install`.
+
+## Configuration
+
+If you don't have a Metrics account already, [sign up](https://metrics.librato.com/). In order to send measurements to Metrics you need to provide your account credentials to `librato-rails`. You can provide these one of two ways:
+
+Create a `config/librato.yml` like the following:
+
+    production:
+      email: <your-email>
+      api_key: <your-api-key>
+      
+OR provide `LIBRATO_METRICS_USER` and `LIBRATO_METRICS_TOKEN` environment variables. If both env variables and a config file are present, environment variables will take precendence.
+
+Note that using a configuration file allows you to specify configurations per-environment. Submission will be disabled in any environment without credentials. However, if environment variables are set they will be used in all environments. 
+
+Full information on configuration options is available on the [configuration wiki page](https://github.com/librato/librato-rails/wiki/Configuration).
+
+## Automatic Measurements
+
+After installing `librato-rails` and restarting your app and you will see a number of new metrics appear in your Metrics account. These track request performance, sql queries, mail handling, and other key stats. All built-in performance metrics start with the prefix `rails` by convention &mdash; for example: `rails.request.total` is the total number of requests received during an interval. 
+
+If you have multiple apps reporting to the same Metrics account you can change this prefix in your [configuration](https://github.com/librato/librato-rails/wiki/Configuration).
+
+## Custom Measurements
+
+Tracking anything that interests you is easy with Metrics. There are four primary helpers available:
+
+#### increment
+
+Use for tracking a running total of something _across_ requests, examples:
+
+    # increment the 'sales_completed' metric by one
+    Librato.increment 'sales_completed'
+    
+    # increment by five
+    Librato.increment 'items_purchased', 5
+    
+Other things you might track this way: user signups, requests of a certain type or to a certain route, total jobs queued or processed, emails sent or received
+
+#### measure
+
+Use when you want to track an average value _per_-request. Examples:
+
+    Librato.measure 'user.social_graph.nodes', 212
+
+    Librato.measure 'jobs.queued', 3
+    
+
+#### timing
+
+Like `Librato.measure` this is per-request, but specialized for timing information:
+
+    Librato.timing 'twitter.lookup.time', 21.2
+	
+The block form auto-submits the time it took for its contents to execute as the measurement value:
+
+    Librato.timing 'twitter.lookup.time' do
+      @twitter = Twitter.lookup(user)
+    end
+
+#### group
+
+There is also a grouping helper, to make managing nested metrics easier. So this:
+
+    Librato.measure 'memcached.gets', 20
+    Librato.measure 'memcached.sets', 2
+    Librato.measure 'memcached.hits', 18
+    
+Can also be written as:
+
+    Librato.group 'memcached' do |g|
+      g.measure 'gets', 20
+      g.measure 'sets', 2
+      g.measure 'hits', 18
+    end
+
+Symbols can be used interchangably with strings for metric names.
+
+## Contribution
+
+* Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet.
+* Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it.
+* Fork the project and submit a pull request from a feature or bugfix branch.
+* Please include tests. This is important so we don't break your changes unintentionally in a future version.
+* Please don't modify the gemspec, Rakefile, version, or changelog. If you do change these files, please isolate a separate commit so we can cherry-pick around it.
+
+## Copyright
+
+Copyright (c) 2012 [Librato Inc.](http://librato.com) See LICENSE for details.

data/Rakefile

@@ -0,0 +1,41 @@
+#!/usr/bin/env rake
+begin
+  require 'bundler/setup'
+rescue LoadError
+  puts 'You must `gem install bundler` and `bundle install` to run rake tasks'
+end
+begin
+  require 'rdoc/task'
+rescue LoadError
+  require 'rdoc/rdoc'
+  require 'rake/rdoctask'
+  RDoc::Task = Rake::RDocTask
+end
+
+RDoc::Task.new(:rdoc) do |rdoc|
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.title    = 'LibratoRails'
+  rdoc.options << '--line-numbers'
+  rdoc.rdoc_files.include('README.rdoc')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+end
+
+# IRB
+desc "Open an irb session preloaded with this library"
+task :console do
+  sh "pry -rubygems -r rails -r ./lib/librato-rails.rb"
+end
+
+Bundler::GemHelper.install_tasks
+
+require 'rake/testtask'
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << 'lib'
+  t.libs << 'test'
+  t.pattern = 'test/**/*_test.rb'
+  t.verbose = false
+end
+
+
+task :default => :test

data/lib/librato-rails.rb

@@ -0,0 +1 @@
+require 'librato/rails'

data/lib/librato/rack.rb

@@ -0,0 +1,4 @@
+module Librato::Rack
+end
+
+require 'librato/rack/middleware'

data/lib/librato/rack/middleware.rb

@@ -0,0 +1,47 @@
+class Librato::Rack::Middleware
+  def initialize(app, metrics = Librato::Rails)
+    @app, @metrics = app, metrics
+  end
+
+  def call(env)
+    @metrics.check_worker if @metrics.send(:forking_server?)
+
+    header_metrics env
+
+    time     = Time.now
+    response = @app.call(env)
+    duration = (Time.now - time) * 1000.0
+
+    request_metrics response.first, duration
+
+    response
+  end
+
+  private
+
+  def header_metrics(env)
+    return unless env.keys.include?('HTTP_X_HEROKU_QUEUE_DEPTH')
+
+    @metrics.group 'rack.heroku' do |group|
+      group.measure 'queue.depth',     env['HTTP_X_HEROKU_QUEUE_DEPTH'].to_f
+      group.timing  'queue.wait_time', env['HTTP_X_HEROKU_QUEUE_WAIT_TIME'].to_f
+      group.measure 'queue.dynos',     env['HTTP_X_HEROKU_DYNOS_IN_USE'].to_f
+    end
+  end
+
+  def request_metrics(status, duration)
+    @metrics.group 'rack.request' do |group|
+      group.increment 'total'
+      group.timing    'time', duration
+      group.increment 'slow' if duration > 200.0
+
+      group.group 'status' do |s|
+        s.increment status
+        s.increment "#{status.to_s[0]}xx"
+
+        s.timing "#{status}.time", duration
+        s.timing "#{status.to_s[0]}xx.time", duration
+      end
+    end
+  end
+end

data/lib/librato/rails.rb

@@ -0,0 +1,209 @@
+require 'socket'
+require 'thread'
+
+require 'active_support/core_ext/module/attribute_accessors'
+require 'active_support/notifications'
+require 'librato/metrics'
+
+require 'librato/rack'
+require 'librato/rails/aggregator'
+require 'librato/rails/counter_cache'
+require 'librato/rails/group'
+require 'librato/rails/helpers'
+require 'librato/rails/worker'
+require 'librato/rails/version'
+
+module Librato
+  extend SingleForwardable
+  def_delegators Librato::Rails, :increment, :measure, :timing, :group
+
+  module Rails
+    extend SingleForwardable
+    CONFIG_SETTABLE = %w{api_key email flush_interval prefix source}
+    FORKING_SERVERS = [:unicorn, :passenger]
+
+    mattr_accessor :config_file
+    self.config_file = 'config/librato.yml'
+
+    # config options
+    mattr_accessor :api_key
+    mattr_accessor :email
+    mattr_accessor :flush_interval
+    mattr_accessor :prefix
+
+    # config defaults
+    self.flush_interval = 60 # seconds
+    self.prefix = 'rails'
+
+    def_delegators :counters, :increment
+    def_delegators :aggregate, :measure, :timing
+
+    class << self
+
+      # access to internal aggregator object
+      def aggregate
+        @aggregator_cache ||= Aggregator.new
+      end
+
+      # set custom api endpoint
+      def api_endpoint=(endpoint)
+        @api_endpoint = endpoint
+      end
+
+      # access to client instance
+      def client
+        @client ||= prepare_client
+      end
+
+      # detect / update configuration
+      def check_config
+        if self.config_file && File.exists?(self.config_file)
+          configs = YAML.load_file(config_file)
+          if env_specific = configs[::Rails.env]
+            settable = CONFIG_SETTABLE & env_specific.keys
+            settable.each { |key| self.send("#{key}=", env_specific[key]) }
+          end
+        end
+        self.api_key = ENV['LIBRATO_METRICS_TOKEN'] if ENV['LIBRATO_METRICS_TOKEN']
+        self.email = ENV['LIBRATO_METRICS_USER'] if ENV['LIBRATO_METRICS_USER']
+      end
+
+      # check to see if we've forked into a process where a worker
+      # isn't running yet, if so start it up!
+      def check_worker
+        if @pid != $$
+          start_worker
+          #aggregate.clear
+          #counters.clear
+        end
+      end
+
+      # access to internal counters object
+      def counters
+        @counter_cache ||= CounterCache.new
+      end
+
+      # remove any accumulated but unsent metrics
+      def delete_all
+        aggregate.delete_all
+        counters.delete_all
+      end
+
+      # send all current data to Metrics
+      def flush
+        logger.debug "[librato-rails] flushing #{Process.pid} (#{Time.now}):"
+        queue = client.new_queue(:source => qualified_source)
+        # thread safety is handled internally for both stores
+        counters.flush_to(queue)
+        aggregate.flush_to(queue)
+        logger.debug queue.queued
+        queue.submit unless queue.empty?
+      rescue Exception => error
+        logger.error "[librato-rails] submission failed permanently, worker exiting: #{error}"
+      end
+
+      def group(prefix)
+        group = Group.new(prefix)
+        yield group
+      end
+
+      def logger
+        @logger ||= ::Rails.logger
+      end
+
+      # source including process pid
+      def qualified_source
+        "#{source}.#{$$}"
+      end
+
+      # run once during Rails startup sequence
+      def setup
+        check_config
+        #return unless self.email && self.api_key
+        logger.info "[librato-rails] starting up with #{app_server}..."
+        @pid = $$
+        if forking_server?
+          install_worker_check
+        else
+          start_worker # start immediately
+        end
+      end
+
+      def source
+        @source ||= Socket.gethostname
+      end
+
+      # set a custom source
+      def source=(src)
+        @source = src
+      end
+
+      # start the worker thread, one is needed per process.
+      # if this process has been forked from an one with an active
+      # worker thread we don't need to worry about cleanup as only
+      # the forking thread is copied.
+      def start_worker
+        return if @worker # already running
+        @pid = $$
+        logger.debug "[librato-rails] >> starting up worker for pid #{@pid}..."
+        @worker = Thread.new do
+          worker = Worker.new
+          worker.run_periodically(self.flush_interval) do
+            flush
+          end
+        end
+      end
+
+    private
+
+      def app_server
+        if defined?(::Unicorn) && defined?(::Unicorn::HttpServer) && !::Unicorn.listener_names.empty?
+          :unicorn
+        elsif defined?(::IN_PHUSION_PASSENGER) || (defined?(::Passenger) && defined?(::Passenger::AbstractServer))
+          :passenger
+        elsif defined?(::Thin) && defined?(::Thin::Server)
+          :thin
+        else
+          :other
+        end
+      end
+
+      def forking_server?
+        FORKING_SERVERS.include?(app_server)
+      end
+
+      def install_worker_check
+        ::ApplicationController.prepend_before_filter do |c|
+          Librato::Rails.check_worker
+        end
+      end
+
+      def prepare_client
+        check_config
+        client = Librato::Metrics::Client.new
+        client.authenticate email, api_key
+        client.api_endpoint = @api_endpoint if @api_endpoint
+        client.custom_user_agent = user_agent
+        client
+      end
+
+      def ruby_engine
+        return RUBY_ENGINE if Object.constants.include?(:RUBY_ENGINE)
+        RUBY_DESCRIPTION.split[0]
+      end
+
+      def user_agent
+        ua_chunks = []
+        ua_chunks << "librato-rails/#{Librato::Rails::VERSION}"
+        ua_chunks << "(#{ruby_engine}; #{RUBY_VERSION}p#{RUBY_PATCHLEVEL}; #{RUBY_PLATFORM}; #{app_server})"
+        ua_chunks.join(' ')
+      end
+
+    end # end class << self
+
+  end
+end
+
+# must load after all module setup
+require 'librato/rails/railtie' if defined?(Rails)
+require 'librato/rails/subscribers'

data/lib/librato/rails/aggregator.rb

@@ -0,0 +1,48 @@
+module Librato
+  module Rails
+    class Aggregator
+      extend Forwardable
+      
+      def_delegators :@cache, :empty?
+      
+      def initialize
+        @cache = Librato::Metrics::Aggregator.new
+        @lock = Mutex.new
+      end
+      
+      def [](key)
+        return nil if @cache.empty?
+        gauges = nil
+        @lock.synchronize { gauges = @cache.queued[:gauges] }
+        gauges.each do |metric|
+          return metric if metric[:name] == key.to_s
+        end
+        nil
+      end
+      
+      def delete_all
+        @lock.synchronize { @cache.clear }
+      end
+      
+      # transfer all measurements to a queue and 
+      # reset internal status
+      def flush_to(queue, options={})
+        queued = nil
+        @lock.synchronize do
+          return if @cache.empty?
+          queued = @cache.queued
+          @cache.clear
+        end
+        queue.merge!(queued) if queued
+      end
+      
+      def measure(event, duration)
+        @lock.synchronize do
+          @cache.add event.to_s => duration
+        end
+      end
+      alias :timing :measure
+      
+    end
+  end
+end