librato-logreporter 0.1.0

data/CHANGELOG.md

@@ -0,0 +1,2 @@
+### Version 0.1.0
+* Initial version

data/LICENSE

@@ -0,0 +1,24 @@
+Copyright (c) 2011. Librato, Inc.
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+    * Redistributions of source code must retain the above copyright
+      notice, this list of conditions and the following disclaimer.
+    * Redistributions in binary form must reproduce the above copyright
+      notice, this list of conditions and the following disclaimer in the
+      documentation and/or other materials provided with the distribution.
+    * Neither the name of Librato, Inc. nor the names of project contributors 
+      may be used to endorse or promote products derived from this software 
+      without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL LIBRATO, INC. BE LIABLE FOR ANY
+DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

data/README.md

@@ -0,0 +1,138 @@
+librato-logreporter
+=======
+
+[![Build Status](https://secure.travis-ci.org/librato/librato-logreporter.png?branch=master)](http://travis-ci.org/librato/librato-logreporter) [![Code Climate](https://codeclimate.com/github/librato/librato-logreporter.png)](https://codeclimate.com/github/librato/librato-logreporter)
+
+NOTE: This library is in active development and is suggested for early-adopter use only.
+
+`librato-logreporter` provides an easy interface to write metrics ultimately bound for [Librato Metrics](https://metrics.librato.com/) to your logs or another IO stream. It is fully format-compliant with [l2met](https://github.com/ryandotsmith/l2met). If you are running on Heroku it will allow you to easily insert metrics which can be retrieved via a [log drain](https://devcenter.heroku.com/articles/logging#syslog-drains).
+
+This library is ideally suited for custom or short-lived processes where the overhead of in-process collection will be costly and external metric collectors are unavailable.
+
+If you are considering using `librato-logreporter` for a rails or rack-based web app, first explore [librato-rails](https://github.com/librato/librato-rails) and/or [librato-rack](https://github.com/librato/librato-rack). In most cases one of these libraries will be a better solution for your web applications.
+
+Currently Ruby 1.9.2+ is required.
+
+## Quick Start
+
+Install `librato-logreporter` in your application:
+
+    require 'librato/logreporter'
+
+You can now track custom metrics by adding simple one-liners to your code:
+
+    # keep counts of key events
+    Librato.increment 'jobs.worked'
+
+    # benchmark sections of code to verify performance
+    Librato.timing 'my.complicated.work' do
+      # do work
+    end
+
+    # track averages across processes/jobs/requests
+    Librato.measure 'payload.size', payload.length_in_bytes
+
+## Installation & Configuration
+
+Install the gem:
+
+    $ gem install librato-logreporter
+
+Or add to your Gemfile if using bundler:
+
+    gem "librato-logreporter"
+
+Then require it:
+
+    require 'librato-reporter'
+
+If you don't have a [Librato Metrics](https://metrics.librato.com/) account already, [sign up](https://metrics.librato.com/). In order to send measurements to Librato you will need to provide your account credentials to the processor for output from `librato-reporter`.
+
+##### Environment variables
+
+There are a few optional environment variables you may find useful:
+
+* `LIBRATO_SOURCE` - the default source to use for submitted metrics. If not set your metrics will be submitted without a source.
+* `LIBRATO_PREFIX` - a prefix which will be appended to all metric names
+
+##### Running on Heroku
+
+You should specify a custom source for your app to track properly. You can set the source in your environment:
+
+    heroku config:add LIBRATO_SOURCE=myappname
+
+NOTE: if Heroku idles your process no measurements will be sent until it receives a request and is restarted. If you see intermittent gaps in your measurements during periods of low traffic this is the most likely cause.
+
+##### Harvesting your metrics from your logs
+
+There are few options for this which we will document further going forward. For the moment, [come ask us about it](http://chat.librato.com/).
+
+## Custom Measurements
+
+Tracking anything that interests you is easy with Librato. There are four primary helpers available:
+
+#### increment
+
+Use for tracking a running total of something _across_ jobs or requests, examples:
+
+    # increment the 'jobs.completed' metric by one
+    Librato.increment 'jobs.completed'
+
+    # increment by five
+    Librato.increment 'items.purchased', :by => 5
+
+    # increment with a custom source
+    Librato.increment 'user.purchases', :source => user.id
+
+Other things you might track this way: user activity, requests of a certain type or to a certain route, total jobs queued or processed, emails sent or received
+
+#### measure
+
+Use when you want to track an average value _per_-measurement period. Examples:
+
+    Librato.measure 'payload.size', 212
+
+    # report from a custom source
+    Librato.measure 'jobs.by.user', 3, :source => job.requestor.id
+
+#### timing
+
+Like `Librato.measure` this is per-period, but specialized for timing information:
+
+    Librato.timing 'twitter.lookup.time', 21.2
+
+The block form auto-submits the time it took for its contents to execute as the measurement value:
+
+    Librato.timing 'twitter.lookup.time' do
+      @twitter = Twitter.lookup(user)
+    end
+
+#### group
+
+There is also a grouping helper, to make managing nested metrics easier. So this:
+
+    Librato.measure 'memcached.gets', 20
+    Librato.measure 'memcached.sets', 2
+    Librato.measure 'memcached.hits', 18
+
+Can also be written as:
+
+    Librato.group 'memcached' do |g|
+      g.measure 'gets', 20
+      g.measure 'sets', 2
+      g.measure 'hits', 18
+    end
+
+Symbols can be used interchangeably with strings for metric names.
+
+## Contribution
+
+* Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet.
+* Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it.
+* Fork the project and submit a pull request from a feature or bugfix branch.
+* Please include tests. This is important so we don't break your changes unintentionally in a future version.
+* Please don't modify the gemspec, Rakefile, version, or changelog. If you do change these files, please isolate a separate commit so we can cherry-pick around it.
+
+## Copyright
+
+Copyright (c) 2013 [Librato Inc.](http://librato.com) See LICENSE for details.

data/Rakefile

@@ -0,0 +1,25 @@
+#!/usr/bin/env rake
+begin
+  require 'bundler/setup'
+rescue LoadError
+  puts 'You must `gem install bundler` and `bundle install` to run rake tasks'
+end
+
+# console
+desc "Open an console session preloaded with this library"
+task :console do
+  sh "pry -r ./lib/librato-logreporter.rb"
+end
+
+Bundler::GemHelper.install_tasks
+
+require 'rake/testtask'
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << 'lib'
+  t.libs << 'test'
+  t.pattern = 'test/**/*_test.rb'
+  t.verbose = false
+end
+
+task :default => :test

data/lib/librato-logreporter.rb

@@ -0,0 +1 @@
+require 'librato/logreporter'

data/lib/librato/logreporter.rb

@@ -0,0 +1,123 @@
+require_relative 'logreporter/configuration'
+require_relative 'logreporter/group'
+require_relative 'logreporter/version'
+
+module Librato
+  extend SingleForwardable
+  def_delegators :log_reporter, :increment, :measure, :timing, :group
+
+  def self.log_reporter
+    @log_reporter ||= LogReporter.new
+  end
+
+  # Provides a common interface to reporting metrics with methods
+  # like #increment, #measure, #timing, #group - all written
+  # to your preferred IO stream.
+  #
+  class LogReporter
+    include Configuration
+
+    # Group a set of metrics by common prefix
+    #
+    # @example
+    #   # write a 'performance.hits' increment and
+    #   # a 'performance.response.time' measure
+    #   group(:performance) do |perf|
+    #     perf.increment :hits
+    #     perf.measure 'response.time', response_time
+    #   end
+    #
+    def group(prefix)
+      yield Group.new(collector: self, prefix: prefix)
+    end
+
+    # Increment a given metric
+    #
+    # @example Increment metric 'foo' by 1
+    #   increment :foo
+    #
+    # @example Increment metric 'bar' by 2
+    #   increment :bar, :by => 2
+    #
+    # @example Increment metric 'foo' by 1 with a custom source
+    #   increment :foo, :source => user.id
+    #
+    def increment(counter, options={})
+      by = options[:by] || 1
+      log_write(counter => by, :source => options[:source])
+    end
+
+    # @example Simple measurement
+    #   measure 'sources_returned', sources.length
+    #
+    # @example Simple timing in milliseconds
+    #   timing 'myservice.lookup', 2.31
+    #
+    # @example Block-based timing
+    #   timing 'db.query' do
+    #     do_my_query
+    #   end
+    #
+    # @example Custom source
+    #    measure 'user.all_orders', user.order_count, :source => user.id
+    #
+    def measure(*args, &block)
+      options = {}
+      event = args[0].to_s
+      returned = nil
+
+      # handle block or specified argument
+      if block_given?
+        start = Time.now
+        returned = yield
+        value = ((Time.now - start) * 1000.0).to_i
+      elsif args[1]
+        value = args[1]
+      else
+        raise "no value provided"
+      end
+
+      # detect options hash if present
+      if args.length > 1 and args[-1].respond_to?(:each)
+        options = args[-1]
+      end
+
+      log_write(event => value, :source => options[:source])
+      returned
+    end
+    alias :timing :measure
+
+    private
+
+    # take key/value pairs and return an array of measure strings
+    def add_prefixes(measures)
+      measure_prefix = 'measure.'
+      measure_prefix << "#{prefix}." if prefix
+      measures.map { |keyval|
+        joined = keyval.join('=')
+        if keyval[0].to_sym == :source
+          joined
+        else
+          measure_prefix + joined
+        end
+      }
+    end
+
+    # add default source as appropriate
+    def manage_source(measures)
+      if source && !measures[:source]
+        measures[:source] = source  # set default source
+      end
+      if !source && measures.has_key?(:source) && !measures[:source]
+        measures.delete(:source)    # remove empty source
+      end
+      measures
+    end
+
+    def log_write(measures)
+      measure_chunks = add_prefixes(manage_source(measures))
+      log.puts measure_chunks.join(' ')
+    end
+
+  end
+end

data/lib/librato/logreporter/configuration.rb

@@ -0,0 +1,41 @@
+module Librato
+  class LogReporter
+
+    # Handles configuration options and intelligent defaults
+    # for the LogReporter class.
+    #
+    module Configuration
+
+      # current IO to log to
+      def log
+        @log ||= $stdout
+      end
+
+      # set IO to log to
+      def log=(io)
+        @log = io
+      end
+
+      # current prefix
+      def prefix
+        @prefix ||= ENV['LIBRATO_PREFIX']
+      end
+
+      # set prefix
+      def prefix=(prefix)
+        @prefix = prefix
+      end
+
+      # current default source
+      def source
+        @source ||= ENV['LIBRATO_SOURCE']
+      end
+
+      # set default source
+      def source=(source)
+        @source = source
+      end
+
+    end
+  end
+end

data/lib/librato/logreporter/group.rb

@@ -0,0 +1,37 @@
+module Librato
+  class LogReporter
+
+    # Encapsulates state for grouping operations
+    #
+    class Group
+
+      def initialize(options={})
+        @collector = options[:collector]
+        @prefix = "#{options[:prefix]}."
+      end
+
+      def group(prefix)
+        prefix = apply_prefix(prefix)
+        yield self.class.new(collector: @collector, prefix: prefix)
+      end
+
+      def increment(counter, options={})
+        counter = apply_prefix(counter)
+        @collector.increment counter, options
+      end
+
+      def measure(*args, &block)
+        args[0] = apply_prefix(args[0])
+        @collector.measure(*args, &block)
+      end
+      alias :timing :measure
+
+      private
+
+      def apply_prefix(str)
+        "#{@prefix}#{str}"
+      end
+
+    end
+  end
+end

data/lib/librato/logreporter/version.rb

@@ -0,0 +1,5 @@
+module Librato
+  class LogReporter
+    VERSION = '0.1.0'
+  end
+end

data/test/test_helper.rb

@@ -0,0 +1,7 @@
+require 'bundler'
+Bundler.setup
+
+require 'pry'
+require 'minitest/autorun'
+
+require 'librato/logreporter'

data/test/unit/logreporter/configuration_test.rb

@@ -0,0 +1,41 @@
+require 'test_helper'
+require 'stringio'
+
+module Librato
+  class LogReporter
+    class ConfigurationTest < MiniTest::Test
+
+      def setup
+        @reporter = LogReporter.new
+      end
+
+      def test_default_log
+        assert_equal $stdout, @reporter.log
+      end
+
+      def test_setting_log
+        @reporter.log = $stderr
+        assert_equal $stderr, @reporter.log
+      end
+
+      def test_default_prefix
+        assert_equal nil, @reporter.prefix
+      end
+
+      def test_setting_prefix
+        @reporter.prefix = 'mypref'
+        assert_equal 'mypref', @reporter.prefix
+      end
+
+      def test_default_source
+        assert_equal nil, @reporter.source
+      end
+
+      def test_setting_source
+        @reporter.source = 'librato'
+        assert_equal 'librato', @reporter.source
+      end
+
+    end
+  end
+end

data/test/unit/logreporter/group_test.rb

@@ -0,0 +1,48 @@
+require 'test_helper'
+
+module Librato
+  class LogReporter
+    class GroupTest < MiniTest::Test
+
+      def setup
+        @reporter = MiniTest::Mock.new
+        @group = Group.new(collector: @reporter, prefix: 'mypref')
+      end
+
+      def test_increment
+        @reporter.expect :increment, true, ['mypref.foo', {}]
+        @group.increment 'foo'
+        @reporter.verify
+
+        @reporter.expect :increment, true, ['mypref.bar', {:source => 'baz'}]
+        @group.increment 'bar', :source => 'baz'
+        @reporter.verify
+      end
+
+      def test_measure
+        @reporter.expect :measure, true, ['mypref.mpg', 52]
+        @group.measure 'mpg', 52
+        @reporter.verify
+
+        @reporter.expect :measure, true, ['mypref.mpg', 8, {:source => 'vette'}]
+        @group.measure 'mpg', 8, :source => 'vette'
+        @reporter.verify
+      end
+
+      def test_timing
+        @reporter.expect :measure, true, ['mypref.completion']
+        @group.timing('completion') { sleep 0.01 }
+        @reporter.verify
+      end
+
+      def test_nesting_group
+        @reporter.expect :increment, true, ['mypref.more.foo', {}]
+        @group.group :more do |m|
+          m.increment :foo
+        end
+        @reporter.verify
+      end
+
+    end
+  end
+end

data/test/unit/logreporter_test.rb

@@ -0,0 +1,102 @@
+require 'test_helper'
+require 'stringio'
+
+module Librato
+  class LogReporterTest < MiniTest::Test
+
+    def setup
+      @buffer   = StringIO.new
+      @reporter = LogReporter.new
+      @reporter.log = @buffer
+    end
+
+    def test_increment
+      @reporter.increment :foo
+      assert_last_logged 'measure.foo=1'
+
+      @reporter.increment 'foo.bar', :by => 2
+      assert_last_logged 'measure.foo.bar=2'
+    end
+
+    def test_increment_supports_source
+      @reporter.source = 'sf'
+
+      # default source
+      @reporter.increment 'days.foggy'
+      assert_last_logged 'measure.days.foggy=1 source=sf'
+
+      # custom source
+      @reporter.increment 'days.foggy', :source => 'seattle'
+      assert_last_logged 'measure.days.foggy=1 source=seattle'
+    end
+
+    def test_measure
+      @reporter.measure 'documents.rendered', 12
+      assert_last_logged 'measure.documents.rendered=12'
+
+      # custom source
+      @reporter.measure 'cycles.wasted', 23, :source => 'cpu_1'
+      assert_last_logged 'measure.cycles.wasted=23 source=cpu_1'
+    end
+
+    def test_timing
+      @reporter.timing('do.stuff') { sleep 0.1 }
+      last = last_logged
+      assert last =~ /\=/, 'should have a measure pair'
+      key, value = last.split('=')
+      assert_equal 'measure.do.stuff', key, 'should have timing key'
+      assert_in_delta 100, value.to_i, 10
+
+      # custom source
+      @reporter.timing('do.more', :source => 'worker') do
+        sleep 0.05
+      end
+      assert last_logged.index('source=worker'), 'should use custom source'
+    end
+
+    def test_basic_grouping
+      @reporter.group :pages do |p|
+        p.increment :total
+        p.timing :render_time, 63
+        # nested
+        p.group('public') { |pub| pub.increment 'views', :by => 2 }
+      end
+
+      @buffer.rewind
+      lines = @buffer.readlines
+      assert_equal 'measure.pages.total=1', lines[0].chomp
+      assert_equal 'measure.pages.render_time=63', lines[1].chomp
+      assert_equal 'measure.pages.public.views=2', lines[2].chomp
+    end
+
+    def test_custom_prefix
+      @reporter.prefix = 'librato'
+
+      # increment
+      @reporter.increment 'views'
+      assert_last_logged 'measure.librato.views=1'
+
+      # measure/timing
+      @reporter.measure 'sql.queries', 6
+      assert_last_logged 'measure.librato.sql.queries=6'
+
+      # group
+      @reporter.group :private do |priv|
+        priv.increment 'secret'
+      end
+      assert_last_logged 'measure.librato.private.secret=1'
+    end
+
+    private
+
+    def assert_last_logged(string)
+      assert_equal string, last_logged, "Last logged should be '#{string}'."
+    end
+
+    def last_logged
+      @buffer.rewind
+      @buffer.readlines[-1].to_s.chomp
+    end
+
+  end
+end

metadata

@@ -0,0 +1,85 @@
+--- !ruby/object:Gem::Specification
+name: librato-logreporter
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+  prerelease: 
+platform: ruby
+authors:
+- Matt Sanders
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2013-06-21 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: minitest
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+description: Provides a simple interface to log Librato metrics to your log files
+  in l2met format.
+email:
+- matt@librato.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- lib/librato/logreporter/configuration.rb
+- lib/librato/logreporter/group.rb
+- lib/librato/logreporter/version.rb
+- lib/librato/logreporter.rb
+- lib/librato-logreporter.rb
+- LICENSE
+- Rakefile
+- README.md
+- CHANGELOG.md
+- test/test_helper.rb
+- test/unit/logreporter/configuration_test.rb
+- test/unit/logreporter/group_test.rb
+- test/unit/logreporter_test.rb
+homepage: https://github.com/librato/librato-logreporter
+licenses: []
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+      segments:
+      - 0
+      hash: -2188963699629568403
+required_rubygems_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+      segments:
+      - 0
+      hash: -2188963699629568403
+requirements: []
+rubyforge_project: 
+rubygems_version: 1.8.25
+signing_key: 
+specification_version: 3
+summary: Write Librato metrics to your logs with a convenient interface
+test_files:
+- test/test_helper.rb
+- test/unit/logreporter/configuration_test.rb
+- test/unit/logreporter/group_test.rb
+- test/unit/logreporter_test.rb