librato-metrics 0.7.5 → 1.0.0

data/CHANGELOG.md

@@ -1,8 +1,13 @@
 ## Changelog
 
+### Version 1.0.0
+* Add support for annotation submission, listing, management
+* Auto-convert Time objects anywhere a time is accepted
+* Don't raise exception anymore for empty queue submission
+
 ### Version 0.7.5
 * Catch a broader range of connection failures for retrying
-* Add Metrics.faraday_adapter config option (Mathieu Ravaux)_
+* Add Metrics.faraday_adapter config option (Mathieu Ravaux)
 
 ### Version 0.7.4
 * Support global measure_time option for Queues/Aggregators

data/README.md

@@ -14,13 +14,13 @@ In your shell:
 Then, in your application or script:
 
     require 'librato/metrics'
-    
+
 ### Optional steps
 
 For best performance we recommend installing [yajl-ruby](https://github.com/brianmario/yajl-ruby):
 
     gem install yajl-ruby
-    
+
 If you are using jruby, you need to ensure [jruby-openssl](https://github.com/jruby/jruby-ossl) is available:
 
     gem install jruby-openssl
@@ -52,18 +52,18 @@ Queue up a simple gauge metric named `temperature`:
 
     queue = Librato::Metrics::Queue.new
     queue.add :temperature => 32.2
-    
+
 While symbols are used by convention for metric names, strings will work just as well:
 
 	queue.add 'myapp.request_time' => 86.7
 
-If you are tracking measurements over several seconds/minutes, the queue will handle storing measurement time for you (otherwise all metrics will be recorded as measured when they are submitted). 
+If you are tracking measurements over several seconds/minutes, the queue will handle storing measurement time for you (otherwise all metrics will be recorded as measured when they are submitted).
 
 If you want to specify a time other than queuing time for the measurement:
 
 	# use a epoch integer
 	queue.add :humidity => {:measure_time => 1336508422, :value => 48.2}
-	
+
 	# use a Time object to correct for a 5 second delay
 	queue.add :humidity => {:measure_time => Time.now-5, :value => 37.6}
 
@@ -125,23 +125,52 @@ If you need extra attributes for a `Queue` timing measurement, simply add them o
     queue.time :my_measurement, :source => 'app1' do
       # do work...
     end
-    
+
+## Annotations
+
+Annotation streams are a great way to track events like deploys, backups or anything else that might affect your system. They can be overlaid on any other metric stream so you can easily see the impact of changes.
+
+At a minimum each annotation needs to be assigned to a stream and to have a title. Let's add an annotation for deploying v45 of our app to the `deployments` stream:
+
+    Librato::Metrics.annotate :deployments, 'deployed v45'
+
+There are a number of optional fields which can make annotations even more powerful:
+
+    Librato::Metrics.annotate :deployments, 'deployed v46', :source => 'frontend',
+    						  :start_time => 1354662596, :end_time => 1354662608,
+    						  :description => 'Deployed 6f3bc6e67682: fix lotsa bugs…'
+
+More fine-grained control of annotations is available via the `Annotator` object:
+
+    annotator = Librato::Metrics::Annotator.new
+
+    # list annotation streams
+    streams = annotator.list
+
+    # fetch a list of events in the last hour from a stream
+    annotator.fetch :deployments, :start_time => (Time.now.to_i-3600)
+
+    # delete an event
+    annotator.delete_event 'deployments', 23
+
+See the documentation of `Annotator` for more details and examples of use.
+
 ## Auto-Submitting Metrics
 
 Both `Queue` and `Aggregator` support automatically submitting measurements on a given time interval:
 
 	# submit once per minute
 	timed_queue = Librato::Metrics::Queue.new(:autosubmit_interval => 60)
-	
+
 	# submit every 5 minutes
 	timed_aggregator = Librato::Metrics::Aggregator.new(:autosubmit_interval => 300)
-	
+
 `Queue` also supports auto-submission based on measurement volume:
 
 	# submit when the 400th measurement is queued
 	volume_queue = Librato::Metrics::Queue.new(:autosubmit_count => 400)
 
-These options can also be combined for more flexible behavior. 
+These options can also be combined for more flexible behavior.
 
 Both options are driven by the addition of measurements. Specifically for time-based autosubmission if you are adding measurements irregularly (less than once per second), submission may lag past your specified interval until the next measurement is added.
 
@@ -181,7 +210,7 @@ If you ever need to remove a metric and all of its measurements, doing so is eas
 
 	# Delete the metrics 'temperature' and 'humidity'
 	Librato::Metrics.delete :temperature, :humidity
-	
+
 Note that deleted metrics and their measurements are unrecoverable, so use with care.
 
 ## Using Multiple Accounts Simultaneously
@@ -190,7 +219,7 @@ If you need to use metrics with multiple sets of authentication credentials simu
 
     joe = Librato::Metrics::Client.new
     joe.authenticate 'email1', 'api_key1'
-    
+
     mike = Librato::Metrics::Client.new
     mike.authenticate 'email2', 'api_key2'
 
@@ -198,16 +227,16 @@ All of the same operations you can call directly from `Librato::Metrics` are ava
 
 	# list Joe's metrics
 	joe.list
-	
-	# fetch the last 20 data points for Mike's metric, humidity 
+
+	# fetch the last 20 data points for Mike's metric, humidity
 	mike.fetch :humidity, :count => 20
-	
+
 There are two ways to associate a new queue with a client:
 
 	# these are functionally equivalent
 	joe_queue = Librato::Metrics::Queue.new(:client => joe)
 	joe_queue = joe.new_queue
-    
+
 Once the queue is associated you can use it normally:
 
 	joe_queue.add :temperature => {:source => 'sf', :value => 65.2}

data/lib/librato/metrics.rb

@@ -5,6 +5,7 @@ require 'base64'
 require 'multi_json'
 
 require 'metrics/aggregator'
+require 'metrics/annotator'
 require 'metrics/client'
 require 'metrics/collection'
 require 'metrics/connection'
@@ -67,12 +68,14 @@ module Librato
     PLURAL_TYPES = [:counters, :gauges]
     MIN_MEASURE_TIME = (Time.now-(3600*24*365)).to_i
 
-    # Expose class methods of Simple via Metrics itself.
+    # Most of the singleton methods of Librato::Metrics are actually
+    # being called on a global Client instance. See further docs on
+    # Client.
     #
-    def_delegators :client, :agent_identifier, :api_endpoint,
+    def_delegators :client, :agent_identifier, :annotate, :api_endpoint,
                    :api_endpoint=, :authenticate, :connection, :delete,
-                   :faraday_adapter, :faraday_adapter=, :fetch, :list, 
-                   :persistence, :persistence=, :persister, :submit, 
+                   :faraday_adapter, :faraday_adapter=, :fetch, :list,
+                   :persistence, :persistence=, :persister, :submit,
                    :update
 
     # The Librato::Metrics::Client being used by module-level

data/lib/librato/metrics/annotator.rb

@@ -0,0 +1,106 @@
+module Librato::Metrics
+
+  # manages writing and reading annotation streams for a
+  # given client connection
+  class Annotator
+
+    def initialize(options={})
+      @client = options[:client] || Librato::Metrics.client
+    end
+
+    # Creates a new annotation on the annotation stream
+    #
+    # @example Simple annotation
+    #   annotator.add :deployments, 'deployed v45'
+    #
+    # @example Annotation with start and end times
+    #   annotator.add :deployments, 'deployed v56', :start_time => start,
+    #                 :end_time => end
+    #
+    # @example Annotation with a specific source
+    #   annotator.add :deployments, 'deployed v60', :source => 'app12'
+    #
+    # @example Annotation with a description
+    #   annotator.add :deployments, 'deployed v61',
+    #                 :description => '9b562b2: shipped new feature foo!'
+    #
+    def add(stream, title, options={})
+      options[:title] = title
+      if options[:start_time]
+        options[:start_time] = options[:start_time].to_i
+      end
+      if options[:end_time]
+        options[:end_time] = options[:end_time].to_i
+      end
+      payload = SmartJSON.write(options)
+      # expects 200
+      connection.post("annotations/#{stream}", payload)
+    end
+
+    def client
+      @client
+    end
+
+    def connection
+      client.connection
+    end
+
+    # Delete an annotation streams
+    #
+    # @example Delete 'deployment' annotation stream
+    #  annotator.delete :deployment
+    #
+    def delete(stream)
+      connection.delete do |request|
+        request.url connection.build_url("annotations/#{stream}")
+      end
+      # expects 204, middleware will raise exception otherwise
+      true
+    end
+
+    # Delete an event from a given annotation stream
+    #
+    # @example Delete event with id 42 from 'deployment'
+    #   annotator.delete_event :deployment, 42
+    #
+    def delete_event(stream, id)
+      connection.delete do |request|
+        request.url connection.build_url("annotations/#{stream}/#{id}")
+      end
+      # expects 204, middleware will raise exception otherwise
+      true
+    end
+
+    # Get a list of annotation events on a given annotation stream
+    #
+    # @example See properties of the 'deployments' annotation stream
+    #   annotator.fetch :deployments
+    #
+    # @example Get events on 'deployments' between start and end times
+    #   annotator.fetch :deployments, :start_time => start, :end_time => end
+    #
+    # @example Source-limited listing
+    #   annotator.fetch :deployments, :sources => ['foo','bar','baz'],
+    #                   :start_time => start, :end_time => end
+    #
+    def fetch(stream, options={})
+      response = connection.get("annotations/#{stream}", options)
+      SmartJSON.read(response.body)
+    end
+
+    # List currently existing annotation streams
+    #
+    # @example List all annotation streams
+    #   streams = annotator.list
+    #
+    # @example List annotator streams with 'deploy' in the name
+    #   deploy_streams = annotator.list :name => 'deploy'
+    #
+    def list(options={})
+      response = connection.get("annotations", options)
+      SmartJSON.read(response.body)
+    end
+
+  end
+
+end

data/lib/librato/metrics/client.rb

@@ -2,6 +2,10 @@ module Librato
   module Metrics
 
     class Client
+      extend Forwardable
+
+      def_delegator :annotator, :add, :annotate
+
       attr_accessor :email, :api_key
 
       # Provide agent identifier for the developer program. See:
@@ -26,6 +30,10 @@ module Librato
         @agent_identifier ||= ''
       end
 
+      def annotator
+        @annotator ||= Annotator.new(:client => self)
+      end
+
       # API endpoint to use for queries and direct
       # persistence.
       #
@@ -57,7 +65,7 @@ module Librato
       def connection
         # prevent successful creation if no credentials set
         raise CredentialsMissing unless (self.email and self.api_key)
-        @connection ||= Connection.new(:client => self, :api_endpoint => api_endpoint, 
+        @connection ||= Connection.new(:client => self, :api_endpoint => api_endpoint,
                                        :adapter => faraday_adapter)
       end
 
@@ -94,14 +102,14 @@ module Librato
         # otherwise.
         true
       end
-      
+
       # Return current adapter this client will use.
       # Defaults to Metrics.faraday_adapter if set, otherwise
       # Faraday.default_adapter
       def faraday_adapter
         @faraday_adapter ||= default_faraday_adapter
       end
-      
+
       # Set faraday adapter this client will use
       def faraday_adapter=(adapter)
         @faraday_adapter = adapter
@@ -234,7 +242,7 @@ module Librato
       end
 
     private
-    
+
       def default_faraday_adapter
         if Metrics.client == self
           Faraday.default_adapter

data/lib/librato/metrics/errors.rb

@@ -5,7 +5,6 @@ module Librato
     class MetricsError < StandardError; end
 
     class CredentialsMissing < MetricsError; end
-    class NoMetricsQueued < MetricsError; end
     class NoMetricsProvided < MetricsError; end
     class NoClientProvided < MetricsError; end
     class InvalidMeasureTime < MetricsError; end

data/lib/librato/metrics/processor.rb

@@ -27,7 +27,7 @@ module Librato
       #
       # @return Boolean
       def submit
-        raise(NoMetricsQueued, "No metrics queued.") if self.queued.empty?
+        return true if self.queued.empty?
         options = {:per_request => @per_request}
         if persister.persist(self.client, self.queued, options)
           @last_submit_time = Time.now
@@ -84,7 +84,7 @@ module Librato
         @client = options[:client] || Librato::Metrics.client
         @per_request = options[:per_request] || MEASUREMENTS_PER_REQUEST
         @source = options[:source]
-        @measure_time = options[:measure_time]
+        @measure_time = options[:measure_time] && options[:measure_time].to_i
         @create_time = Time.now
         @clear_on_failure = options[:clear_failures] || false
         @prefix = options[:prefix]

data/lib/librato/metrics/queue.rb

@@ -33,6 +33,7 @@ module Librato
           end
           type = ("#{type}s").to_sym
           if metric[:measure_time]
+            metric[:measure_time] = metric[:measure_time].to_i
             check_measure_time(metric)
           elsif !skip_measurement_times
             metric[:measure_time] = epoch_time
@@ -123,7 +124,7 @@ module Librato
     private
 
       def check_measure_time(data)
-        if data[:measure_time].to_i < Metrics::MIN_MEASURE_TIME
+        if data[:measure_time] < Metrics::MIN_MEASURE_TIME
           raise InvalidMeasureTime, "Measure time for submitted metric (#{data}) is invalid."
         end
       end

data/lib/librato/metrics/version.rb

@@ -1,5 +1,5 @@
 module Librato
   module Metrics
-    VERSION = "0.7.5"
+    VERSION = "1.0.0"
   end
 end

data/spec/integration/metrics/annotator_spec.rb

@@ -0,0 +1,139 @@
+require 'spec_helper'
+
+module Librato
+  module Metrics
+
+    describe Annotator do
+      before(:all) { prep_integration_tests }
+      before(:each) { delete_all_annotations }
+
+      describe "#add" do
+        it "should create new annotation" do
+          subject.add :deployment, "deployed v68"
+          annos = subject.fetch(:deployment, :start_time => Time.now.to_i-60)
+          annos["events"]["unassigned"].length.should == 1
+          annos["events"]["unassigned"][0]["title"].should == 'deployed v68'
+        end
+        it "should support sources" do
+          subject.add :deployment, 'deployed v69', :source => 'box1'
+          annos = subject.fetch(:deployment, :start_time => Time.now.to_i-60)
+          annos["events"]["box1"].length.should == 1
+          first = annos["events"]["box1"][0]
+          first['title'].should == 'deployed v69'
+        end
+        it "should support start and end times" do
+          start_time = Time.now.to_i-120
+          end_time = Time.now.to_i-30
+          subject.add :deployment, 'deployed v70', :start_time => start_time,
+                      :end_time => end_time
+          annos = subject.fetch(:deployment, :start_time => Time.now.to_i-180)
+          annos["events"]["unassigned"].length.should == 1
+          first = annos["events"]["unassigned"][0]
+          first['title'].should == 'deployed v70'
+          first['start_time'].should == start_time
+          first['end_time'].should == end_time
+        end
+        it "should support description" do
+          subject.add :deployment, 'deployed v71', :description => 'deployed foobar!'
+          annos = subject.fetch(:deployment, :start_time => Time.now.to_i-180)
+          annos["events"]["unassigned"].length.should == 1
+          first = annos["events"]["unassigned"][0]
+          first['title'].should == 'deployed v71'
+          first['description'].should == 'deployed foobar!'
+        end
+      end
+
+      describe "#delete" do
+        it "should remove annotation streams" do
+          subject.add :deployment, "deployed v45"
+          subject.fetch :deployment # should exist
+          subject.delete :deployment
+          lambda {
+            subject.fetch(:deployment)
+          }.should raise_error(Metrics::NotFound)
+        end
+      end
+
+      describe "#delete_event" do
+        it "should remove an annotation event" do
+          subject.add :deployment, 'deployed v46'
+          subject.add :deployment, 'deployed v47'
+          events = subject.fetch(:deployment, :start_time => Time.now.to_i-60)
+          events = events['events']['unassigned']
+          ids = events.each_with_object({}) do |event, hash|
+            hash[event['title']] = event['id']
+          end
+          subject.delete_event :deployment, ids['deployed v47']
+          events = subject.fetch(:deployment, :start_time => Time.now.to_i-60)
+          events = events['events']['unassigned']
+          events.length.should == 1
+          events[0]['title'].should == 'deployed v46'
+        end
+      end
+
+      describe "#fetch" do
+        context "without a time frame" do
+          it "should return stream properties" do
+            subject.add :backups, "backup 21"
+            properties = subject.fetch :backups
+            properties['name'].should == 'backups'
+          end
+        end
+
+        context "with a time frame" do
+          it "should return set of annotations" do
+            subject.add :backups, "backup 22"
+            subject.add :backups, "backup 23"
+            annos = subject.fetch :backups, :start_time => Time.now.to_i-60
+            events = annos['events']['unassigned']
+            events[0]['title'].should == 'backup 22'
+            events[1]['title'].should == 'backup 23'
+          end
+          it "should respect source limits" do
+            subject.add :backups, "backup 24", :source => 'server_1'
+            subject.add :backups, "backup 25", :source => 'server_2'
+            subject.add :backups, "backup 26", :source => 'server_3'
+            annos = subject.fetch :backups, :start_time => Time.now.to_i-60,
+                                  :sources => %w{server_1 server_3}
+            annos['events']['server_1'].should_not be_nil
+            annos['events']['server_2'].should be_nil
+            annos['events']['server_3'].should_not be_nil
+          end
+        end
+
+        it "should return exception if annotation is missing" do
+          lambda {
+            subject.fetch :backups
+          }.should raise_error(Metrics::NotFound)
+        end
+      end
+
+      describe "#list" do
+        before(:each) do
+          subject.add :backups, 'backup 1'
+          subject.add :deployment, 'deployed v74'
+        end
+
+        context "without arguments" do
+          it "should list annotation streams" do
+            streams = subject.list
+            streams['annotations'].length.should == 2
+            streams = streams['annotations'].map{|i| i['name']}
+            streams.should include('backups')
+            streams.should include('deployment')
+          end
+        end
+        context "with an argument" do
+          it "should list annotation streams which match" do
+            streams = subject.list :name => 'back'
+            streams['annotations'].length.should == 1
+            streams = streams['annotations'].map{|i| i['name']}
+            streams.should include('backups')
+          end
+        end
+      end
+
+    end
+
+  end
+end

data/spec/integration/metrics_spec.rb

@@ -4,6 +4,45 @@ module Librato
   describe Metrics do
     before(:all) { prep_integration_tests }
 
+    describe "#annotate" do
+      before(:all) { @annotator = Metrics::Annotator.new }
+      before(:each) { delete_all_annotations }
+
+      it "should create new annotation" do
+        Metrics.annotate :deployment, "deployed v68"
+        annos = @annotator.fetch(:deployment, :start_time => Time.now.to_i-60)
+        annos["events"]["unassigned"].length.should == 1
+        annos["events"]["unassigned"][0]["title"].should == 'deployed v68'
+      end
+      it "should support sources" do
+        Metrics.annotate :deployment, 'deployed v69', :source => 'box1'
+        annos = @annotator.fetch(:deployment, :start_time => Time.now.to_i-60)
+        annos["events"]["box1"].length.should == 1
+        first = annos["events"]["box1"][0]
+        first['title'].should == 'deployed v69'
+      end
+      it "should support start and end times" do
+        start_time = Time.now.to_i-120
+        end_time = Time.now.to_i-30
+        Metrics.annotate :deployment, 'deployed v70', :start_time => start_time,
+                    :end_time => end_time
+        annos = @annotator.fetch(:deployment, :start_time => Time.now.to_i-180)
+        annos["events"]["unassigned"].length.should == 1
+        first = annos["events"]["unassigned"][0]
+        first['title'].should == 'deployed v70'
+        first['start_time'].should == start_time
+        first['end_time'].should == end_time
+      end
+      it "should support description" do
+        Metrics.annotate :deployment, 'deployed v71', :description => 'deployed foobar!'
+        annos = @annotator.fetch(:deployment, :start_time => Time.now.to_i-180)
+        annos["events"]["unassigned"].length.should == 1
+        first = annos["events"]["unassigned"][0]
+        first['title'].should == 'deployed v71'
+        first['description'].should == 'deployed foobar!'
+      end
+    end
+
     describe "#delete" do
       before(:each) { delete_all_metrics }
 

data/spec/spec_helper.rb

@@ -21,6 +21,16 @@ RSpec.configure do |config|
     end
   end
 
+  # purge all annotations from test account
+  def delete_all_annotations
+    annotator = Librato::Metrics::Annotator.new
+    streams = annotator.list
+    if streams['annotations']
+      names = streams['annotations'].map{|s| s['name']}
+      names.each { |name| annotator.delete name}
+    end
+  end
+
   # set up test account credentials for integration tests
   def prep_integration_tests
     raise 'no TEST_API_USER specified in environment' unless ENV['TEST_API_USER']

data/spec/unit/metrics/queue_spec.rb

@@ -67,7 +67,7 @@ module Librato
               :description => 'current disk utilization', :measure_time => measure_time,
               :source => 'db2'}
             expected = {:gauges => [{:value => 35.4, :name => 'disk_use', :period => 2,
-              :description => 'current disk utilization', :measure_time => measure_time,
+              :description => 'current disk utilization', :measure_time => measure_time.to_i,
               :source => 'db2'}]}
             subject.queued.should equal_unordered(expected)
           end
@@ -116,7 +116,7 @@ module Librato
           it "should accept time objects" do
             time = Time.now-5
             subject.add :foo => {:measure_time => time, :value => 123}
-            subject.queued[:gauges][0][:measure_time].should == time
+            subject.queued[:gauges][0][:measure_time].should == time.to_i
           end
 
           it "should accept integers" do
@@ -128,7 +128,7 @@ module Librato
           it "should accept strings" do
             time = '1336574713'
             subject.add :foo => {:measure_time => time, :value => 123}
-            subject.queued[:gauges][0][:measure_time].should == time
+            subject.queued[:gauges][0][:measure_time].should == time.to_i
           end
 
           it "should raise exception in invalid time" do

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: librato-metrics
 version: !ruby/object:Gem::Version
-  version: 0.7.5
+  version: 1.0.0
   prerelease: 
 platform: ruby
 authors:
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-11-27 00:00:00.000000000 Z
+date: 2012-12-04 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: faraday
@@ -173,6 +173,7 @@ files:
 - benchmarks/array_vs_set.rb
 - lib/librato/metrics.rb
 - lib/librato/metrics/aggregator.rb
+- lib/librato/metrics/annotator.rb
 - lib/librato/metrics/client.rb
 - lib/librato/metrics/collection.rb
 - lib/librato/metrics/connection.rb
@@ -189,6 +190,7 @@ files:
 - lib/librato/metrics/smart_json.rb
 - lib/librato/metrics/version.rb
 - librato-metrics.gemspec
+- spec/integration/metrics/annotator_spec.rb
 - spec/integration/metrics/connection_spec.rb
 - spec/integration/metrics/middleware/count_requests_spec.rb
 - spec/integration/metrics/queue_spec.rb
@@ -216,7 +218,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '0'
       segments:
       - 0
-      hash: -366569108077334223
+      hash: 3364933911255209794
 required_rubygems_version: !ruby/object:Gem::Requirement
   none: false
   requirements:
@@ -230,6 +232,7 @@ signing_key:
 specification_version: 2
 summary: Ruby wrapper for Librato's Metrics API
 test_files:
+- spec/integration/metrics/annotator_spec.rb
 - spec/integration/metrics/connection_spec.rb
 - spec/integration/metrics/middleware/count_requests_spec.rb
 - spec/integration/metrics/queue_spec.rb