drone 0.0.3 → 1.0.1

data/.rvmrc

@@ -0,0 +1 @@
+rvm use 1.9.2

data/README.md

@@ -14,74 +14,87 @@ A fully working example is included in examples/simple, to run it
   
 The example will output the collected statistics directly on the console every second.
 
+# Supported Runtimes
+
+- MRI 1.8.7+
+- Rubinius 1.2.2+
+
+
+# Status
+ - Most of the features I wanted in are (see below for usage examples):
+  - timing method calls
+  - method calls rate
+  - counters
+  - gauges
+
+ - Good test coverage
+ 
+ The gem was created for a specific need and is currently used in preproduction environment,
+ no major bugs until now.
+
+
 # How is it done
 
 The library is split in different parts
 
-- the core
+- the core:<br/>
   it contains all the API used to declare which data to collect and how as well as the storage for them
 
-- the metrics
+- the metrics:<br/>
   that is all the metrics type the library know.
 
-- the interfaces
+- the interfaces:<br/>
   those are the parts which will decides how the stored data are made available.
 
-- the schedulers
+- the schedulers:</br>
   this is where the timers are scheduled, currently there is only one scheduler: eventmachine
 
+- the storage:<br/>
+  this part decides where the actual data for the metrics are stored, the default is to store them
+  in memory but other possible options are: redis, memcached, etc...
+  The goal for external storage is to allow concurrent applications to share the same metrics, an
+  immediate example of such application is a rails application ran under passenger or any other spawner
+
 ## Constraints
 
 - the name of each metric can be formatted how it pleases you (note that output interfaces may expect some format)
   but the name is expected to be unique or you could end up reusing the same metric without wanting it.
   (this only applies to monitor_time and monitor_rate helpers but could apply anywhere else as needed)
 
-
-# Supported Runtimes
-
-- MRI 1.8.7+
-- Rubinius 1.2.2+
-
-
-# Status
- - Most of the features I wanted in are:
-  - timing method calls
-  - method calls rate
-  - counters
-  - gauges
-
- - Decent test coverage (Simplecov report ~ 87% for what is worth)
-
 # Usage
   
   I try to keep things as simple as possible, there is currently two ways to use
   this library:
   
   - the first one is to just instantiate metrics by hand and use them directly
+  
+      ``` ruby
+      require 'drone'
+      Drone::init_drone()
+      @counter = Drone::Metris::Counter.new('my_counter')
     
-        require 'drone'
-        Drone::init_drone()
-        @counter = Drone::Metris::Counter.new('my_counter')
-        
-        def some_method
-          @counter.inc()
-        end
+      def some_method
+        @counter.inc()
+      end
+      ```
   
   - the other way is to instrument a class:
   
-        require 'drone'
-        Drone::init_drone()
+      ``` ruby
+      require 'drone'
+      Drone::init_drone()
+      
+      class User
+        include Drone::Monitoring
+        
+        monitor_rate("users/new")
+        def initialize(login, pass); end
         
-        class User
-          include Drone::Monitoring
-          
-          monitor_rate("users/new")
-          def initialize(login, pass); end
-          
-          monitor_time("users/rename")
-          def rename(new_login); end
-          
-        end
+        monitor_time("users/rename")
+        def rename(new_login); end
+        
+      end
+      ```
       
       This code will create three metrics:
       - "users/new"       : how many users are created each second
@@ -93,13 +106,23 @@ Once you have your data you need to add a way to serve them, each lives in a sep
 gem to limit the core's dependencies so the only one in core is:
   
   - console output (puts), mainly for debug:
-      
-        require 'drone'
-        Drone::init_drone()
-        Drone::add_output(:console, 1)
+  
+      ``` ruby
+      require 'drone'
+      Drone::init_drone()
+      Drone::add_output(:console, 1)
+      ```
       
       The values will be printed on the console at the inter
   
+  The others output are available in their own gems:
+  
+  - drone_json:<br/>
+    The stats are served by a thin server in json
+  
+  - drone_collectd:<br/>
+    The stats are send to a collectd daemon.
+  
 # Goals
 
   My goal is to be able to serve stats efficiently from any ruby 1.9 application built
@@ -113,23 +136,24 @@ gem to limit the core's dependencies so the only one in core is:
   optional part instead of in the core. There should not be any problem to implements it
   in an includable module not included as default (it may requires some modifications in the core):
   
-    require 'drone'
-    require 'drone/threadsafe'
+  ``` ruby
+  require 'drone'
+  require 'drone/threadsafe'
     
-    [...]
-  
+  # [...]
+  ```
   
 # Development
 
   Installing the development environment is pretty simple thanks to bundler:
-    
+  
     gem install bundler
     bundle
   
 ## Running specs
   
   The specs are written with bacon, mocha and em-spec, they can be ran with:
-    
+  
     rake spec
   
 ## Build the doc

data/Rakefile

@@ -15,6 +15,6 @@ begin
   require 'yard'
   require 'bluecloth'
   YARD::Rake::YardocTask.new(:doc)
-rescue
+rescue LoadError
   
 end

data/examples/simple.rb

@@ -1,8 +1,7 @@
-require 'rubygems'
-require 'bundler/setup'
+require File.expand_path('../common', __FILE__)
+init_environment()
 
-$LOAD_PATH.unshift(File.expand_path('../../lib', __FILE__))
-require 'drone'
+require 'fiber'
 
 Drone::init_drone()
 Drone::register_gauge("cpu:0/user"){ rand(200) }
@@ -22,11 +21,9 @@ class User
   monitor_time("users:do_something:time")
   monitor_rate("users:do_something:rate")
   def do_something
-    # just eat some cpu
-    0.upto(rand(2000)) do |n|
-      str = "a"
-      200.times{ str << "b" }
-    end
+    # fb = Fiber.current
+    # EM::add_timer(1){ fb.resume() }
+    # Fiber.yield
   end
 end
 
@@ -45,7 +42,9 @@ EM::run do
     counter1.increment()
   end
   
-  EM::add_periodic_timer(1) do
-    a.do_something()
+  EM::add_periodic_timer(2) do
+    Fiber.new do
+      a.do_something()
+    end.resume
   end
 end

data/lib/drone/core.rb

@@ -1,6 +1,10 @@
+require 'forwardable'
+
 
 require File.expand_path('../schedulers/eventmachine', __FILE__)
 
+require File.expand_path('../storage/memory', __FILE__)
+
 module Drone
   ##
   # This module contains all the metrics you can use to collect data
@@ -20,11 +24,20 @@ module Drone
   # 
   module Schedulers; end
   
+  
+  ##
+  # This module contains the class used for storage,
+  # they determine where the metric's data are stored
+  # 
+  module Storage; end
+  
   class <<self
+    extend Forwardable
     
-    def init_drone(scheduler = Schedulers::EMScheduler)
-      @meters = []
+    def init_drone(scheduler = Schedulers::EMScheduler, storage = Storage::Memory.new)
+      @metrics = []
       @scheduler = scheduler
+      @storage = storage
       @monitored_classes = []
       @output_modules = []
     end
@@ -41,7 +54,7 @@ module Drone
     
     def each_metric
       raise "Block expected" unless block_given?
-      @meters.each{|m| yield(m) }
+      @metrics.each{|m| yield(m) }
     end
     
     
@@ -51,7 +64,7 @@ module Drone
     # @param [String] name The mtric's name
     # 
     def find_metric(name)
-      @meters.detect{|m| m.name == name }
+      @metrics.detect{|m| m.name == name }
     end
     
     ##
@@ -67,59 +80,62 @@ module Drone
       @output_modules << klass.new(*args)
     end
     
-    ##
-    # Register a monitored class.
-    # @private
-    # 
-    def register_monitored_class(klass)
-      @monitored_classes << klass
-    end
-    
     ##
     # Register a new counter
+    # @see Drone::Metrics::Counter
     # @param [String] type Name of this metric
     # @api public
     # 
     def register_counter(type)
-      register_meter( Drone::Metrics::Counter.new(type) )
+      register_metric( Drone::Metrics::Counter.new(type) )
+    end
+    
+    
+    ##
+    # Register an Histogram
+    # @see Drone::Metrics::Histogram
+    # @param [String] name Name of this metric
+    # @param [optional,Enum] type one of Drone::Metrics::Histogram::TYPE_UNIFORM or Drone::Metrics::Histogram::TYPE_BIASED
+    # 
+    def register_histogram(name, type = :uniform)
+      register_metric( Drone::Metrics::Histogram.new(name, type) )
     end
     
     ##
     # Register a new gauge
+    # @see Drone::Metrics::Gauge
     # @param [String] type Name of this metric
     # @api public
     # 
     def register_gauge(type, &block)
-      register_meter( Drone::Metrics::Gauge.new(type, &block) )
+      register_metric( Drone::Metrics::Gauge.new(type, &block) )
     end
     
     ##
-    # Register a new meter
+    # Register a new metric
     # This method can be used bu the user but the prefered
     # way is to use the register_counter / register_gauge methods
     # 
-    # @param [Meter] meter The Meter to register
-    # @api private
+    # @param [Metric] metric The Metric to register
     # @private
     # 
-    def register_meter(meter)
-      @meters << meter
-      meter
+    def register_metric(metric)
+      @metrics << metric
+      metric
     end
     
-    ##
-    # (see EMScheduler#schedule_periodic)
-    # 
-    def schedule_periodic(*args, &block)
-      @scheduler.schedule_periodic(*args, &block)
-    end
+    
+    def_delegators :@storage, :request_fixed_size_array, :request_number, :request_hash
+    def_delegators :@scheduler, :schedule_periodic, :schedule_once
+    
+    
     
     ##
-    # (see EMScheduler#schedule_once)
+    # Register a monitored class.
+    # @private
     # 
-    def schedule_once(*args, &block)
-      @scheduler.schedule_once(*args, &block)
+    def register_monitored_class(klass)
+      @monitored_classes << klass
     end
-    
   end
 end

data/lib/drone/metrics/counter.rb

@@ -1,26 +1,38 @@
+require File.expand_path('../metric', __FILE__)
+
 module Drone
   module Metrics
     
-    class Counter
-      attr_reader :value, :name
+    ##
+    # A Counter store a number which can go up or down,
+    # the counter can change a counter value with
+    # the methods increment and decrement aliased
+    # as inc and dec
+    # 
+    class Counter < Metric
     
       def initialize(name, initial_value = 0)
-        @name = name
-        @value = initial_value
+        super(name)
+        
+        @value = Drone::request_number("#{name}:value", initial_value)
+      end
+      
+      def value
+        @value.get
       end
     
       def increment(n = 1)
-        @value += n
+        @value.inc(n)
       end
       alias :inc :increment
     
       def decrement(n = 1)
-        @value -= n
+        @value.dec(n)
       end
       alias :dec :decrement
     
       def clear
-        @value = 0
+        @value.set(0)
       end
     end
     

data/lib/drone/metrics/gauge.rb

@@ -1,3 +1,5 @@
+require File.expand_path('../metric', __FILE__)
+
 module Drone
   module Metrics
     
@@ -6,12 +8,11 @@ module Drone
     # will be called when the value is asked, the block
     # is expected to return a number
     # 
-    class Gauge
-      attr_reader :name
+    class Gauge < Metric
     
       def initialize(name, &block)
         raise "Block expected" unless block
-        @name = name
+        super(name)
         @block = block
       end
     

data/lib/drone/metrics/histogram.rb

@@ -1,22 +1,42 @@
 require File.expand_path('../../utils/uniform_sample', __FILE__)
 require File.expand_path('../../utils/exponentially_decaying_sample', __FILE__)
+require File.expand_path('../metric', __FILE__)
 
 module Drone
-  class Histogram
-    TYPE_UNIFORM  = lambda{ UniformSample.new(1028) }
-    TYPE_BIASED   = lambda{ ExponentiallyDecayingSample.new(1028, 0.015) }
-    
+  
+  ##
+  # An Histogram store a list of values (1028) and can
+  # compute on demand statistics on those values:
+  # - min/max
+  # - mean
+  # - stddev
+  # - percentiles
+  # 
+  class Histogram < Metric    
     MIN = (-(2**63)).freeze
     MAX = ((2**64) - 1).freeze
     
-    def initialize(sample_or_type = TYPE_UNIFORM)
-      if sample_or_type.is_a?(Proc)
-        @sample = sample_or_type.call()
+    def initialize(name, sample_or_type = :uniform)
+      super(name)
+      
+      if sample_or_type.is_a?(Symbol)
+        case sample_or_type
+        when :uniform   then  @sample = UniformSample.new("#{name}:sample", 1028)
+        when :biased    then  @sample = ExponentiallyDecayingSample.new("#{name}:sample", 1028, 0.015)
+        else
+          raise ArgumentError, "unknown type: #{sample_or_type}"
+        end
       else
         @sample = sample_or_type
       end
       
-      clear()
+      @count = Drone::request_number("#{name}:count", 0)
+      @_min = Drone::request_number("#{name}:min", MAX)
+      @_max = Drone::request_number("#{name}:max", MIN)
+      @_sum = Drone::request_number("#{name}:max", 0)
+      @varianceM = Drone::request_number("#{name}:varianceM", -1)
+      @varianceS = Drone::request_number("#{name}:varianceS", 0)
+      
     end
     
     def clear
@@ -30,37 +50,37 @@ module Drone
     end
     
     def update(val)
-      @count += 1
+      @count.inc
       @sample.update(val)
       set_max(val);
       set_min(val);
-      @_sum += val
+      @_sum.inc(val)
       update_variance(val)
     end
     
     def count
-      @count
+      @count.get
     end
     
     def max
-      (@count > 0) ? @_max : 0.0
+      (@count.get > 0) ? @_max.get : 0.0
     end
     
     def min
-      (@count > 0) ? @_min : 0.0
+      (@count.get > 0) ? @_min.get : 0.0
     end
     
     def mean
-      (@count > 0) ? @_sum.to_f / @count : 0.0
+      (@count.get > 0) ? @_sum.get.to_f / @count.get : 0.0
     end
     
     def stdDev
-      (@count > 0) ? Math.sqrt( variance() ) : 0.0
+      (@count.get > 0) ? Math.sqrt( variance() ) : 0.0
     end
     
     def percentiles(*percentiles)
       scores = Array.new(percentiles.size, 0)
-      if @count > 0
+      if @count.get > 0
         values = @sample.values.sort
         percentiles.each.with_index do |p, i|
           pos = p * (values.size + 1)
@@ -94,36 +114,37 @@ module Drone
     end
     
     def update_variance(val)
-      if @varianceM == -1
-        @varianceM = doubleToLongBits(val)
+      if @varianceM.get == -1
+        @varianceM.set( doubleToLongBits(val) )
       else
-        oldMCas = @varianceM
+        oldMCas = @varianceM.get
         oldM = longBitsToDouble(oldMCas)
         newM = oldM + ((val - oldM) / count())
         
-        oldSCas = @varianceS
+        oldSCas = @varianceS.get
         oldS = longBitsToDouble(oldSCas)
         newS = oldS + ((val - oldM) * (val - newM))
         
-        @varianceM = doubleToLongBits(newM)
-        @varianceS = doubleToLongBits(newS)
+        @varianceM.set( doubleToLongBits(newM) )
+        @varianceS.set( doubleToLongBits(newS) )
       end
     end
     
     def variance
-      if @count <= 1
+      count = @count.get
+      if count <= 1
         0.0
       else
-        longBitsToDouble(@varianceS) / (count() - 1)
+        longBitsToDouble(@varianceS.get) / (count - 1)
       end
     end
     
     def set_max(val)
-      (@_max >= val) || @_max = val
+      (@_max.get >= val) || @_max.set(val)
     end
     
     def set_min(val)
-      (@_min <= val) || @_min = val
+      (@_min.get <= val) || @_min.set(val)
     end