camayoc 0.1.0 → 0.2.0

data/HISTORY.md

@@ -0,0 +1,9 @@
+## 0.2.0
+
+* Added the Camayoc::Stats#event method to do general event logging
+* Refactored handler event propagation; handlers now implement an event(stat_event) method instead of count and timing
+* Logging and IO handler formatter Procs now take a single StatEvent as an argument
+
+## 0.1.0
+
+* Initial release

data/README.md

@@ -1,9 +1,9 @@
 Overview
 ========
-Camayoc is a flexible way to keep track of application stats. Inspired by 
-various logging frameworks (especially Log4r) it makes it easy to send stats 
-information to multiple destinations via the use of Handlers. Right out of the 
-box, it supports sending stats to the following:
+Camayoc is a flexible way to keep track of application stats and events. 
+Inspired by various logging frameworks (especially Log4r) it makes it easy to 
+send stats information to multiple destinations via the use of Handlers. Right 
+out of the box, it supports sending stats to the following:
 
 * A [statsd daemon](https://github.com/etsy/statsd/) for ridiculously easy [Graphite](http://graphite.wikidot.com/) stats
 * An in-memory hash for in-process stats
@@ -12,13 +12,25 @@ box, it supports sending stats to the following:
 
 Philosophy
 ----------
-Application stats are critical. But even critical things get ignored if they're 
-hard (believe me, we know). Stats should be easy:
+Keeping track of what goes on in your application is critical. But even 
+critical things get ignored if they're hard (believe me, we know). Camayoc's aim 
+is to make logging events and capturing stats easy:
 
-* Collecting stats should take just one line of code
-* All stat collection should be fire-and-forget with no error handling required
-* Organizing stats should be easy
-* There should be essentially no performance cost to collecting stats
+* Collecting this data should take just one line of code
+* All data collection should be fire-and-forget with no error handling required
+* Organizing stats and events should be easy
+* There should be essentially no performance cost to collecting this data
+
+Events vs Stats
+---------------
+In general, events are things that happen in your application that you care 
+about. There are two ways to keep track of events:
+
+* Keep summary stats about event occurances around (counts, etc)
+* Keep a detailed record of events that occur in a some sort of log or collection
+
+Through its use of flexible handlers, Camayoc makes it possilbe to do one or 
+both of these through a single simple interface.
 
 Examples
 ==========
@@ -96,6 +108,24 @@ There are other options as well like :if and :unless that take Procs that can
 be executed to determine if a metric should be sent to the specified handler.
 See Camayoc::Handlers::Filter for more.
 
+Event Logging
+-------------
+Sometimes you may want to keep a detailed log of events that occur instead of 
+summarizing then via counts. Below is an example of logging event data in a 
+space-delimited format to an IO stream:
+
+    event_log = Camayoc["events"]
+    fmt = Proc.new do |event| 
+      ([event.ns_stat, Time.now.to_i] + Array(event.value)).join(" ") 
+    end
+    event_log.add(Camayoc::Handlers::IO.new(STDOUT,:formatter=>fmt))
+
+    event_log.event("foo",["bar","baz"])
+
+This will produce the following on stdout:
+
+    events:foo 1321564751 bar baz
+
 Available Handlers
 ==================
 Statsd
@@ -105,6 +135,12 @@ Class: Camayoc::Handlers::Statsd
 This handler sends data to the statd daemon for use in graphite. If you can get 
 graphite and statsd going, then you'll get pretty graphs.
 
+This handler does not support logging details about events since this isn't 
+really what statsd and graphite are for. Any calls to the event method will be 
+treated as counts in statsd. If the event value can be converted to an Integer 
+using Ruby's Integer method, then it will be sent as the count. Otherwise, a 
+value of 1 will be sent.
+
 Memory
 ------
 Class: Camayoc::Handlers::Memory
@@ -112,6 +148,10 @@ Class: Camayoc::Handlers::Memory
 Stores counts and timing data in an in-memory hash. This might be handy for 
 storing some temporary in-process stats.
 
+If you use this handler for event logging, data will be stored in an in-memory 
+array with a configurable max size. If that size is exceeded, older events will 
+be evicted to make room for new events.
+
 Logger
 ------
 Class: Camayoc::Handlers::Logger
@@ -120,6 +160,9 @@ Writes out stat values and a timestamp to a logger instance for later analysis.
 The format and method called on the logger can be controlled by constructor 
 arguments. See the class for details.
 
+This handler is best for event logging, especially when combined with a custom 
+formatter.
+
 IO
 --
 Class: Camayoc::Handlers::IO
@@ -127,6 +170,8 @@ Class: Camayoc::Handlers::IO
 Writes out stats values and a timestamp to some stream of your choice. See 
 class for configuration details.
 
+Another good event logging handler.
+
 Redis (Experimental)
 --------------------
 Class: Camayoc::Handlers::Redis (require "camayoc/handlers/redis" first) 
@@ -137,23 +182,37 @@ time-based stats system. It does make it easy to collect some simple counts and
 some timing data. You can easily retrieve the stored data from redis by using 
 the #get method.
 
+This handler does not currently support event logging.
+
 
 Implementing a Handler
 ======================
-Let's say you want to implement your own handler, pehaps to MongoDB. Handlers 
+
+v0.2+ Incompatibility Notice
+----------------------------
+As of Version 0.2.0, handlers must implement an event method. In 0.1.0, 
+handlers had to implement count and timing methods. Now handlers are encouraged 
+to dispatch events as described below.
+
+Implementation Example
+---------------------- 
+Let's say you want to implement your own handler, perhaps to MongoDB. Handlers 
 just need to respond to a simple interface. See Camayoc::Handlers::StatEvent 
-for info on the argument to the two methods.
+for info on the argument to the event method.
 
     class SomeHandler
-      def count(stat_event)
-        # Do something
-      end
-      
-      def timing(stat_event)
-        # Do something
+      def event(evt)
+        case evt.type
+          when :count   then handle_count(evt)  # do something with a count stat
+          when :timing  then handle_timing(evt) # do something with a timing stat
+          when :generic then handle_other(evt)  # do something with a generic event
       end
     end
 
+If you were writing a MongoDB handler, the above might increment a value in 
+a collection on :count and :timing and store raw data to a collection for 
+:generic events.
+
 If you write a handler and would like it included in Camayoc, please fork 
 and send a pull request and we'll get it integrated in.
 

data/lib/camayoc.rb

@@ -10,7 +10,7 @@ require 'camayoc/handlers/memory'
 require 'camayoc/handlers/statsd'
 require 'camayoc/stats'
 
-module Camayoc  
+module Camayoc
 
   DELIMITER = ":"
 
@@ -19,7 +19,7 @@ module Camayoc
 
   class << self
     def [](name)
-      @lock.synchronize { @registry[name] ||= _new(name) }  
+      @lock.synchronize { @registry[name] ||= _new(name) }
     end
 
     def new(name)
@@ -33,11 +33,15 @@ module Camayoc
     def thread_safe=(value)
       @thread_safe = value
     end
-    
+
     def thread_safe?
       @thread_safe == true
     end
 
+    def join(*names)
+      names.flatten.join(DELIMITER)
+    end
+
     private
       def _new(name)
         stats = Stats.new(name,ancestor(name))
@@ -58,10 +62,10 @@ module Camayoc
         ancestor = nil
         path = name.split(DELIMITER)
         while path.pop && !ancestor
-          ancestor = @registry[path.join(DELIMITER)]
+          ancestor = @registry[join(path)]
         end
         ancestor
       end
   end
 
-end
+end

data/lib/camayoc/handlers/filter.rb

@@ -1,43 +1,38 @@
 module Camayoc
   module Handlers
     class Filter
-      
+
+      # Constructor - by default, returns true
+      # * +dest+ :: Handler
+      # * +options[]+
+      # * +  :with+ :: Regexp matching against event namespace
+      # * +  :if+ :: Proc taking type and event returning true
+      # * +  :unless+ :: Converse of +if+
       def initialize(dest,options={},&block)
         @dest = dest
         if block_given?
           @filter = block
         elsif options[:with]
           pattern = options[:with]
-          @filter = Proc.new {|type,event| event.ns_stat =~ pattern }
+          @filter = Proc.new {|event| event.ns_stat =~ pattern }
         elsif options[:if]
           @filter = options[:if]
         elsif options[:unless]
           proc = options[:unless]
-          @filter = Proc.new do |type,event| 
-            !proc.call(type,event)
+          @filter = Proc.new do |event|
+            !proc.call(event)
           end
         else
-          @filter = Proc.new {|args| true }
-        end
-      end
-
-      def count(event)
-        if allowed?(:count,event)
-          @dest.count(event)
+          @filter = Proc.new { true }
         end
       end
 
-      def timing(event)
-        if allowed?(:timing,event)
-          @dest.timing(event)
+      def event(ev)
+        if @filter.call(ev)
+          @dest.event(ev)
         end
       end
 
-      private
-        def allowed?(type,event)
-          @filter.call(type,event)
-        end
-
     end
   end
-end
+end

data/lib/camayoc/handlers/io.rb

@@ -1,25 +1,25 @@
 module Camayoc
   module Handlers
 
-    # Write to a raw IO stream with optional thread-safe locking before writing 
-    # to the stream. By default, each stat message is written on a single line 
+    # Write to a raw IO stream with optional thread-safe locking before writing
+    # to the stream. By default, each stat message is written on a single line
     # using puts. See the options in Camayoc::Handlers:Logger for options.
     class IO < Logger
 
       include ThreadSafety
 
-      def initialize(io=$stdout,options={})
-        super(io,{:method=>:puts}.merge(options))
+      def initialize(io=$stdout,options={},&block)
+        super(io,{:method=>:puts}.merge(options),&block)
         self.thread_safe = Camayoc.thread_safe?
       end
 
       protected
-        def write(type,event)
+        def write(event)
           synchronize do
-            super(type,event)
+            super(event)
           end
         end
 
     end
   end
-end
+end

data/lib/camayoc/handlers/logger.rb

@@ -1,44 +1,39 @@
 module Camayoc
   module Handlers
-    
-    # Write stats to a logger. Specify the method to call on the logger instance 
-    # with :method (usually something like :info). If not :method is specified 
-    # :debug will be called on the logger. You can control the format of the 
+
+    # Write stats to a logger. Specify the method to call on the logger instance
+    # with :method (usually something like :info). If not :method is specified
+    # :debug will be called on the logger. You can control the format of the
     # message passed to the logger method using the :formatter Proc.
     class Logger
 
       attr_accessor :logger, :method, :formatter
 
-      def initialize(logger, options={})
+      def initialize(logger, options={}, &block)
         self.logger = logger
         self.method = options[:method]
-        self.formatter = (options[:formatter] || default_formatter)
-      end
-
-      def count(event)
-        write(:count,event)
+        if block_given?
+          self.formatter = block
+        else
+          self.formatter = (options[:formatter] || default_formatter)
+        end
       end
 
-      def timing(event)
-        write(:timing,event)
+      def event(ev)
+        msg = formatter.call(ev)
+        if @method
+          @logger.send(@method,msg)
+        else
+          @logger.debug(msg)
+        end
       end
 
       def default_formatter
-        Proc.new do |type,event|
-          "#{type} #{event.ns_stat} #{event.value} #{Time.now.utc.to_i}"
-        end
-      end     
-
-      protected
-        def write(type,event)
-          msg = formatter.call(type,event) 
-          if @method
-            @logger.send(@method,msg)
-          else
-            @logger.debug(msg)
-          end
+        Proc.new do |event|
+          "#{event.type} #{event.ns_stat} #{event.value} #{Time.now.utc.to_i}"
         end
+      end
 
     end
   end
-end
+end

data/lib/camayoc/handlers/memory.rb

@@ -1,25 +1,25 @@
 module Camayoc
   module Handlers
     class Memory
-      
+
       include ThreadSafety
 
+      DEFAULT_MAX_EVENTS = 1024
+
+      attr_reader :max_events
+
       def initialize(options={})
         @data = {}
+        @max_events = options[:max_events].to_i rescue 0
+        @max_events = DEFAULT_MAX_EVENTS if @max_events <= 0
         self.thread_safe = Camayoc.thread_safe?
       end
 
-      def count(event)
-        stat = event.ns_stat
-        synchronize do
-          @data[stat] ||= 0
-          @data[stat] += event.value
-        end
-      end
-
-      def timing(event)
-        synchronize do
-          (@data[event.ns_stat] ||= Timing.new) << event.value
+      def event(ev)
+        case ev.type
+          when :count then count(ev)
+          when :timing then timing(ev)
+          when :generic then generic(ev)
         end
       end
 
@@ -30,6 +30,32 @@ module Camayoc
       end
       alias_method :[], :get
 
+      private
+        def count(ev)
+          stat = ev.ns_stat
+          synchronize do
+            @data[stat] ||= 0
+            @data[stat] += ev.value
+          end
+        end
+
+        def timing(ev)
+          synchronize do
+            (@data[ev.ns_stat] ||= Timing.new) << ev.value
+          end
+        end
+
+        def generic(ev)
+          stat = ev.ns_stat
+          synchronize do
+            @data[stat] ||= []
+            @data[stat].unshift ev.value
+            if @data[stat].length > @max_events
+              @data[stat] = @data[stat][0,@max_events]
+            end
+          end
+        end
+
     end
   end
-end
+end

data/lib/camayoc/handlers/redis.rb

@@ -5,29 +5,25 @@ module Camayoc
 
     # This class is experimental.
     class Redis
-      
+
       def initialize(options={})
         @namespace = options.delete(:namespace)
         @redis = (options[:redis] || ::Redis.new(options))
       end
 
-      def count(event)
-        @redis.incrby(key(event.ns_stat),event.value)
-      end
-
-      def timing(event)
-        stat = event.ns_stat
-        ms = event.value
-        @redis.set(key(stat),"timing")
-        @redis.incrby(key("#{stat}:_count"),1)
-        @redis.incrby(key("#{stat}:_total"),ms)
+      def event(ev)
+        case ev.type
+          when :count  then count(ev)
+          when :timing then timing(ev)
+          # generic not implemented!
+        end
       end
 
       def get(stat)
         value = @redis.get(key(stat))
         if value == "timing"
           Timing.new(
-            @redis.get(key("#{stat}:_total")).to_i, 
+            @redis.get(key("#{stat}:_total")).to_i,
             @redis.get(key("#{stat}:_count")).to_i,
             nil,nil # Don't support min and max right now in redis
           )
@@ -40,10 +36,22 @@ module Camayoc
       alias_method :[], :get
 
       private
+        def count(event)
+          @redis.incrby(key(event.ns_stat),event.value)
+        end
+
+        def timing(event)
+          stat = event.ns_stat
+          ms = event.value
+          @redis.set(key(stat),"timing")
+          @redis.incrby(key("#{stat}:_count"),1)
+          @redis.incrby(key("#{stat}:_total"),ms)
+        end
+
         def key(stat)
           @namespace ? "#{@namespace}:#{stat}" : stat
         end
 
     end
   end
-end
+end