resque-clues 0.1.0

data/.gitignore

@@ -0,0 +1,20 @@
+*.gem
+*.rbc
+.bundle
+.config
+.yardoc
+doc/
+Gemfile.lock
+InstalledFiles
+_yardoc
+coverage
+lib/bundler/man
+pkg
+rdoc
+spec/reports
+test/tmp
+test/version_tmp
+tmp
+*.swp
+*.swo
+.rvmrc

data/.travis.yml

@@ -0,0 +1,15 @@
+---
+language: ruby
+rvm:
+- 1.9.3
+services:
+  - redis-server
+notifications:
+  email: false
+  campfire:
+    rooms:
+    - secure: ! 'j8CJevgIDpE/cQwKm9z5uVY8nXmvQFOoQP/OIieEBMPd7IaSv6YVNMmjHju3
+
+        rKVL0471g0/+hfK1etfxeaygHjLVg1GmOnYH/CK97l3I3BnP5pEu7zz0wSs3
+
+        5Legz1OnsK11U1FAjfSkv088V3/IgyTVegU/SaSiSi03uUsRSVs='

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in resque-clues.gemspec
+gemspec

data/LICENSE.txt

@@ -0,0 +1,22 @@
+Copyright (c) 2013 PeopleAdmin
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,163 @@
+# Resque::Clues
+
+Resque-Clues allows Resque to publish job lifecycle events to some external
+store for analysis.  It also allows for decorating jobs stored in Redis with
+metadata that will be included with the published events.  When coupled with
+tools like Splunk, Logstash/Graphite or Cube, this can be used to:
+
+* Quantify results of balancing efforts, hardware changes, etc...
+* See how your background processes perform over time, before and after
+releases, etc...
+* Break down performance metrics on metadata specific to your business or
+  domain.
+* Provide searchability for specific jobs entering the queue to aid in
+  debugging or support efforts.
+
+Coupled with those tools, it will enable you to create views into your
+background processes like the following:
+
+![splunk dashboard](http://i.imgur.com/0sZEw1L.png?1)
+
+## Lifecycle events
+
+Four lifecycle events will be published for each job entering a queue:
+
+1. enqueued
+2. dequeued
+3. perform_started
+4. perform_finished -or- failed
+
+Each will contain the following information (plus anything added to the
+metadata via an item preprocessor):
+
+* event_type: Either enqueued, dequeued, perform_started, perform_finished or
+  failed.
+* event_hash: Unique hash grouping all events associated with a single 
+  background job.
+* worker_class: The job class that contains the perform logic.
+* queue: The queue the job is placed into.
+* timestamp: The time the event occurs.
+* hostname: The hostname of the machine where the event originates from.
+* process: The process on the host machine where the event originates from.
+* args: The arguments passed to the perform method.
+
+dequeued events will also include time_in_queue, which is the amount of time
+the job spent in the queue. perform_finished and failed events will include
+time_to_perform, which is the time it took to perform the job after it was
+dequeued.  Failed events will include the exception class, the exception
+message and a backtrace. 
+
+## Event Publishers
+
+Event publishers are use to receive event data and publish them in some way.
+The following event publishers are currently provided:
+
+```ruby
+Resque::Plugins::Clues::StandardOutPublisher
+Resque::Plugins::Clues::LogPublisher
+Resque::Plugins::Clues::CompositePublisher
+```
+
+You can implement your own publishers as long as they implement event handling
+methods as follows:
+
+```ruby
+def publish(event_type, timestamp, queue, metadata, klass, *args)
+  ...
+end
+```
+
+Where event_type is enqueued, dequeued, perform_started, perform_finished and
+failed.
+
+## Event Marshallers
+
+An event marshallers is used to coerce event data into a format suitable for
+sending to an event publisher's destination.  This is a proc or lambda with the
+following call signature:  
+
+```ruby
+lambda do |event_type, timestamp, queue, metadata, worker_class, args|
+  # something that returns a string
+end
+```
+
+By default, clues will use an event_marshaller that will simply marshall this
+data to a JSON object in the following format:
+
+```
+{
+  "event_type":"dequeued",
+  "timestamp":"2013-06-04T20:59:58Z",
+  "queue":"test_queue",
+  "metadata": {
+    "event_hash":"0695f49c5e70fc18da91961113e1769a"
+    "hostname":"Lances-MacBook-Air.local",
+    "process":30731
+  },
+  "worker_class":"TestWorker",
+  "args":[1,2]
+}
+```
+
+## Item Preprocessors
+
+Immediately before Resque puts job data into a queue, the queue and the payload
+hash will be sent to a configurable item_preprocessor proc.  The payload hash
+will contain:
+
+* class:  The class of the job to perform.
+* args:  The args to pass to its perform method.
+* metadata:  The metadata hash that contains the event_hash identifier, the
+hostname and the process doing the enqueing.
+
+You can then inject whatever you need into the metadata hash and it will be 
+included in the published events.  At PeopleAdmin, we are using this to inject
+our customer identifiers so we can look at Resque analytics broken down on a
+per-customer basis.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+    gem 'resque-clues'
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install resque-clues
+
+## Usage
+
+Resque-clues requires configuration of the event publishers and an item
+preprocessor to be used, and this should occur before any use of Resque.  Here 
+is an example configuration:
+
+```ruby
+require 'resque'
+require 'resque-clues'
+
+publisher = Resque::Plugins::Clues::CompositePublisher.new
+publisher << Resque::Plugins::Clues::StandardOutPublisher.new
+publisher << Resque::Plugins::Clues::LogPublisher.new("/var/log/resque-clues.log")
+Resque::Plugins::Clues.event_publisher = publisher
+
+Resque::Plugins::Clues.item_preprocessor = proc do |queue, item| 
+  ...
+end
+```
+
+If used in a Rails application, this will need to be executed in an initalizer.
+After this, you should see events publishe as appropriate for your configured
+event publisher.
+
+## Contributing
+
+1. Fork it
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create new Pull Request

data/Rakefile

@@ -0,0 +1,6 @@
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+task :default => :spec

data/TODO

@@ -0,0 +1,4 @@
+* Modify the publish signature to accept a hash instead of all the params.
+* Add file-like streams that wrap TCP/IP and UDP sockets?  (not needed for our
+  case).  Would allow use of StreamPublisher to communicate over those
+  protocols

data/lib/resque-clues.rb

@@ -0,0 +1,78 @@
+require 'resque'
+require 'resque/plugins/clues/util'
+require 'resque/plugins/clues/queue_extension'
+require 'resque/plugins/clues/job_extension'
+require 'resque/plugins/clues/event_publisher'
+require 'resque/plugins/clues/version'
+
+module Resque
+  module Plugins
+    module Clues
+      class << self
+        attr_accessor :item_preprocessor
+        attr_accessor :event_marshaller
+
+        def configured?
+          !event_publisher.nil?
+        end
+
+        def enable!
+          # Patch resque to support event broadcasting.
+          Resque.send(:extend, Resque::Plugins::Clues::QueueExtension)
+          Resque::Job.send(:include, Resque::Plugins::Clues::JobExtension)
+          Resque.instance_exec do
+            alias :_base_push :push
+            alias :_base_pop :pop
+
+            def push(queue, item)
+              _clues_push(queue, item)
+            end
+
+            def pop(queue)
+              _clues_pop(queue)
+            end
+          end
+
+          Resque::Job.class_exec do
+            alias :_base_perform :perform
+            alias :_base_fail :fail
+
+            def perform
+              _clues_perform
+            end
+
+            def fail(exception)
+              _clues_fail(exception)
+            end
+          end
+        end
+      end
+    end
+  end
+end
+
+# Constructs a string event for the passed args.  Delegates to the
+# Resque::Plugins::Clues.event_marshaller proc/lambda to do this.  The
+# default version will simply marshall the args to a JSON object.
+#
+# event_type:: enqueued, dequeued, perform_started, perform_finished or
+# failed.
+# timestamp:: the time the event occurred.
+# queue:: the queue the job was in
+# metadata:: metadata for the job, such as host, process, etc...
+# worker_class:: the worker job class
+# args:: arguments passed to the perform_method.
+Resque::Plugins::Clues.event_marshaller =
+  lambda do |event_type, timestamp, queue, metadata, worker_class, args|
+    event = MultiJson.encode({
+      event_type: event_type,
+      timestamp: timestamp,
+      queue: queue,
+      metadata: metadata,
+      worker_class: worker_class,
+      args: args
+    })
+    "#{event}\n"
+  end
+
+Resque::Plugins::Clues.enable!

data/lib/resque/plugins/clues/event_publisher.rb

@@ -0,0 +1,91 @@
+require 'pp'
+require 'delegate'
+
+module Resque
+  module Plugins
+    module Clues
+      class << self
+        attr_accessor :event_publisher
+      end
+      EVENT_TYPES = %w[enqueued dequeued destroyed perform_started perform_finished failed]
+
+      # Event publisher that publishes events to a file-like stream in a JSON
+      # format.  Each message is punctuated with a terminus character, which
+      # defaults to newline ("\n")
+      class StreamPublisher
+        attr_reader :stream
+
+        # Creates a new StreamPublisher that writes to the passed stream,
+        # terminating each event with the terminus.
+        #
+        # stream:: The file-like stream to write events to.
+        def initialize(stream)
+          @stream = stream
+        end
+
+        # Publishes an event to the stream.
+        def publish(event_type, timestamp, queue, metadata, klass, *args)
+          event = Clues.event_marshaller.call(event_type, timestamp, queue, metadata, klass, args)
+          stream.write(event)
+        end
+      end
+
+      # Event publisher that publishes events to standard output in a json
+      # format.
+      class StandardOutPublisher < StreamPublisher
+        def initialize
+          super(STDOUT)
+        end
+      end
+
+      # Event publisher that publishes events to a log file using ruby's
+      # stdlib logger and an optional formatter..
+      class LogPublisher
+        attr_reader :logger
+
+        # Creates a new LogPublisher that writes events to a log file at the
+        # specified log_path, using an optional formatter.  The default format
+        # will simply be the event in a json format, one per line.
+        #
+        # log_path:: The path to the log file.
+        # formatter:: A lambda formatter for log messages.  Defaults to writing
+        # one event per line.  See
+        # http://www.ruby-doc.org/stdlib-1.9.3/libdoc/logger/rdoc/Logger/Formatter.html
+        def initialize(log_path, formatter=nil)
+          @logger = Logger.new(log_path)
+          @logger.formatter = formatter || lambda {|severity, time, program, msg| msg}
+        end
+
+        # Publishes an event to the log.
+        def publish(event_type, timestamp, queue, metadata, klass, *args)
+          logger.info(Clues.event_marshaller.call(
+            event_type, timestamp, queue, metadata, klass, args))
+        end
+      end
+
+      # A composite event publisher that groups several child publishers so
+      # that events received are delegated to each of the children for
+      # further processing.
+      class CompositePublisher < SimpleDelegator
+        def initialize
+          super([])
+        end
+
+        # Invokes publish on each child publisher for them to publish the event
+        # in their own way.
+        def publish(event_type, timestamp, queue, metadata, klass, *args)
+          each do |child|
+            child.publish(
+              event_type, timestamp, queue, metadata, klass, *args
+            ) rescue error(event_type, child)
+          end
+        end
+
+        private
+        def error(event_type, child)
+          p "Error processing #{event_type} in #{child}"
+        end
+      end
+    end
+  end
+end

data/lib/resque/plugins/clues/job_extension.rb

@@ -0,0 +1,65 @@
+require 'digest/md5'
+require 'time'
+
+module Resque
+  module Plugins
+    module Clues
+      # Module capable of redefining the Job#perform and Job#failed methods so
+      # that they publish perform_started, perform_finished and failed events.
+      module JobExtension
+        # Invoked when this module is included by a class.  Will redefine the
+        # perform and failed methods on that class.
+        #
+        # klass:: The klass including this module.
+        def self.included(klass)
+          define_perform(klass)
+          define_failed(klass)
+        end
+
+        private
+        # (Re)defines the perform method so that it will broadcast a
+        # perform_started event, invoke the original perform method, and
+        # then broadcast a perform_finished event if no exceptions are
+        # encountered.  The time to perform is calculated and included in
+        # the metadata of the perform_finished event.
+        #
+        # klass:: The class to define the perform method on.
+        def self.define_perform(klass) # :doc:
+          klass.send(:define_method, :_clues_perform) do
+            if Clues.configured? and payload['clues_metadata']
+              Clues.event_publisher.publish(:perform_started, Clues.now, queue, payload['clues_metadata'], payload['class'], *payload['args'])
+              @perform_started = Time.now
+              _base_perform.tap do 
+                payload['clues_metadata']['time_to_perform'] = Clues.time_delta_since(@perform_started)
+                Clues.event_publisher.publish(:perform_finished, Clues.now, queue, payload['clues_metadata'], payload['class'], *payload['args'])
+              end
+            else
+              _base_perform
+            end
+          end
+        end
+
+        # (Re)defines the failed method so that it will add time to perform,
+        # exception, error message and backtrace data to the job's payload
+        # metadata, then broadcast a failed event including that information.
+        #
+        # klass::  The class to define the failed method on.
+        #
+        def self.define_failed(klass) # :doc:
+          klass.send(:define_method, :_clues_fail) do |exception|
+            _base_fail(exception).tap do
+              metadata = payload['clues_metadata']
+              if Clues.configured? and metadata
+                metadata['time_to_perform'] = Clues.time_delta_since(@perform_started)
+                metadata['exception'] = exception.class
+                metadata['message'] = exception.message
+                metadata['backtrace'] = exception.backtrace
+                Clues.event_publisher.publish(:failed, Clues.now, queue, metadata, payload['class'], *payload['args'])
+              end
+            end
+          end
+        end
+      end
+    end
+  end
+end