wvanbergen-request-log-analyzer 0.2.2 → 0.3.0

data/{README → README.textile}

@@ -1,96 +1,93 @@
-Request log analyzer
---------------------------------
+h1. Request log analyzer
 
 This is a simple command line tool to analyze request log files of both Rails and
 Merb. Its purpose is to find what actions are best candidates for optimization.
 
-This tool will parse all requests in the logfile and aggregate the
-information. Once it is finished parsing the log file, it will show the
-requests that take op most server time. Different metrics are used (cumulative
-time, average time, blockers, DB time, etc)
+* Analyzes Rails logs (all versions) and Merb logs
+* Can combine multiple files (handy if you are using logrotate)
+* Uses several metrics (cumulative time, average time, blockers, DB time, etc)
+* Low memory footprint (server-safe)
+* MIT licensed
+* Fast
 
+h2. Installation
 
-Installation
---------------------------------
-gem sources -a http://gems.github.com
-sudo gem install wvanbergen-request-log-analyzer
+@sudo gem install wvanbergen-request-log-analyzer --source http://gems.github.com@
 
-Usage
---------------------------------
+h2. Usage
 
+<pre>
 Usage: request-log-analyzer [FILE] [OPTION]
 Analyze the given log FILE with the given OPTION
-Example: request-log-analyzer mongrel.log
+Example: request-log-analyzer mongrel.log -z
 
-  --fast, -t:                 Only use completed requests
+  --fast, -f:                 Only use completed requests
   --guess-database-time, -g:  Guesses the database duration of requests if they are not in the log
   --output, -o:               Comma-separated list of reports to show    
   --amount, -c:               Displays the top <amount> elements in the reports
   --colorize, -z:             Fancy bash coloring
+  --install rails, -i rails:  Install Rails task rake log:analyze
+</pre>
 
 
-
-Example
---------------------------------
+h2. Example result
 
 Note that this example was shortened for your viewing pleasure.
-$ request-log-analyzer /var/log/my_app.log
+@$ request-log-analyzer /var/log/my_app.log -o 5@
 
+<pre>
 Request log analyzer, by Willem van Bergen and  Bart ten Brinke
 
+Processing started, failed, completed log lines from /var/log/my_app.log...
+
 Processing all log lines...
 ========================================================================
-Successfully analyzed 58908 requests from log file
-
 Timestamp first request: 2008-07-13T06:25:58+00:00
 Timestamp last request:  2008-07-20T06:18:53+00:00
 Total time analyzed: 7 days
-Methods: DELETE (1%), GET (50%), POST (22%), PUT (25%).
 
-Top 10 most requested actions
+Total requests analyzed 58908 requests from log file
+Methods: GET (50.6%), POST (22.8%), PUT (25.4%), DELETE (1.1%), unknown (0.0%).
+
+Top 5 most requested actions
 ========================================================================
 /overview/:date/                                  : 19359 requests
 /overview/day/:date/                              : 6365 requests
 /overview/:date/set/                              : 5589 requests
 /overview/                                        : 3985 requests
 /clients/:id/                                     : 1976 requests
-........
 
-Top 10 actions by time - cumulative
+Top 5 actions by time - cumulative
 ========================================================================
 /overview/:date/                                  :   9044.582s [19359 requests]
 /overview/                                        :   8478.767s [3985 requests]
 /overview/:date/set/                              :   3309.041s [5589 requests]
 /clients/:id/products/:id/                        :   1479.911s [924 requests]
 /clients/:id/                                     :    750.080s [1976 requests]
-........
 
-Top 10 actions by time - per request mean
+Top 5 actions by time - per request mean
 ========================================================================
 /overview/                                        :      2.128s [3985 requests]
 /clients/:id/products/:id/                        :      1.602s [924 requests]
 /overview/:date/set/                              :      0.592s [5589 requests]
 /overview/:date/                                  :      0.467s [19359 requests]
 /clients/:id/                                     :      0.380s [1976 requests]
-........
 
-Top 10 worst DB offenders - cumulative time
+Top 5 worst DB offenders - cumulative time
 ========================================================================
 /overview/:date/                                  :   8773.993s [19359 requests]
 /overview/                                        :   8394.754s [3985 requests]
 /overview/:date/set/                              :   3307.928s [5589 requests]
 /clients/:id/products/:id/                        :   1425.220s [924 requests]
 /clients/:id/                                     :    535.229s [1976 requests]
-........
 
-Top 10 worst DB offenders - mean time
+Top 5 worst DB offenders - mean time
 ========================================================================
 /overview/:id/:id/:id/print/                      :      6.994s [448 requests]
 /overview/                                        :      2.128s [3985 requests]
 /clients/:id/products/:id/                        :      1.602s [924 requests]
 /overview/:date/set/                              :      0.592s [5589 requests]
 /overview/:date/                                  :      0.467s [19359 requests]
-........
 
 Mongrel process blockers (> 1.0 seconds)
 ========================================================================
@@ -99,7 +96,6 @@ Mongrel process blockers (> 1.0 seconds)
 /overview/:date/set/                              :   1149.235s [803 requests]
 /overview/:id/:id/:id/print/new/                  :    613.693s [341 requests]
 /clients/:id/products/:id/                        :   1370.693s [313 requests]
-........
 
 Requests graph - per hour
 ========================================================================
@@ -120,13 +116,10 @@ Requests graph - per hour
 
 Errors
 ========================================================================
-ArgumentError: [237 requests]
- -> invalid date
 StaleObjectError: [28 requests]
  -> Attempted to update a stale object
-RuntimeError: [3 requests]
- -> Cannot destroy rule before it was created
 StatementError: [2 requests]
  -> Mysql::Error: Deadlock found when trying to get lock; try restarting transaction
 NoMethodError: [1 requests]
  -> undefined method `code' for nil:NilClass
+</pre>

data/Rakefile

@@ -1,72 +1,5 @@
-require 'rubygems'
-
-load 'test/tasks.rake'
+Dir[File.dirname(__FILE__) + "/tasks/*.rake"].each { |file| load(file) }
  
-desc 'Default: run unit tests for request-log-analyzer.'
-task :default => :test
-
-
-namespace :gem do
+desc 'Default: run RSpec for request-log-analyzer.'
+task :default => :spec
 
-  desc "Sets the version and date of the gem. Requires the VERSION environment variable."
-  task :version => [:manifest] do
-    
-    require 'date'
-    
-    new_version = ENV['VERSION']
-    raise "VERSION is required" unless /\d+(\.\d+)*/ =~ new_version
-    
-    spec_file = Dir['*.gemspec'].first
-    
-    spec = File.read(spec_file)
-    spec.gsub!(/^(\s*s\.version\s*=\s*)('|")(.+)('|")(\s*)$/) { "#{$1}'#{new_version}'#{$5}" }
-    spec.gsub!(/^(\s*s\.date\s*=\s*)('|")(.+)('|")(\s*)$/) { "#{$1}'#{Date.today.strftime('%Y-%m-%d')}'#{$5}" }    
-    File.open(spec_file, 'w') { |f| f << spec }
-  end
-  
-  desc "Creates a git tag for the provided VERSION"
-  task :tag => [:version] do
-    
-    new_version = ENV['VERSION']
-    raise "VERSION is required" unless /\d+(\.\d+)*/ =~ new_version
-        
-    sh "git add request-log-analyzer.gemspec"
-    sh "git commit -m \"Set gem version to #{new_version}\""
-    sh "git push origin"    
-    sh "git tag -a \"request-log-analyzer-#{new_version}\" -m \"Tagged version #{new_version}\""
-    sh "git push --tags"
-  end
-
-  desc "Builds a ruby gem for request-log-analyzer"
-  task :build => [:manifest] do
-    system %[gem build request-log-analyzer.gemspec]
-  end
-
-  desc %{Update ".manifest" with the latest list of project filenames. Respect\
-  .gitignore by excluding everything that git ignores. Update `files` and\
-  `test_files` arrays in "*.gemspec" file if it's present.}
-  task :manifest do
-    list = Dir['**/*'].sort
-    spec_file = Dir['*.gemspec'].first
-    list -= [spec_file] if spec_file
-  
-    File.read('.gitignore').each_line do |glob|
-      glob = glob.chomp.sub(/^\//, '')
-      list -= Dir[glob]
-      list -= Dir["#{glob}/**/*"] if File.directory?(glob) and !File.symlink?(glob)
-      puts "excluding #{glob}"
-    end
- 
-    if spec_file
-      spec = File.read spec_file
-      spec.gsub! /^(\s* s.(test_)?files \s* = \s* )( \[ [^\]]* \] | %w\( [^)]* \) )/mx do
-        assignment = $1
-        bunch = $2 ? list.grep(/^test.*_test\.rb$/) : list
-        '%s%%w(%s)' % [assignment, bunch.join(' ')]
-      end
-      
-      File.open(spec_file,   'w') {|f| f << spec }
-    end
-    File.open('.manifest', 'w') {|f| f << list.join("\n") }
-  end
-end

data/TODO

@@ -2,17 +2,52 @@ TODO items for Rails-log-analyzer
 =================================
 Contact willem AT vanbergen DOT org if you want to help out with the development.
 
-Database:
+General:
+- Add more tests / specs
+
+Datamining:
 - Add query functionality for the resulting database file (interactive reports?)
-- Link request processing line to request completed line
+- Link request processing line to request completed line (VirtualMongrel?)
+- Fix the database inserter and make it more robust for future changes
 
 Rails integration:
-- Create script that calls request-log-analyzer 
 - Optionally use local or specific routes.rb file to parse URLs
-- Add rake tasks to Rails application when included
 
-General:
-- Add useful rake tasks
-- Add more tests
-- Fix multiple file handling
+Other:
 - World domination
+
+
+
+
+Datamining should look something like this:
+
+> request-log-analyzer myapp.log --interactive
+Request log analyzer builds a new database.
+Columns come from the log_parser as the LOG_LINES store all the keys that they can detect.
+Also add some extra columns like hashed_request_url etc.
+
+Request log analyzer then parses the logfile for its individual requests using something like the 
+virtual mongrel (we need a new name for this, database_summarizer agregator? sheepdog?) combined with our
+default log parser.
+
+When this is done the user enters an interactive mode (like irb).
+> Filters: None
+> Total requests in database: 53232
+> $
+
+The user can add filters like this:
+> $ FILTER SQL ["date > ?", Date.today-1]
+
+The user will then see this:
+> Filters:
+>  1. ["date > ?", Date.today-1]
+> Total requests: 2120
+> $
+
+At any point the user can destroy filters, show the raw requests or show reports
+> $ REPORT ALL
+
+The request remaining after the filter chain will then be processed through the summarizer and then trough the
+output templates, generating reports specificly for the selected dataset.
+Partials should also be possible
+> $ REPORT TIMESPAN

data/bin/request-log-analyzer

@@ -1,61 +1,23 @@
 #!/usr/bin/ruby
+require File.dirname(__FILE__) + '/../lib/request_log_analyzer'
 require File.dirname(__FILE__) + '/../lib/command_line/arguments'
-require File.dirname(__FILE__) + '/../lib/base/log_parser'
-require File.dirname(__FILE__) + '/../lib/base/summarizer'
-require File.dirname(__FILE__) + '/../lib/rails_analyzer/log_parser'
-require File.dirname(__FILE__) + '/../lib/rails_analyzer/summarizer'
-require File.dirname(__FILE__) + '/../lib/merb_analyzer/log_parser'
-require File.dirname(__FILE__) + '/../lib/merb_analyzer/summarizer'
-require File.dirname(__FILE__) + '/../lib/bashcolorizer'
-require File.dirname(__FILE__) + '/../lib/ruby-progressbar/progressbar.rb'
 
 puts "Request log analyzer, by Willem van Bergen and Bart ten Brinke\n\n"
 
-# Substitutes variable elements in a url (like the id field) with a fixed string (like ":id")
-# This is used to aggregate simular requests. 
-# <tt>request</tt> The request to evaluate.
-# Returns uniformed url string.
-# Raises on mailformed request.
-def request_hasher(request)
-  if request[:url]
-    url = request[:url].downcase.split(/^http[s]?:\/\/[A-z0-9\.-]+/).last.split('?').first # only the relevant URL part
-    url << '/' if url[-1] != '/'[0] && url.length > 1 # pad a trailing slash for consistency
-
-    url.gsub!(/\/\d+-\d+-\d+(\/|$)/, '/:date') # Combine all (year-month-day) queries
-    url.gsub!(/\/\d+-\d+(\/|$)/, '/:month') # Combine all date (year-month) queries
-    url.gsub!(/\/\d+[\w-]*/, '/:id') # replace identifiers in URLs
-        
-    return url
-  elsif request[:controller] && request[:action]
-    return "#{request[:controller]}##{request[:action]}"
-  else
-    raise 'Cannot hash this request! ' + request.inspect
-  end
-end
-
-# Print results using a ASCII table.
-# <tt>summarizer</tt> The summarizer containg information to draw the table.
-# <tt>field</tt> The field containing the data to be printed
-# <tt>amount</tt> The length of the table (defaults to 20)
-def print_table(summarizer, field, amount = 20)
-  summarizer.sort_actions_by(field).reverse[0, amount.to_i].each do |a|
-    # As we show count by default, show totaltime if we sort by count
-    field = :total_time if field == :count
-    
-    puts "%-50s: %10.03fs [#{green("%d requests")}]" % [a[0], a[1][field], a[1][:count]]
-  end
-end
-
 # Parse the arguments given via commandline
 begin
-  $arguments = CommandLine::Arguments.parse do |command_line|
-    command_line.switch(:guess_database_time, :g)
-    command_line.switch(:fast, :f)
-    command_line.switch(:colorize, :z)
-    command_line.switch(:merb, :m)
-    command_line.flag(:output, :alias => :o)
-    command_line.flag(:amount, :alias => :c)
-    command_line.required_files = 1
+  arguments = CommandLine::Arguments.parse do |command_line|
+    
+    #command_line.flag(:install, :alias => :i) # command_line.command(:install)
+    
+    command_line.flag(:format, :default => 'rails')
+    command_line.flag(:aggregator, :alias => :a, :multiple => true)
+    command_line.flag(:database, :alias => :d)
+        
+    command_line.switch(:combined_requests, :c)
+    command_line.switch(:colorize, :z) 
+    #command_line.switch(:estimate_database_time, :e) 
+    #command_line.switch(:fast, :f) #
   end
   
 rescue CommandLine::Error => e
@@ -65,51 +27,22 @@ rescue CommandLine::Error => e
   exit(0) 
 end
 
-
-if $arguments[:merb]
-  $summarizer = MerbAnalyzer::Summarizer.new(:calculate_database => $arguments[:guess_database_time])
-else
-  $summarizer = RailsAnalyzer::Summarizer.new(:calculate_database => $arguments[:guess_database_time])
-end
-
-if $arguments[:fast]
-  line_types = [:completed]
-elsif $arguments[:merb]
-  line_types = MerbAnalyzer::LogParser::LOG_LINES.keys
-else
-  line_types = RailsAnalyzer::LogParser::LOG_LINES.keys
-end
-
-# Walk through al the files given via the arguments.
-$arguments.files.each do |log_file|
-  puts "Processing #{line_types.join(', ')} log lines from #{log_file}..."
-
-  if $arguments[:merb]
-    parser = MerbAnalyzer::LogParser.new(log_file)
-  else
-    parser = RailsAnalyzer::LogParser.new(log_file)
-  end
-  
-  # add progress bar 
-  unless $arguments[:fast]
-    pbar = ProgressBar.new(green(log_file), File.size(log_file))
-    parser.progress { |pos, total| (pos == :finished) ? pbar.finish : pbar.set(pos) }
-  end
-  
-  parser.each(*line_types) do |request|
-    $summarizer.group(request)  { |r| request_hasher(r) }
-  end
-end
-
-# Select the reports to output and generate them.
-output_reports = $arguments[:output].split(',') rescue [:timespan, :most_requested, :total_time, :mean_time, :total_db_time, :mean_db_time, :mean_rendering_time, :blockers, :hourly_spread, :errors] 
-
-output_reports.each do |report|
-  report_location = "#{File.dirname(__FILE__)}/../output/#{report}.rb"
-
-  if File.exist?(report_location)
-    load report_location
-  else
-    puts "\nERROR: Output report #{report} not found!"
-  end
-end
+# if arguments[:install]
+#   if arguments[:install] == 'rails'
+#     require 'ftools'
+#     if File.directory?('./lib/tasks/')
+#       File.copy(File.dirname(__FILE__) + '/../tasks/request_log_analyzer.rake', './lib/tasks/request_log_analyze.rake')
+#       puts "Installed rake tasks."
+#       puts "To use, run: rake log:analyze"
+#     else
+#       puts "Cannot find /lib/tasks folder. Are you in your Rails directory?"
+#       puts "Installation aborted."
+#     end
+#   else
+#     raise "Cannot perform this install type!"
+#   end
+#   exit(0)
+# end
+
+# Run the request_log_analyzer!
+request_log_analyzer = RequestLogAnalyzer::Controller.build(arguments).run!

data/lib/base/summarizer.rb

@@ -67,5 +67,19 @@ module Base
       errors = min_count.nil? ? @errors.to_a : @errors.delete_if { |k, v| v[:count] < min_count}.to_a
       errors.sort { |a, b| a[1][field.to_sym] <=> b[1][field.to_sym] } 
     end
+    
+    # Compare date strings fast
+    # Assumes date formats: "2008-07-14 12:11:20" or alike.
+    # <tt>first_date</tt> The first date string
+    # <tt>second_date</tt> The second date string
+    # Returns -1 if first_date < second_date, nil if equal
+    # and 1 if first_date > second_date (<=>)
+    def hamburger_compare_string_dates first_date, second_date
+      return nil if first_date.nil? || second_date.nil?
+      
+      first_date.gsub(/[^0-9|\s]/,'').to_i \
+        <=> \
+      second_date.gsub(/[^0-9|\s]/,'').to_i
+    end
   end
 end 

data/lib/bashcolorizer.rb

@@ -2,7 +2,7 @@
 # <tt>text</tt> The text to colorize.
 # <tt>color_code</tt> The color code string to set
 # <tt>color</tt> Does not color if false. Defaults to ($arguments && $arguments[:colorize])
-def colorize(text, color_code, color = $arguments && $arguments[:colorize])
+def colorize(text, color_code, color = $colorize)
   color ? "#{color_code}#{text}\e[0m" : text
 end
 

data/lib/command_line/arguments.rb

@@ -46,7 +46,7 @@ module CommandLine
     # <tt>flag</tt> A flag symbol like :fast
     # Options
     # * <tt>:expects</tt> Expects a value after the flag
-    def flag(flag, options)
+    def flag(flag, options = {})
       options[:expects] = String unless options.has_key?(:expects)
       argument = Flag.new(flag, options)
       @flag_definitions[argument.to_argument]  = argument
@@ -85,7 +85,14 @@ module CommandLine
             if flag.expects_argument?
               
               if @arguments.length > (i + 1) && @arguments[i + 1]
-                @flags[flag.name] = @arguments[i + 1]
+                
+                if flag.multiple?
+                  @flags[flag.name] ||= []
+                  @flags[flag.name] << @arguments[i + 1]
+                else
+                  @flags[flag.name] = @arguments[i + 1]
+                end
+                                
                 i += 1
               else
                 raise CommandLine::FlagExpectsArgument.new(arg)
@@ -115,6 +122,12 @@ module CommandLine
     # Check if the parsed arguments meet their requirements.
     # Raises CommandLineexception on error.
     def check_parsed_arguments!
+      
+      @flag_definitions.each do |flag, definition| 
+        @flags[definition.name] ||= [] if definition.multiple? && !definition.default?
+        @flags[definition.name] ||= definition.default if definition.default?
+      end
+
       if @begins_with_command && @command.nil? 
         raise CommandLine::CommandMissing.new
       end

data/lib/command_line/flag.rb

@@ -6,6 +6,8 @@ module CommandLine
     attr_reader :name
     attr_reader :alias
     attr_reader :argument
+    attr_reader :default
+    attr_reader :multiple
     
     # Initialize new Flag
     # <tt>name</tt> The name of the flag
@@ -15,6 +17,8 @@ module CommandLine
       @alias    = definition[:alias].to_sym if definition[:alias]
       @required = definition.has_key?(:required) && definition[:required] == true
       @argument = definition[:expects] if definition[:expects]
+      @multiple = definition[:multiple] || false
+      @default  = definition[:default] if definition[:default]
     end
     
     # Argument representation of the flag (--fast)
@@ -37,6 +41,14 @@ module CommandLine
       !@required
     end
     
+    def multiple?
+      @multiple
+    end
+    
+    def default?
+      !@default.nil?
+    end
+    
     # Check if flag is required
     def required?
       @required

data/lib/rails_analyzer/summarizer.rb

@@ -22,16 +22,24 @@ module RailsAnalyzer
       case request[:type]   
         when :started 
           if request[:timestamp]
-            @first_request_at ||= request[:timestamp] # assume time-based order of file
-            @last_request_at  = request[:timestamp]   # assume time-based order of file
-            @request_time_graph[request[:timestamp][11..12].to_i] +=1 
+            if @first_request_at.nil? || hamburger_compare_string_dates(request[:timestamp], @first_request_at) == -1
+              @first_request_at = request[:timestamp]
+            end
+
+            if @last_request_at.nil? || hamburger_compare_string_dates(request[:timestamp], @last_request_at) == 1
+              @last_request_at = request[:timestamp]
+            end
+
+            @request_time_graph[request[:timestamp][11..12].to_i] +=1
           end
+
           if request[:method]
             @methods[request[:method].to_sym] ||= 0
             @methods[request[:method].to_sym] += 1
           else
             @methods[:unknown] += 1
           end
+
         when :completed
           @request_count += 1 
           hash = block_given? ? yield(request) : request.hash
@@ -42,7 +50,7 @@ module RailsAnalyzer
           @actions[hash][:count] += 1
           @actions[hash][:total_time] += request[:duration]
           @actions[hash][:total_db_time] += request[:db] if request[:db]
-          @actions[hash][:total_db_time] += request[:duration] - request[:rendering] if @calculate_database
+          @actions[hash][:total_db_time] += request[:duration] - request[:rendering] if @calculate_database && request[:duration] && request[:rendering]
 
           @actions[hash][:total_rendering_time] += request[:rendering] if request[:rendering]
       

data/lib/rails_analyzer/virtual_mongrel.rb

@@ -0,0 +1,91 @@
+# Can calculate request counts, duratations, mean times etc. of all the requests given.
+class VirtualMongrel
+  STATUS = [:started, :completed]
+
+  attr_reader :status
+  attr_reader :start_line
+  attr_reader :start_time
+  attr_reader :die_line
+  attr_reader :die_time
+  attr_reader :calculate_database
+  attr_reader :running_mongrels
+  
+  attr_reader :data_hash
+
+  def initialize(options = {})
+    @status       = :started
+
+    @start_line   = options[:start_line] || 0
+    @die_line     = options[:die_line] || @start_line + 10
+
+    @start_time   = options[:start_time] || 0
+    @die_time     = options[:die_time] || @start_time + 10
+    
+    @data_hash    = {}
+    @calculate_database = false
+    @running_mongrels = options[:running_mongrels] || 1
+  end
+  
+  def update_running_mongrels(number)
+    @running_mongrels = number if number > @running_mongrels
+  end
+    
+  
+  def group(request, &block)
+    case request[:type]
+      when :started 
+        data_hash.store(:timestamp,   request[:timestamp])
+        data_hash.store(:method,      request[:method])
+        @status       = :started
+
+      when :completed
+        data_hash.store(:url,             request[:url])
+        data_hash.store(:hashed_request,  request_hasher(request))
+        data_hash.store(:rendering,       request[:rendering])
+        data_hash.store(:duration,        request[:duration])
+        data_hash.store(:db_time,         request[:db])
+    
+        if @calculate_database && request[:duration] && request[:rendering]
+          data_hash.store(:db_time, request[:duration] - request[:request])
+        end
+    
+        @status = :completed
+
+      when :failed
+        data_hash.store(:error,               request[:error])
+        data_hash.store(:exception_string,    request[:exception_string])
+        @status = :completed
+
+    end
+  end
+
+  # Substitutes variable elements in a url (like the id field) with a fixed string (like ":id")
+  # This is used to aggregate simular requests. 
+  # <tt>request</tt> The request to evaluate.
+  # Returns uniformed url string.
+  # Raises on mailformed request.
+  def request_hasher(request)
+    if request[:url]
+      url = request[:url].downcase.split(/^http[s]?:\/\/[A-z0-9\.-]+/).last.split('?').first # only the relevant URL part
+      url << '/' if url[-1] != '/'[0] && url.length > 1 # pad a trailing slash for consistency
+
+      url.gsub!(/\/\d+-\d+-\d+(\/|$)/, '/:date') # Combine all (year-month-day) queries
+      url.gsub!(/\/\d+-\d+(\/|$)/, '/:month') # Combine all date (year-month) queries
+      url.gsub!(/\/\d+[\w-]*/, '/:id') # replace identifiers in URLs
+
+      return url
+    elsif request[:controller] && request[:action]
+      return "#{request[:controller]}##{request[:action]}"
+    else
+      raise 'Cannot hash this request! ' + request.inspect
+    end
+  end  
+ 
+  # Store this mongrel in the database
+  def save
+    puts 'Saving mongrel!'
+    puts "Number of other running mongrels (certainty) #{running_mongrels}"
+    puts data_hash.to_s
+  end
+
+end