railsbench 0.8.4 → 0.9.0

data/INSTALL

@@ -5,7 +5,7 @@ INSTALLATION
 STEP 1: installing railsbench and PATH modifcation
 
 If you obtained and installed railsbench as a gem, a script called
-railsbench is available in you Ruby bin directory. railsbench is a
+railsbench is available in your Ruby bin directory. railsbench is a
 simple driver for the railsbench scripts, which are located in the gem
 installation directory. Example:
 
@@ -16,10 +16,18 @@ path. The exact place can be found running
 
     railsbench base
 
-which prints the script directory path. After that change, individual
-commands can be run directly:
+which prints the script directory path. Another option is to run
 
-    railsbench perf_run 100 -bm=all
+    eval `railsbench path`
+
+Since gem packages don't support running postinstall scripts, you will
+need to make the scripts executable by running
+
+   sudo railsbench postinstall
+
+After performing these steps, individual commands can be run directly:
+
+    perf_run 100 -bm=all
 
 The environment variable RAILS_ROOT must be set and point to your apps
 base directory, in order for railsbench to work properly.
@@ -41,15 +49,16 @@ Manual installation:
 
 Automatic installation:
 
-  Run railsbench install. This will perform all of the above steps and
-  create a section in database.yml pointing to your development
-  database. In case you are worrying about possible results, you can
-  call railsbench install --dry-run first.
+  Run 'railsbench install'. This will perform all of the above steps
+  and create a configuration named 'benchmarking' in database.yml
+  pointing to your development database. In case you are worrying
+  about possible results, you can call railsbench install --dry-run
+  first.
 
 
 STEP 3: optional (but highly recommended)
 
 Patch the ruby garbage collector using rubygc18{45}.patch, depending
 on your ruby version. Recompile ruby and copy the ruby binary and
-shared libs to where it was previously installed. See GCPATCH for
+shared libs to where they were previously installed. See GCPATCH for
 details on how to influence the patched ruby gc.

data/Manifest.txt

@@ -1,38 +1,38 @@
-BUGS
-CHANGELOG
-INSTALL
-install.rb
-LICENSE
-Manifest.txt
-README
-Rakefile
-setup.rb
-ruby184gc.patch
-ruby185gc.patch
-bin/railsbench
-lib/railsbench/version.rb
-lib/railsbench/gc_info.rb
-lib/railsbench/perf_info.rb
-lib/railsbench/perf_utils.rb
-lib/railsbench/write_headers_only.rb
-lib/railsbench/railsbenchmark.rb
-lib/benchmark.rb
-script/perf_bench
-script/perf_comp
-script/perf_comp_gc
-script/perf_diff
-script/perf_diff_gc
-script/perf_html
-script/perf_loop
-script/perf_plot
-script/perf_plot_gc
-script/perf_prof
-script/perf_run
-script/perf_run_gc
-script/perf_tex
-script/perf_times
-script/perf_times_gc
-script/run_urls
-config/benchmarking.rb
-config/benchmarks.rb
-config/benchmarks.yml
+BUGS
+CHANGELOG
+INSTALL
+install.rb
+postinstall.rb
+LICENSE
+Manifest.txt
+README
+Rakefile
+setup.rb
+ruby184gc.patch
+ruby185gc.patch
+bin/railsbench
+lib/railsbench/version.rb
+lib/railsbench/gc_info.rb
+lib/railsbench/perf_info.rb
+lib/railsbench/perf_utils.rb
+lib/railsbench/write_headers_only.rb
+lib/railsbench/railsbenchmark.rb
+lib/benchmark.rb
+script/perf_bench
+script/perf_comp
+script/perf_comp_gc
+script/perf_diff
+script/perf_diff_gc
+script/perf_html
+script/perf_plot
+script/perf_plot_gc
+script/perf_prof
+script/perf_run
+script/perf_run_gc
+script/perf_tex
+script/perf_times
+script/perf_times_gc
+script/run_urls
+config/benchmarking.rb
+config/benchmarks.rb
+config/benchmarks.yml

data/README

@@ -1,13 +1,13 @@
-Railsbench is a small collection of ruby and shell scripts which make
-measuring raw performance of rails apps a snap. All tests are run from
-the command prompt making performance regression testing easy.
+Railsbench is a small collection of ruby scripts which make measuring
+raw performance of rails apps a snap. All tests are run from the
+command prompt, making performance regression testing easy.
 
 In addition, a patch for the ruby garbage collector is provided, which
 can be used to reduce the amount of time spent doing garbage
 collection, trading memory for speed, as usual (see file GCPATCH for
-details). In addition, applying the patch will enhance performance
-data obtained from the various scripts (and some won't run at all
-without the patch).
+details). Applying the patch will enhance performance data obtained
+from the various scripts (and some won't run at all without the
+patch).
 
 This software was written and conceived by Stefan Kaes. The author can
 be reached via email: <skaes@gmx.net>. Please send comments, bug

data/Rakefile

@@ -1,51 +1,52 @@
-require 'rubygems'
-require 'rake'
-require 'rake/clean'
-require 'rake/testtask'
-require 'rake/packagetask'
-require 'rake/gempackagetask'
-require 'rake/rdoctask'
-require 'rake/contrib/rubyforgepublisher'
-require 'fileutils'
-require 'hoe'
-include FileUtils
-require File.join(File.dirname(__FILE__), 'lib', 'railsbench', 'version')
-
-AUTHOR = "Stefan Kaes"  # can also be an array of Authors
-EMAIL = "skaes@gmx.net"
-DESCRIPTION = "rails benchmarking tools"
-GEM_NAME = "railsbench" # what ppl will type to install your gem
-RUBYFORGE_PROJECT = "railsbench" # The unix name for your project
-HOMEPATH = "http://#{RUBYFORGE_PROJECT}.rubyforge.org"
-RELEASE_TYPES = %w(gem tar zip) # can use: gem, tar, zip
-
-
-NAME = "railsbench"
-REV = nil # UNCOMMENT IF REQUIRED: File.read(".svn/entries")[/committed-rev="(d+)"/, 1] rescue nil
-VERS = (ENV['VERSION'] ||= (Railsbench::VERSION::STRING + (REV ? ".#{REV}" : "")))
-CLEAN.include ['**/.*.sw?', '*.gem', '.config']
-RDOC_OPTS = ['--quiet', '--title', "railsbench documentation",
-    "--opname", "index.html",
-    "--line-numbers",
-    "--main", "README",
-    "--inline-source"]
-
-# Generate all the Rake tasks
-# Run 'rake -T' to see list of generated tasks (from gem root directory)
-hoe = Hoe.new(GEM_NAME, VERS) do |p|
-  p.author = AUTHOR
-  p.description = DESCRIPTION
-  p.email = EMAIL
-  p.summary = DESCRIPTION
-  p.url = HOMEPATH
-  p.rubyforge_name = RUBYFORGE_PROJECT if RUBYFORGE_PROJECT
-  p.test_globs = ["test/**/*_test.rb"]
-  p.clean_globs = CLEAN  #An array of file patterns to delete on clean.
-  p.need_zip = true
-  p.spec_extras = {:has_rdoc => false}
-
-  # == Optional
-  #p.changes        - A description of the release's latest changes.
-  #p.extra_deps     - An array of rubygem dependencies.
-  #p.spec_extras    - A hash of extra values to set in the gemspec.
-end
+require 'rubygems'
+require 'rake'
+require 'rake/clean'
+require 'rake/testtask'
+require 'rake/packagetask'
+require 'rake/gempackagetask'
+require 'rake/rdoctask'
+require 'rake/contrib/rubyforgepublisher'
+require 'fileutils'
+require 'hoe'
+include FileUtils
+require File.join(File.dirname(__FILE__), 'lib', 'railsbench', 'version')
+
+AUTHOR = "Stefan Kaes"  # can also be an array of Authors
+EMAIL = "skaes@gmx.net"
+DESCRIPTION = "rails benchmarking tools"
+GEM_NAME = "railsbench" # what ppl will type to install your gem
+RUBYFORGE_PROJECT = "railsbench" # The unix name for your project
+HOMEPATH = "http://#{RUBYFORGE_PROJECT}.rubyforge.org"
+RELEASE_TYPES = %w(gem tar zip) # can use: gem, tar, zip
+
+
+NAME = "railsbench"
+REV = nil # UNCOMMENT IF REQUIRED: File.read(".svn/entries")[/committed-rev="(d+)"/, 1] rescue nil
+VERS = (ENV['VERSION'] ||= (Railsbench::VERSION::STRING + (REV ? ".#{REV}" : "")))
+CLEAN.include ['**/.*.sw?', '*.gem', '.config']
+RDOC_OPTS = ['--quiet', '--title', "railsbench documentation",
+    "--opname", "index.html",
+    "--line-numbers",
+    "--main", "README",
+    "--inline-source"]
+
+# Generate all the Rake tasks
+# Run 'rake -T' to see list of generated tasks (from gem root directory)
+hoe = Hoe.new(GEM_NAME, VERS) do |p|
+  p.author = AUTHOR
+  p.description = DESCRIPTION
+  p.email = EMAIL
+  p.summary = DESCRIPTION
+  p.url = HOMEPATH
+  p.rubyforge_name = RUBYFORGE_PROJECT if RUBYFORGE_PROJECT
+  p.test_globs = ["test/**/*_test.rb"]
+  p.clean_globs = CLEAN  #An array of file patterns to delete on clean.
+  p.need_zip = true
+  p.spec_extras = {:has_rdoc => false}
+  p.changes = `cat latest_changes`
+
+  # == Optional
+  #p.changes        - A description of the release's latest changes.
+  #p.extra_deps     - An array of rubygem dependencies.
+  #p.spec_extras    - A hash of extra values to set in the gemspec.
+end

data/bin/railsbench

@@ -1,50 +1,48 @@
-#!/usr/bin/env ruby
-
-CMDS = %w(
-base
-perf_comp
-perf_comp_gc
-perf_diff
-perf_diff_gc
-perf_html
-perf_plot
-perf_plot_gc
-perf_prof
-perf_run
-perf_run_gc
-perf_tex
-perf_times
-perf_times_gc
-install
-run_urls
-)
-
-cmd = ARGV.shift
-unless CMDS.include? cmd
-  $stderr.puts "railsbench: unkown command: #{cmd}"
-  $stderr.puts "use one of: #{CMDS.sort.join(', ')}"
-  exit 1
-end
-
-RAILSBENCH_BASE = File.expand_path(File.dirname(__FILE__) + '/..')
-
-def quoted_args
-  ARGV.map{|s| "'#{s}'" }.join(' ')
-end
-
-case cmd
-when 'base'
-  puts "railsbench is installed in: #{RAILSBENCH_BASE}"
-  puts "load path: #{$:.join("\n")}"
-  exit
-else
-  if cmd == 'install'
-    script = RAILSBENCH_BASE + '/install.rb'
-  else
-    script = RAILSBENCH_BASE + '/script/' + cmd
-  end
-  command = "sh -c \"#{script} #{quoted_args}\""
-  # puts command
-  exec(command)
-end
-
+#!/usr/bin/env ruby
+
+RAILSBENCH_CMDS = %w(
+base
+install
+path
+perf_comp
+perf_comp_gc
+perf_diff
+perf_diff_gc
+perf_html
+perf_plot
+perf_plot_gc
+perf_prof
+perf_run
+perf_run_gc
+perf_tex
+perf_times
+perf_times_gc
+postinstall
+run_urls
+).inject({}){ |h, cmd| h[cmd] = cmd; h[cmd.sub('perf_', '')] = cmd; h}
+
+cmd = ARGV.shift
+unless real_cmd = RAILSBENCH_CMDS[cmd]
+  $stderr.puts "railsbench: unknown command: #{cmd}"
+  $stderr.puts "use one of: #{RAILSBENCH_CMDS.keys.sort.join(', ')}"
+  exit 1
+end
+
+RAILSBENCH_BASE = File.expand_path(File.dirname(__FILE__) + '/..')
+
+case real_cmd
+when 'base'
+  puts "railsbench is installed in: #{RAILSBENCH_BASE}"
+  exit
+when 'path'
+  puts "PATH=#{RAILSBENCH_BASE}/script:$PATH"
+  exit
+when 'install', 'postinstall'
+  load "#{RAILSBENCH_BASE}/#{real_cmd}.rb"
+else
+  unless ENV['RAILS_ROOT']
+    $stderr.puts "railsbench: RAILS_ROOT not set"
+    exit 1
+  end
+  load "#{RAILSBENCH_BASE}/script/#{real_cmd}"
+end

data/config/benchmarking.rb

@@ -1,8 +1,21 @@
-Dependencies.mechanism                             = :require
+# Settings specified here will take precedence over those in config/environment.rb
 
-ActionController::Base.consider_all_requests_local = false
-ActionController::Base.perform_caching             = true
+# Code is not reloaded between requests
+config.cache_classes = true
 
-ActionView::Base.cache_template_loading = true
-ActionController::Base.view_controller_internals = false
+# Use a different logger for distributed setups
+# config.logger = SyslogLogger.new
 
+# Full error reports are disabled and caching is turned on
+config.action_controller.consider_all_requests_local = false
+config.action_controller.perform_caching             = true
+config.action_view.cache_template_loading            = true
+
+# we don't want to see controller internals
+config.action_controller.view_controller_internals   = false
+
+# Enable serving of images, stylesheets, and javascripts from an asset server
+# config.action_controller.asset_host                  = "http://assets.example.com"
+
+# Disable delivery errors if you bad email addresses should just be ignored
+# config.action_mailer.raise_delivery_errors = false

data/install.rb

@@ -44,7 +44,7 @@ __END__
 ### End: ***
 
 #  Copyright (C) 2005, 2006  Stefan Kaes
-# 
+#
 #  This program is free software; you can redistribute it and/or modify
 #  it under the terms of the GNU General Public License as published by
 #  the Free Software Foundation; either version 2 of the License, or
@@ -54,7 +54,7 @@ __END__
 #  but WITHOUT ANY WARRANTY; without even the implied warranty of
 #  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 #  GNU General Public License for more details.
-# 
+#
 #  You should have received a copy of the GNU General Public License
 #  along with this program; if not, write to the Free Software
 #  Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA

data/lib/railsbench/perf_utils.rb

@@ -63,3 +63,117 @@ def truncate(text, length = 32, truncate_string = "...")
     chars.length > length ? chars[0..(length - l)].join + truncate_string : text
   end
 end
+
+RAILSBENCH_BINDIR = File.expand_path(File.dirname(__FILE__) + "/../../script")
+
+def enable_gc_stats(file)
+  ENV['RUBY_GC_STATS'] = "1"
+  ENV['RUBY_GC_DATA_FILE'] = file
+end
+
+def disable_gc_stats
+  ENV.delete 'RUBY_GC_STATS'
+  ENV.delete 'RUBY_GC_DATA_FILE'
+end
+
+def unset_gc_variables
+  %w(RUBY_HEAP_MIN_SLOTS RUBY_GC_MALLOC_LIMIT RUBY_HEAP_FREE_MIN).each{|v| ENV.delete v}
+end
+
+def load_gc_variables(gc_spec)
+  File.open_or_die("#{ENV['RAILS_ROOT']}/config/#{gc_spec}.gc").each_line do |line|
+    ENV[$1] = $2 if line =~ /^(.*)=(.*)$/
+  end
+end
+
+def set_gc_variables(argv)
+  gc_spec = nil
+  argv.each{|arg| gc_spec=$1 if arg =~ /-gc=([^ ]*)/}
+
+  if gc_spec
+    load_gc_variables(gc_spec)
+  else
+    unset_gc_variables
+  end
+end
+
+def benchmark_file_name(benchmark, config_name, prefix=nil, suffix=nil)
+  perf_data_dir = (ENV['RAILS_PERF_DATA'] ||= ENV['HOME'])
+  date = Time.now.strftime '%m-%d'
+  suffix = ".#{suffix}" if suffix
+  ENV['RAILS_BENCHMARK_FILE'] =
+    if config_name
+      "#{perf_data_dir}/#{date}.#{benchmark}.#{config_name}#{suffix}.txt"
+    else
+      "#{perf_data_dir}/perf_run#{prefix}.#{benchmark}#{suffix}.txt"
+    end
+end
+
+def quote_arguments(argv)
+  argv.map{|a| a.include?(' ') ? "'#{a}'" : a.to_s}.join(' ')
+end
+
+def perf_run(script, iterations, options, raw_data_file)
+  perf_runs = (ENV['RAILS_PERF_RUNS'] ||= "3").to_i
+
+  disable_gc_stats
+  set_gc_variables([iterations, options])
+
+  perf_options = "#{iterations} #{options}"
+  null = (RUBY_PLATFORM =~ /win32/i) ? 'nul' : '/dev/null'
+
+  perf_cmd = "ruby #{RAILSBENCH_BINDIR}/perf_bench #{perf_options}"
+  print_cmd = "ruby #{RAILSBENCH_BINDIR}/perf_times #{raw_data_file}"
+
+  puts "benchmarking #{perf_runs} runs with options #{perf_options}"
+
+  File.open(raw_data_file, "w"){ |f| f.puts perf_cmd }
+  perf_runs.times do
+    system("#{perf_cmd} >#{null}") || die("#{script}: #{perf_cmd} returned #{$?}")
+  end
+  File.open(raw_data_file, "a" ){|f| f.puts }
+
+  unset_gc_variables
+  system(print_cmd) || die("#{script}: #{print_cmd} returned #{$?}")
+end
+
+def perf_run_gc(script, iterations, options, raw_data_file)
+  warmup = "-warmup"
+  warmup = "" if options =~ /-warmup/
+
+  enable_gc_stats(raw_data_file)
+  set_gc_variables([options])
+
+  perf_options = "#{iterations} #{warmup} #{options}"
+  null = (RUBY_PLATFORM =~ /win32/) ? 'nul' : '/dev/null'
+
+  perf_cmd = "ruby #{RAILSBENCH_BINDIR}/run_urls #{perf_options} >#{null}"
+  print_cmd = "ruby #{RAILSBENCH_BINDIR}/perf_times_gc #{raw_data_file}"
+
+  puts "benchmarking GC performance with options #{perf_options}"
+  puts
+
+  system(perf_cmd) || die("#{script}: #{perf_cmd} returned #{$?}")
+
+  disable_gc_stats
+  unset_gc_variables
+  system(print_cmd) || die("#{script}: #{print_cmd} returned #{$?}")
+end
+
+__END__
+
+#  Copyright (C) 2005, 2006  Stefan Kaes
+#
+#  This program is free software; you can redistribute it and/or modify
+#  it under the terms of the GNU General Public License as published by
+#  the Free Software Foundation; either version 2 of the License, or
+#  (at your option) any later version.
+#
+#  This program is distributed in the hope that it will be useful,
+#  but WITHOUT ANY WARRANTY; without even the implied warranty of
+#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#  GNU General Public License for more details.
+#
+#  You should have received a copy of the GNU General Public License
+#  along with this program; if not, write to the Free Software
+#  Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA