railsbench 0.8.4

data/BUGS

@@ -0,0 +1,3 @@
+- POST isn't supported (yet)
+- documentation could be improved
+- hoe's rake install doesn't work (which is actually hoe's problem)

data/INSTALL

@@ -0,0 +1,55 @@
+============
+INSTALLATION
+============
+
+STEP 1: installing railsbench and PATH modifcation
+
+If you obtained and installed railsbench as a gem, a script called
+railsbench is available in you Ruby bin directory. railsbench is a
+simple driver for the railsbench scripts, which are located in the gem
+installation directory. Example:
+
+    railsbench perf_run 100 -bm=all
+
+Alternatively, add the railsbench script directory to your search
+path. The exact place can be found running
+
+    railsbench base
+
+which prints the script directory path. After that change, individual
+commands can be run directly:
+
+    railsbench perf_run 100 -bm=all
+
+The environment variable RAILS_ROOT must be set and point to your apps
+base directory, in order for railsbench to work properly.
+
+If you obtained railsbench as a svn checkout, add the railsbench
+script directory to your search path.
+
+
+STEP 2: prepare your application for benchmarking
+
+Manual installation:
+
+  Copy benchmarks.rb and benchmarks.yml into $RAILS_ROOT/config and
+  edit them to suit your needs. The provided default benchmark.yml
+  will run the root URL of your app.
+
+  Copy benchmarking.rb to $RAILS_ROOT/config/environments and add a
+  corresponding section to database.yml.
+
+Automatic installation:
+
+  Run railsbench install. This will perform all of the above steps and
+  create a section in database.yml pointing to your development
+  database. In case you are worrying about possible results, you can
+  call railsbench install --dry-run first.
+
+
+STEP 3: optional (but highly recommended)
+
+Patch the ruby garbage collector using rubygc18{45}.patch, depending
+on your ruby version. Recompile ruby and copy the ruby binary and
+shared libs to where it was previously installed. See GCPATCH for
+details on how to influence the patched ruby gc.

data/LICENSE

@@ -0,0 +1,222 @@
+		    GNU GENERAL PUBLIC LICENSE
+   TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
+
+  0. This License applies to any program or other work which contains
+a notice placed by the copyright holder saying it may be distributed
+under the terms of this General Public License.  The "Program", below,
+refers to any such program or work, and a "work based on the Program"
+means either the Program or any derivative work under copyright law:
+that is to say, a work containing the Program or a portion of it,
+either verbatim or with modifications and/or translated into another
+language.  (Hereinafter, translation is included without limitation in
+the term "modification".)  Each licensee is addressed as "you".
+
+Activities other than copying, distribution and modification are not
+covered by this License; they are outside its scope.  The act of
+running the Program is not restricted, and the output from the Program
+is covered only if its contents constitute a work based on the
+Program (independent of having been made by running the Program).
+Whether that is true depends on what the Program does.
+
+  1. You may copy and distribute verbatim copies of the Program's
+source code as you receive it, in any medium, provided that you
+conspicuously and appropriately publish on each copy an appropriate
+copyright notice and disclaimer of warranty; keep intact all the
+notices that refer to this License and to the absence of any warranty;
+and give any other recipients of the Program a copy of this License
+along with the Program.
+
+You may charge a fee for the physical act of transferring a copy, and
+you may at your option offer warranty protection in exchange for a fee.
+
+  2. You may modify your copy or copies of the Program or any portion
+of it, thus forming a work based on the Program, and copy and
+distribute such modifications or work under the terms of Section 1
+above, provided that you also meet all of these conditions:
+
+    a) You must cause the modified files to carry prominent notices
+    stating that you changed the files and the date of any change.
+
+    b) You must cause any work that you distribute or publish, that in
+    whole or in part contains or is derived from the Program or any
+    part thereof, to be licensed as a whole at no charge to all third
+    parties under the terms of this License.
+
+    c) If the modified program normally reads commands interactively
+    when run, you must cause it, when started running for such
+    interactive use in the most ordinary way, to print or display an
+    announcement including an appropriate copyright notice and a
+    notice that there is no warranty (or else, saying that you provide
+    a warranty) and that users may redistribute the program under
+    these conditions, and telling the user how to view a copy of this
+    License.  (Exception: if the Program itself is interactive but
+    does not normally print such an announcement, your work based on
+    the Program is not required to print an announcement.)
+
+These requirements apply to the modified work as a whole.  If
+identifiable sections of that work are not derived from the Program,
+and can be reasonably considered independent and separate works in
+themselves, then this License, and its terms, do not apply to those
+sections when you distribute them as separate works.  But when you
+distribute the same sections as part of a whole which is a work based
+on the Program, the distribution of the whole must be on the terms of
+this License, whose permissions for other licensees extend to the
+entire whole, and thus to each and every part regardless of who wrote it.
+
+Thus, it is not the intent of this section to claim rights or contest
+your rights to work written entirely by you; rather, the intent is to
+exercise the right to control the distribution of derivative or
+collective works based on the Program.
+
+In addition, mere aggregation of another work not based on the Program
+with the Program (or with a work based on the Program) on a volume of
+a storage or distribution medium does not bring the other work under
+the scope of this License.
+
+  3. You may copy and distribute the Program (or a work based on it,
+under Section 2) in object code or executable form under the terms of
+Sections 1 and 2 above provided that you also do one of the following:
+
+    a) Accompany it with the complete corresponding machine-readable
+    source code, which must be distributed under the terms of Sections
+    1 and 2 above on a medium customarily used for software interchange; or,
+
+    b) Accompany it with a written offer, valid for at least three
+    years, to give any third party, for a charge no more than your
+    cost of physically performing source distribution, a complete
+    machine-readable copy of the corresponding source code, to be
+    distributed under the terms of Sections 1 and 2 above on a medium
+    customarily used for software interchange; or,
+
+    c) Accompany it with the information you received as to the offer
+    to distribute corresponding source code.  (This alternative is
+    allowed only for noncommercial distribution and only if you
+    received the program in object code or executable form with such
+    an offer, in accord with Subsection b above.)
+
+The source code for a work means the preferred form of the work for
+making modifications to it.  For an executable work, complete source
+code means all the source code for all modules it contains, plus any
+associated interface definition files, plus the scripts used to
+control compilation and installation of the executable.  However, as a
+special exception, the source code distributed need not include
+anything that is normally distributed (in either source or binary
+form) with the major components (compiler, kernel, and so on) of the
+operating system on which the executable runs, unless that component
+itself accompanies the executable.
+
+If distribution of executable or object code is made by offering
+access to copy from a designated place, then offering equivalent
+access to copy the source code from the same place counts as
+distribution of the source code, even though third parties are not
+compelled to copy the source along with the object code.
+
+  4. You may not copy, modify, sublicense, or distribute the Program
+except as expressly provided under this License.  Any attempt
+otherwise to copy, modify, sublicense or distribute the Program is
+void, and will automatically terminate your rights under this License.
+However, parties who have received copies, or rights, from you under
+this License will not have their licenses terminated so long as such
+parties remain in full compliance.
+
+  5. You are not required to accept this License, since you have not
+signed it.  However, nothing else grants you permission to modify or
+distribute the Program or its derivative works.  These actions are
+prohibited by law if you do not accept this License.  Therefore, by
+modifying or distributing the Program (or any work based on the
+Program), you indicate your acceptance of this License to do so, and
+all its terms and conditions for copying, distributing or modifying
+the Program or works based on it.
+
+  6. Each time you redistribute the Program (or any work based on the
+Program), the recipient automatically receives a license from the
+original licensor to copy, distribute or modify the Program subject to
+these terms and conditions.  You may not impose any further
+restrictions on the recipients' exercise of the rights granted herein.
+You are not responsible for enforcing compliance by third parties to
+this License.
+
+  7. If, as a consequence of a court judgment or allegation of patent
+infringement or for any other reason (not limited to patent issues),
+conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License.  If you cannot
+distribute so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you
+may not distribute the Program at all.  For example, if a patent
+license would not permit royalty-free redistribution of the Program by
+all those who receive copies directly or indirectly through you, then
+the only way you could satisfy both it and this License would be to
+refrain entirely from distribution of the Program.
+
+If any portion of this section is held invalid or unenforceable under
+any particular circumstance, the balance of the section is intended to
+apply and the section as a whole is intended to apply in other
+circumstances.
+
+It is not the purpose of this section to induce you to infringe any
+patents or other property right claims or to contest validity of any
+such claims; this section has the sole purpose of protecting the
+integrity of the free software distribution system, which is
+implemented by public license practices.  Many people have made
+generous contributions to the wide range of software distributed
+through that system in reliance on consistent application of that
+system; it is up to the author/donor to decide if he or she is willing
+to distribute software through any other system and a licensee cannot
+impose that choice.
+
+This section is intended to make thoroughly clear what is believed to
+be a consequence of the rest of this License.
+
+  8. If the distribution and/or use of the Program is restricted in
+certain countries either by patents or by copyrighted interfaces, the
+original copyright holder who places the Program under this License
+may add an explicit geographical distribution limitation excluding
+those countries, so that distribution is permitted only in or among
+countries not thus excluded.  In such case, this License incorporates
+the limitation as if written in the body of this License.
+
+  9. The Free Software Foundation may publish revised and/or new versions
+of the General Public License from time to time.  Such new versions will
+be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+Each version is given a distinguishing version number.  If the Program
+specifies a version number of this License which applies to it and "any
+later version", you have the option of following the terms and conditions
+either of that version or of any later version published by the Free
+Software Foundation.  If the Program does not specify a version number of
+this License, you may choose any version ever published by the Free Software
+Foundation.
+
+  10. If you wish to incorporate parts of the Program into other free
+programs whose distribution conditions are different, write to the author
+to ask for permission.  For software which is copyrighted by the Free
+Software Foundation, write to the Free Software Foundation; we sometimes
+make exceptions for this.  Our decision will be guided by the two goals
+of preserving the free status of all derivatives of our free software and
+of promoting the sharing and reuse of software generally.
+
+			    NO WARRANTY
+
+  11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
+FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW.  EXCEPT WHEN
+OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
+PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
+OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.  THE ENTIRE RISK AS
+TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU.  SHOULD THE
+PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
+REPAIR OR CORRECTION.
+
+  12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
+REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
+INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
+OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
+TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
+YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
+PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGES.
+
+		     END OF TERMS AND CONDITIONS

data/Manifest.txt

@@ -0,0 +1,38 @@
+BUGS
+CHANGELOG
+INSTALL
+install.rb
+LICENSE
+Manifest.txt
+README
+Rakefile
+setup.rb
+ruby184gc.patch
+ruby185gc.patch
+bin/railsbench
+lib/railsbench/version.rb
+lib/railsbench/gc_info.rb
+lib/railsbench/perf_info.rb
+lib/railsbench/perf_utils.rb
+lib/railsbench/write_headers_only.rb
+lib/railsbench/railsbenchmark.rb
+lib/benchmark.rb
+script/perf_bench
+script/perf_comp
+script/perf_comp_gc
+script/perf_diff
+script/perf_diff_gc
+script/perf_html
+script/perf_loop
+script/perf_plot
+script/perf_plot_gc
+script/perf_prof
+script/perf_run
+script/perf_run_gc
+script/perf_tex
+script/perf_times
+script/perf_times_gc
+script/run_urls
+config/benchmarking.rb
+config/benchmarks.rb
+config/benchmarks.yml

data/README

@@ -0,0 +1,254 @@
+Railsbench is a small collection of ruby and shell scripts which make
+measuring raw performance of rails apps a snap. All tests are run from
+the command prompt making performance regression testing easy.
+
+In addition, a patch for the ruby garbage collector is provided, which
+can be used to reduce the amount of time spent doing garbage
+collection, trading memory for speed, as usual (see file GCPATCH for
+details). In addition, applying the patch will enhance performance
+data obtained from the various scripts (and some won't run at all
+without the patch).
+
+This software was written and conceived by Stefan Kaes. The author can
+be reached via email: <skaes@gmx.net>. Please send comments, bug
+reports, patches or feature requests to this address.
+
+
+FILES
+
+  railsbench.rb
+                - defines classes RailsBenchmark and RailsBenchmarkWithActiveRecordStore
+                  switches:
+                  -gcXXX        : perform gc after XXX requests
+                  -log[=level]  : turn on rails logging (at given level)
+                  -nochache     : turn off rails caching
+                  -path         : print $: after loading rails and exit
+
+  config/bechmarks.yml
+                - specification of urls to benchmark
+                  install into $RAILS_ROOT/config
+
+  config/benchmarks.rb
+                - defines constant RAILSBENCH which is used by script perf_bench
+                  modify this to add custom argument processing
+                  install to $RAILS_ROOT/config
+
+  perf_bench n options
+                - main ruby script to run a given benchmark
+                  switches (in addition to railsbench switches):
+                  -mix      : alternates urls in given benchmark
+
+  run_urls n options
+                - run a given benchmark (without benchmarking)
+                  useful for checking response codes, like so:
+
+                      run_urls 1 -bm=all | grep Status:
+
+                  switches as for perf_bench plus
+                  -warmup   : run all uris once before measuring
+                  -svlPV    : run test using SVL Ruby Performance Validator
+                  -svlMV    : run test using SVL Ruby Memory Validator
+
+  perf_run n [ option-string [ config-name ] ]
+                - run a given benchmark, store performance data in a file
+                  in directory $RAILS_PERF_DATA and print results
+
+  perf_diff n common-options option-string1 option-string2 [ config-name1 [ config-name2 ] ]
+                - run a given benchmark with two different sets of arguments
+                  store data into directory $RAILS_PERF_DATA and print
+                  comparison data
+
+  perf_loop n options
+                - used by perf_run and perf_diff
+                  calls perf_bench $RAILS_PERF_RUNS times
+
+  perf_times file
+                - analyse and print performance data
+
+  perf_comp [-narrow] [-skip_urls] file1 file2
+                - compare two performance data sets and print results
+                  -narrow    => produce narrow output
+                  -skip_urls => don't print url map (use with -narrow)
+
+  perf_html [-nocss] [-gc] [file]
+                - format output taken from perf_comp and produce a HTML table
+                  for inclusion in HTML pages. Reads from standard input.
+                  options:
+                    -noccs   suppress css output
+                    -notable suppress table output
+                    -gc      include gc statistics
+
+  perf_run_gc n [ option-string [ config-name ] ]
+                - run a given benchmark, store performance data in a file
+                  in directory $RAILS_PERF_DATA and print results
+                - requires Ruby GC patch
+
+  perf_times_gc file
+                - analyse and print garbage collection statistics, which you
+                  can produce by running perf_run_gc
+
+  perf_diff_gc n common-options option-string1 option-string2 [ config-name1 [ config-name2 ] ]
+                - run a given benchmark with two different sets of arguments
+                  store GC data into directory $RAILS_PERF_DATA and print
+                  GC stats comparison using perf_comp_gc
+                - requires Ruby GC patch
+
+  perf_comp_gc file1 file2
+                - compare two GC performance data sets and print results
+
+  perf_prof n [ option-string [ config-name ] ]
+                - run a given benchmark using ruby-prof for profiling,
+                  store profile data in a HTML file in directory $RAILS_PERF_DATA
+                  file name is computed from date and benchmark name as described above
+                  but has a .html extension instead of .txt
+
+  perf_plot n [ options ] file1 file2 ...
+                - plot performance data from raw performance files using gruff
+                  see source for options
+
+ENVIRONMENT
+
+  RAILS_ROOT
+                - must be set to point to your rails app
+
+  RAILS_PERF_DATA
+                - performance data sets will be stored into this directory
+                  if not set, $HOME will be used
+
+  RAILS_PERF_RUNS
+                - the number of times perf_loop will run perf_bench on a single invocation
+                  if not set, 3 runs will be performed
+
+  RAILS_BENCHMARK_FILE
+                - perf_bench sends its output to this file
+
+
+USAGE
+
+The two main scripts are named perf_run and perf_diff.
+
+    perf_run 100
+
+runs the list of urls named "default" specified in benchmkarks.yml
+(see below), requesting each url $RAILS_PERF_RUNS * 100 times.
+
+   perf_run 100 "-bm=list -aws"
+
+runs the benchmark named 'list' and passes the expanded second
+argument to the rails app. By processing arguments inside your
+environment.rb file, you can set performance affecting options. For
+example, you could load action web service only if -aws is passed and
+measure the performance effect of omitting it.
+
+Benchmark data is stored in directory $RAILS_PERF_DATA, which should
+be set in your environment. If not set, $HOME is used. By default,
+data is stored in file $RAILS_PERF_DATA/perf_run.$BENCHMARK.txt, where
+BENCHMARK will be set according to the -bm option.
+
+   perf_run 100 "-bm=index -mail" mail
+
+will store benchmark data in file
+
+   $RAILS_PERF_DATA/`current-date�.index.mail.txt.
+
+You can get nicely formatted output of benchmark data by running
+
+  perf_times file
+
+Script perf_run will automatically print the obtained data after
+finishing the run using perf_times.
+
+Script perf_diff runs a list of urls with two different option lists. So
+
+  perf_diff 100 "-bm=blogs -mysql_session" "-mail=0" "-mail=1" cf1 cf2
+
+would run benchmark 'blogs' twice, first as
+
+  perf_run 100 "-bm=blogs -mysql_session -mail=0" cf1
+
+and then
+
+  perf_run 100 "-bm=blogs -mysql_session -mail=1" cf2
+
+printing a comparison of the bechmark data obtained after finishing
+the second run. cf1 and cf2 can be omitted, in which case data is
+stored in $RAILS_PERF_DATA/perf_run1.$BENCHMARK.txt and
+$RAILS_PERF_DATA/perf_run2.$BENCHMARK.txt.
+
+Script perf_bench can also be invoked manually to run a given
+benchmark like so:
+
+  perf_bench 100 -bm=blogs -mysql_session -mail=1 >/dev/null
+
+Performance data is sent to $RAILS_BENCHMARK_FILE, HTML output ends up
+on stdout. If RAILS_BENCHMARK_FILE is not set, performance data is
+sent to stderr.
+
+Scripts perf_run_gc and perf_times_gc can be used to analyse GC performance:
+
+  perf_run_gc 100 "-bm=all -lib=stable11" gc.log
+  perf_times_gc gc.log
+
+It will produce output looking like this:
+
+GC data file: d:/perfdata/xp/perf_run.uncached.gc.txt
+
+collections            :        40
+garbage total          :   8188429
+gc time total (sec)    :         4.03
+garbage per request    :      2047.11
+requests per collection:       100.00
+
+                     mean  stddev%          min          max
+gc time(ms):       101.38     10.0        93.00       125.00
+heap slots :    400000.00      0.0    400000.00    400000.00
+live       :    192914.31      0.2    191787.00    193197.00
+freed      :    207085.69      0.2    206803.00    208213.00
+freelist   :         0.00      0.0         0.00         0.00
+
+Note that these numbers, especially requests per collection, are only
+an approximation. This is due to the fact that run_urls will add one
+final garbage collection call at the end of the run. Of course, higher
+number of iterations will produce more accurate data. Also, if the
+benchmark lists several uris, garbage per request will not give you
+meaningful information.
+
+
+CONFIGURATION
+
+Benchmarks are configured through file benchmarks.yml. Example:
+
+  default:
+    index, other
+
+  index:
+    uri: /test/index
+    new_session: true
+
+  other:
+    -
+      uri: /test/list
+    -
+      uri: /test/alphabetic
+      query_params: page=7
+
+defines 3 benchmarks:
+
+  "other"    consists of 2 urls (/test/list and /test/alphabetic)
+  "index"    will run (/test/index)
+  "default"  will run all urls in "index" and "other"
+
+uri: is mandatory, query_params: and new_session: are optional.
+
+Instead of
+
+    uri: /test/alphabetic
+    query_params: page=7
+
+one could have written
+
+    uri: /test/alphabetic?page=7
+
+new_session: specifies whether a session cookie is sent with the
+request (default). To make this work correctly, you need to set
+session_data on the benchmarker instance. See benchmarks.rb for details.