deep_test 1.1.4 → 1.2.0

data/CHANGELOG

@@ -1,4 +1,14 @@
-* 1.1.4 (March 11, 2008)
+1.2.0 (March 28, 2008)
+
+* Added capability to run distributed
+* Changed to use straight DRb instead of Rinda (performance)
+* Changed error raised on before/after(:all) to warning
+* Added worker listener to setup mysql database for each worker
+* Run test server in same process
+* Check rspec version when loading rspec support
+* Add support for multiple worker listeners
+
+1.1.4 (March 11, 2008)
 
 * Fixed 2 RSpec pending example bugs
 * Fixed setting spec_opts with DeepTest bug
@@ -6,21 +16,21 @@
 * Added server_port to configuration options
 * Wait for signal from server indicating it is ready on startup
 
-* 1.1.3 (March 10, 2008)
+1.1.3 (March 10, 2008)
 
 * Added Support for RSpec
 * Added DeepTest::TestTask#timeout_in_seconds to configure Rinda IO timeouts
 * Added worker_listener to configuration options
 
-* 1.1.2 (February 9, 2008)
+1.1.2 (February 9, 2008)
 
 * Fix bug with run_test_suite script not exiting with correct exit code.
 
-* 1.1.1 (February 9, 2008)
+1.1.1 (February 9, 2008)
 
 * No longer fork from Rake process.
 
-* 1.1.0 (December 23, 2007)
+1.1.0 (December 23, 2007)
 
 * DeepTest::TestTask#processes= is now DeepTest::TestTask#number_of_workers=
 * No longer depend on the Daemons gem, use DeepTest::Warlock instead.

data/README

@@ -1,6 +1,9 @@
 = DeepTest
 
-DeepTest enables tests to run in parallel using multiple processes.
+DeepTest enables tests to run in parallel using multiple processes.  Processes
+may spawned locally to take advantage of multiple processors on a single
+machine or distributed across many machines to take advantage of distributed
+processing.
 
 == Usage
 
@@ -28,10 +31,16 @@ In your Rakefile:
   end
 
 
-If you want to specify a listener to be notified of worker events:
+=== Specifying a Listener to be Notified of Events
 
 In my_worker_listener.rb
   class MyWorkerListener
+    def before_sync
+    end
+
+    def before_starting_workers
+    end
+
     def starting(worker)
     end
 
@@ -45,14 +54,148 @@ In my_worker_listener.rb
 In your Rakefile
 
   DeepTest::TestTask.new "task_name" do |t|
-    t.pattern = "test/**/*_test.rb"
+    ... 
     t.worker_listener = "MyWorkerListener"
   end
 
 An instance of MyWorkerListener will be created for each worker that is
 started, and will receive events from that worker.  However, you must ensure
 that the MyWorkerListener class is loaded at the time that the test files are
-loaded, so it is available to be instantiated.
+loaded, so it is available to be instantiated.  You can specify multiple
+listener classes by separating them with a comma.  For more information about
+when events are triggered, see the documentation at
+DeepTest::NullWorkerListener.
+
+
+=== Setting Up A New Database For Each Worker
+
+By default, DeepTest will reinitialize ActiveRecord connections if ActiveRecord
+is loaded when the workers are started.  This means all workers are running
+against the same database.  You may want each worker to use a database
+decidicated to it.  To facilitate this, DeepTest provides a worker listener to
+help you.  If you're using Rails with Mysql, simply configure DeepTest as 
+follows in your Rakefile:
+
+  DeepTest::TestTask.new "task_name" do |t|
+    ...
+    t.worker_listener = "DeepTest::Database::MysqlSetupListener"
+  end
+
+
+Before spawning workers, DeepTest will dump the schema of the database for the
+current Rails environment (usually test).  As each worker starts up, the
+listener will create a database dedicated to that worker and load the schema
+into it.  The database will be dropped when the worker process exits.  
+
+If you're using Mysql but not using Rails, you'll have to create a subclass of
+MysqlSetupListener and override +master_database_config+ and +dump_file_name+,
+as the default implementations of these methods are Rails specific.
+
+
+If you're using a database other than Mysql, read the documentation for
+DeepTest::Database::SetupListener and create a new subclass for your database
+type.  If you do this, please consider contributing your subclass back to the
+project so that it can be included in later releases of DeepTest.
+
+=== Distributing Tests Across Multiple Machines
+
+In addition to running your tests in parallel, DeepTest can also distribute
+them across multiple machines.  It does this by first mirroring the local
+working copy that launched the tests on each machine that will be running tests.
+Then workers are launched on each of the machines and consume tests in the same
+fashion as when DeepTest is running locally.
+
+==== Requirements
+
+Before you can distribute tests, you must ensure that all the machines involved
+(including the machine with the local working copy) have rsync installed for
+mirroring working copies.  You must also have either SSH or an RSync daemon
+exposing your local working copy running on the local development machine.  For
+more information about rsync, visit the rsync
+webpage[http://www.samba.org/rsync/].  Currently only passwordless access is
+supported, so you must either setup your RSync daemon to be accessible without
+a password or enable passwordless SSH access from the test machines to the
+local development machine. DeepTest must also be installed as a gem on each
+test machine and available either as a gem or in your project on the local
+machine.
+
+==== Starting a Test Server
+
+On each test machine, execute the following:
+  > deep_test test_server
+
+This will launch a server providing mirroring and worker services.  By default,
+2 workers will be launched for each set of tests run.  If you wish to change
+the number of workers, simply specify the --number_of_workers option.  For
+information about what options are available, use -h.  The test_server will
+print out the uri it is bound to, druby://<hostname>:4022 by default.
+
+==== Starting a Master Test Server
+
+On a single machine (probably one of the test machines), execute this:
+  > deep_test master_test_server <uris of test servers>
+
+The master_test_server will also print it's binding uri,
+druby://<hostname>:4021 by default.  It will also provide a webserver bound
+to port 4020 that provides a status page summarizing the state of all the
+test servers. 
+
+==== Configuring Your Project
+
+If you're using rsync over ssh, create a DeepTest test task similar to that
+below in your Rakefile.
+
+  DeepTest::TestTask.new "deep_test_distributed" do |t|
+    t.pattern = "test/**/*_test.rb" # whatever is appropriate for your project
+    t.distributed_server = <drb uri of master_test_server>
+    t.sync_options = {
+      :source => <absolute path of project root on machine>,
+      :username => "username"
+    }
+  end
+
+The :source entry in sync_options can be easily calculated based on the value
+of \_\_FILE__ when defining the task.  :username will be used by rsync to ssh
+back to the local machine and mirror the working copy.
+
+If you have an rsync daemon running in your local machine, configure the
+rake task as follows.
+
+  DeepTest::TestTask.new "deep_test_distributed" do |t|
+    t.pattern = "test/**/*_test.rb" # whatever is appropriate for your project
+    t.distributed_server = <drb uri of master_test_server>
+    t.sync_options = {
+      :source => <name of rsync module from daemon configuration>,
+      :daemon => true,
+      :username => "username"
+    }
+  end
+
+
+Username is optional in both cases.  You'll need to either setup passwordless
+ssh access or run an rsync daemon that doesn't require passwords.
+
+There may be other options you'd like to pass to rsync in your particular
+scenario.  This can be done by adding an :rsync_options entry to sync_options.
+For example, if you're working on a Rails project you'll probably want to at
+least have something like this:
+
+  DeepTest::TestTask.new "deep_test_distributed" do |t|
+    ...
+    excludes = %w[.svn tmp/** log/**]
+    t.sync_options = {
+      ...
+      :rsync_options => excludes.map {|s| "'--exclude=#{s}'"}.join(' ')
+    }
+  end
+
+That way you can avoid spending any time mirroring tmp and log files that don't
+have any effect on the tests.  If you are running distributed tests against a
+database, consult the section above about creating a new database for each
+worker to see how to configure DeepTest for your project.  
+
+Any number of projects can be run using the same test servers, as long as
+they're all using the same version of Ruby and DeepTest.  
 
 == Contributors
 
@@ -63,4 +206,5 @@ loaded, so it is available to be instantiated.
 * Scott Taylor
 
 == License
-Released under Ruby's[http://www.ruby-lang.org/en/LICENSE.txt] license[http://www.ruby-lang.org/en/LICENSE.txt]
+
+DeepTest Released under Ruby's[http://www.ruby-lang.org/en/LICENSE.txt] license[http://www.ruby-lang.org/en/LICENSE.txt]

data/Rakefile

@@ -6,25 +6,136 @@ require 'rake/contrib/sshpublisher'
 $LOAD_PATH << File.dirname(__FILE__) + "/lib"
 require "deep_test/rake_tasks"
 
-task :default => %w[test spec failing_test deep_test deep_spec]
+task :default => %w[
+  test 
+  spec 
+  failing_test
+  deep_test
+  deep_spec
+  run_distributed
+  run_distributed_with_worker_down
+  run_distributed_with_failover
+  test_rails_project
+]
 
 Rake::TestTask.new do |t|
   t.pattern = "test/**/*_test.rb"
   t.libs += ['test', 'lib']
 end
 
-Spec::Rake::SpecTask.new(:spec) do |t|
-  t.spec_files = FileList['spec/**/*_spec.rb']
+DeepTest::TestTask.new :deep_test do |t|
+  t.number_of_workers = 2
+  t.pattern = "test/**/*_test.rb"
+  t.metrics_file = "deep_test.metrics"
 end
 
-DeepTest::TestTask.new :deep_test do |t|
+DeepTest::TestTask.new(:distributed_test) do |t|
   t.number_of_workers = 2
   t.pattern = "test/**/*_test.rb"
+  t.distributed_server = "druby://localhost:8000"
+  t.sync_options = {:source => File.dirname(__FILE__), 
+                    :local => true,
+                    :rsync_options => "--exclude=.svn"}
+end
+
+def rspec_present?
+  defined?(Spec)
 end
 
-Spec::Rake::SpecTask.new(:deep_spec) do |t|
-  t.spec_files = FileList['spec/**/*_spec.rb']
-  t.deep_test :number_of_workers => 2
+if rspec_present?
+  Spec::Rake::SpecTask.new(:spec) do |t|
+    t.spec_files = FileList['spec/**/*_spec.rb']
+  end
+  
+  Spec::Rake::SpecTask.new(:deep_spec) do |t|
+    t.spec_files = FileList['spec/**/*_spec.rb']
+    t.deep_test :number_of_workers => 2
+  end
+
+  Spec::Rake::SpecTask.new(:distributed_spec) do |t|
+    t.spec_files = FileList['spec/**/*_spec.rb']
+    t.deep_test :number_of_workers => 2, 
+                :distributed_server => "druby://localhost:8000",
+                :sync_options => {:source => File.dirname(__FILE__), 
+                                  :local => true,
+                                  :rsync_options => "--exclude=.svn"}
+  end
+end
+
+
+DeepTest::TestTask.new(:distribute_tests_to_minis) do |t|
+  t.number_of_workers = 2
+  t.pattern = "test/**/*_test.rb"
+  t.distributed_server = "druby://alpha.local:8000"
+  t.sync_options = {:source => File.dirname(__FILE__), 
+                    :username => "tworker",
+                    :password => "thought",
+                    :rsync_options => "--exclude=.svn"}
+end
+
+task :run_distributed do |t|
+  begin
+    FileUtils.mkdir('/tmp/test_1') unless File.exist?('/tmp/test_1')
+    FileUtils.mkdir('/tmp/test_2') unless File.exist?('/tmp/test_2')
+
+    test_1_pid = fork do
+      exec "ruby bin/deep_test test_server --uri drubyall://localhost:8001 --work_dir /tmp/test_1"
+    end
+
+    test_2_pid = fork do
+      exec "ruby bin/deep_test test_server --uri drubyall://localhost:8002 --work_dir /tmp/test_2"
+    end
+
+    master_pid = fork do
+      exec "ruby bin/deep_test master_test_server --uri drubyall://localhost:8000 druby://localhost:8001 druby://localhost:8002"
+    end
+
+    sleep 1
+
+    Rake::Task[:distributed_test].invoke
+    Rake::Task[:distributed_spec].invoke
+
+    sh "ruby bin/deep_test test_throughput druby://localhost:8000 20"
+  ensure
+    Process.kill('TERM', master_pid) if master_pid rescue nil
+    Process.kill('TERM', test_1_pid) if test_1_pid rescue nil
+    Process.kill('TERM', test_2_pid) if test_2_pid rescue nil
+    Process.waitall
+  end
+
+  sleep 1
+end
+
+task :run_distributed_with_worker_down do |t|
+  begin
+    FileUtils.mkdir('/tmp/test_1') unless File.exist?('/tmp/test_1')
+
+    test_1_pid = fork do
+      exec "ruby bin/deep_test test_server --uri drubyall://localhost:8001 --work_dir /tmp/test_1"
+    end
+
+    # don't start worker 2
+
+    master_pid = fork do
+      exec "ruby bin/deep_test master_test_server --uri drubyall://localhost:8000 druby://localhost:8001 druby://localhost:8002"
+    end
+
+    sleep 1
+
+    # Use throughput to make sure we can run tests
+    sh "ruby bin/deep_test test_throughput druby://localhost:8000 20"
+  ensure
+    Process.kill('TERM', master_pid) if master_pid rescue nil
+    Process.kill('TERM', test_1_pid) if test_1_pid rescue nil
+    Process.waitall
+  end
+end
+
+task :run_distributed_with_failover do |t|
+  puts
+  puts "*** Running distributed with no server - expect a failover message ***"
+  puts
+  Rake::Task[:distributed_spec].execute
 end
 
 task :failing_test do
@@ -39,12 +150,21 @@ task :failing_test do
   end
 end
 
+Rake::TestTask.new(:test_rails_project) do |t|
+  t.pattern = "sample_rails_project/deep_test.rb"
+end
+
 desc "Generate documentation"
 Rake::RDocTask.new(:rdoc) do |rdoc|
   rdoc.rdoc_dir = "doc"
   rdoc.title    = "DeepTest"
   rdoc.options << '--line-numbers'
-  rdoc.rdoc_files.include('README', 'CHANGELOG')
+  rdoc.rdoc_files.include(
+    'README', 
+    'CHANGELOG', 
+    'lib/deep_test/null_worker_listener.rb',
+    'lib/deep_test/database/*.rb'
+  )
 end
 
 desc "Upload RDoc to RubyForge"
@@ -58,19 +178,19 @@ specification = Gem::Specification.new do |s|
   s.platform = Gem::Platform::RUBY
 	s.name   = "deep_test"
   s.summary = "DeepTest runs tests in multiple processes."
-	s.version = "1.1.4"
+	s.version = "1.2.0"
 	s.author = "anonymous z, Dan Manges, David Vollbracht"
 	s.description = s.summary
 	s.email = "daniel.manges@gmail.com"
   s.homepage = "http://deep-test.rubyforge.org"
   s.rubyforge_project = "deep-test"
+  s.executables << "deep_test"
 
   s.has_rdoc = true
   s.extra_rdoc_files = ['README', 'CHANGELOG']
   s.rdoc_options << '--title' << "DeepTest" << '--main' << 'README' << '--line-numbers'
 
-  s.autorequire = "deep_test"
-  s.files = FileList['{lib,script,test}/**/*.{rb,rake}', 'README', 'CHANGELOG', 'Rakefile'].to_a
+  s.files = FileList['{lib,script,test,bin}/**/*.{rb,rake,rhtml}', 'README', 'CHANGELOG', 'Rakefile'].to_a
 end
 
 Rake::GemPackageTask.new(specification) do |package|

data/bin/deep_test

@@ -0,0 +1,15 @@
+#! /usr/bin/env ruby
+
+script_dir = File.dirname(__FILE__) + '/../script/public'
+script_file = script_dir + "/#{ARGV.shift}.rb"
+
+unless File.exist?(script_file)
+  puts 'Usage: deep_test <script> [arguments]'
+  puts '  Available Scripts: '
+  Dir.glob(script_dir + '/*.rb').each do |name|
+    puts "    #{File.basename(name, '.rb')}"
+  end
+  exit(1)
+end
+
+load script_file

data/lib/deep_test.rb

@@ -4,29 +4,87 @@ module DeepTest
       @logger ||= DeepTest::Logger.new($stdout)
     end
   end  
+
+  # Fork in a separate thread.  If we fork from a DRb thread
+  #  (a thread handling a method call invoked over drb),
+  #  DRb still thinks the current object is the DRb Front Object,
+  #  and reports its uri as the same as in the parent process, even
+  #  if you restart DRb service.
+  #
+  def self.drb_safe_fork(&block)
+    Thread.new {Process.fork(&block)}.value
+  end
+
+  def self.init(options)
+    return if @initialized
+    @initialized = true
+    Metrics::Gatherer.setup(options)
+  end
+
+  class WorkUnitNeverReceivedError < StandardError
+    def initialize
+      super "DeepTest result never received.  Maybe an error was printed above?"
+    end
+
+    def backtrace
+      []
+    end
+  end
 end
 
 require "logger"
-require "rinda/ring"
-require "rinda/tuplespace"
+require "drb"
+require "timeout"
+require "thread"
+require "socket"
+require "webrick"
+require "timeout"
 
 require File.dirname(__FILE__) + "/deep_test/extensions/object_extension"
+require File.dirname(__FILE__) + "/deep_test/extensions/drb_extension"
 
 require File.dirname(__FILE__) + "/deep_test/deadlock_detector"
-require File.dirname(__FILE__) + "/deep_test/tuple_space_factory"
-require File.dirname(__FILE__) + "/deep_test/rinda_blackboard"
+require File.dirname(__FILE__) + "/deep_test/local_workers"
 require File.dirname(__FILE__) + "/deep_test/logger"
+
+require File.dirname(__FILE__) + "/deep_test/marshallable_exception_wrapper"
 require File.dirname(__FILE__) + "/deep_test/null_worker_listener"
+require File.dirname(__FILE__) + "/deep_test/listener_list"
+require File.dirname(__FILE__) + "/deep_test/option"
 require File.dirname(__FILE__) + "/deep_test/options"
 require File.dirname(__FILE__) + "/deep_test/process_orchestrator"
+require File.dirname(__FILE__) + "/deep_test/result_reader"
 require File.dirname(__FILE__) + "/deep_test/rspec_detector"
 require File.dirname(__FILE__) + "/deep_test/server"
 require File.dirname(__FILE__) + "/deep_test/test_task"
 require File.dirname(__FILE__) + "/deep_test/worker"
 require File.dirname(__FILE__) + "/deep_test/warlock"
 
+require File.dirname(__FILE__) + "/deep_test/database/setup_listener"
+require File.dirname(__FILE__) + "/deep_test/database/mysql_setup_listener"
+
+require File.dirname(__FILE__) + "/deep_test/distributed/dispatch_controller"
+require File.dirname(__FILE__) + "/deep_test/distributed/drb_client_connection_info"
+require File.dirname(__FILE__) + "/deep_test/distributed/filename_resolver"
+require File.dirname(__FILE__) + "/deep_test/distributed/master_test_server"
+require File.dirname(__FILE__) + "/deep_test/distributed/test_server"
+require File.dirname(__FILE__) + "/deep_test/distributed/test_server_status"
+require File.dirname(__FILE__) + "/deep_test/distributed/test_server_workers"
+require File.dirname(__FILE__) + "/deep_test/distributed/multi_test_server_proxy"
+require File.dirname(__FILE__) + "/deep_test/distributed/null_work_unit"
+require File.dirname(__FILE__) + "/deep_test/distributed/remote_worker_client"
+require File.dirname(__FILE__) + "/deep_test/distributed/remote_worker_server"
+require File.dirname(__FILE__) + "/deep_test/distributed/rsync"
+require File.dirname(__FILE__) + "/deep_test/distributed/throughput_runner"
+require File.dirname(__FILE__) + "/deep_test/distributed/throughput_statistics"
+require File.dirname(__FILE__) + "/deep_test/distributed/throughput_worker_client"
+
+require File.dirname(__FILE__) + "/deep_test/metrics/gatherer"
+
 DeepTest::RSpecDetector.if_rspec_available do
   require File.dirname(__FILE__) + "/deep_test/spec"
 end
 require File.dirname(__FILE__) + "/deep_test/test"
 
+require File.dirname(__FILE__) + "/deep_test/ui/console"
+require File.dirname(__FILE__) + "/deep_test/ui/null"