xli-dtr 1.0.0 → 1.1.0

data/CHANGES

@@ -1,5 +1,26 @@
 = DTR Changelog
 
+== release 1.1.0
+* Provided new setup test environment strategy for supporting running tests in multi-processes on one machine.
+  * DTR used to run setup agent environment command before runners started in same directory with runners.
+  * New version split runners into different directory to run test. And run setup agent command after runner started.
+  * Run 'rake db:drop db:create db:test:prepare' for runner to setup database environment before runner runs any test, when there is no agent setup environment command setted and there is 'config/database.yml.dtr' or 'config/database.yml' exists. If './config/database.yml.dtr' specified, it would be copy as ./config/database.yml for setup database. For Sqlite, there is no need to create a './config/database.yml.dtr' for dtr test task. The following is an example for mysql:
+    development:
+      database: development_<%= ENV['DTR_RUNNER_NAME'] %>
+      adapter: mysql
+      username: root
+      password: 
+      host: localhost
+    test:
+      database: test_<%= ENV['DTR_RUNNER_NAME'] %>
+      adapter: mysql
+      username: root
+      password: 
+      host: localhost
+* Moved all DTR options into configuration file '.dtr_env_pstore', including runners, agent environment setup command and working_env.
+* Removed dependence of Process.fork, so that agent works OS independent.
+* Output agent runners stats info after all test finished.
+
 == release 1.0.0
 * support synchronizing codebase
 * lookup agents by broadcast

data/README.rdoc

@@ -18,11 +18,12 @@ DTR has the following features:
 
 DTR works in two parts: Runner Agent and DTR Master.
 
-* Runner Agent is a DRb service hosting on distributed machines to run tests. For using 'fork' to create runner process, Runner Agent can't run on Windows directly. Requisites for running agent: Unix/Linux based OS System for forking sub-process; 'unzip' command for extracting codebase package.
+* Runner Agent is a DRb service hosting on distributed machines to run tests. 
+  Requisites for running agent: 'unzip' command for extracting codebase package.
 
 * DTR Master is the process finding runner service to run tests and collect test results. It works by loading 'dtr/test_unit_injection.rb' with all test files or defining a DTR::TestTask in your rake file. Requisites for running master: 'zip' command for creating codebase package.
 
-DTR (version >= 1.0.0) supports synchronizing codebase by a DTR::PackageTask defined in your rake tasks. The DTR::PackageTask is a similar task with Rake::PackageTask, the following is a simple example:
+DTR (version >= 1.0.0) supports synchronizing codebase by a DTR::PackageTask defined in your rake tasks without any namespace. The DTR::PackageTask is a similar task with Rake::PackageTask, the following is a simple example:
 
   require 'dtr/raketasks'
   DTR::PackageTask.new do |p|
@@ -45,7 +46,11 @@ Note: Exclude('log/*') only excludes all files inside 'log' directory except 'lo
 
 == Rails plugin
 
-For Rails project, you just need copy dtr project directory into your project plugins directory. There are dtr tasks defined for you to run tests within dtr grid, which prefer you have 'config/database.yml.dtr' for setting up database. The test task is 'dtr:test'. 
+For Rails project, you can install dtr for your project by the following script:
+
+  ./script/plugin install git://github.com/xli/dtr.git
+
+There are dtr tasks defined for you to run tests within dtr grid. The test task is named 'dtr:test'. The default dtr package task is very simple, for large project you may need to custom it to decreasing time of synchronizing codebase.
 
 Your project directory name would be the default group name of dtr agents. You can specify an environment variable named 'DTR_GROUP' to change the group name. By default, dtr tasks would lookup the broadcast ip by 'ifconfig' command. If it doesn't work for you, you can specify an environment variable named 'BROADCAST_IP' to overwrite it.
 
@@ -72,6 +77,10 @@ Run the following if you haven't already:
 Install the gem:
 
   sudo gem install xli-dtr
+  
+== Documentation
+
+  http://dtr.rubyforge.org/
 
 == Running the DTR Test Suite
 
@@ -84,28 +93,30 @@ If you wish to run the unit and functional tests that come with DTR:
 
 == Simple Example
 
-Start DTR agent with providing 1 runner in group 'mysql-firefox' as follows:
+Start DTR agent with providing 1 runner in group 'my_project_name' as follows:
 
-  dtr -r runner1 -g mysql-firefox
+  dtr -r runner1 -g my_project_name
 
 Type "dtr --help" for an up-to-date option summary.
-Invoking <tt>dtr</tt> without any options causes dtr to show help too.
+DTR stored your configuration automatically, and configurations would be picked up
+automatically next time you start dtr.
 
-Most of time your project test suite need setup environment before run tests, you
-can set setup command by option '--setup', for example:
+DTR supports Rails test environment setup, so you don't need to configure it unless
+your test environment setup is different with Rails project default command. You can
+setup environment as you need by option '--setup', for example:
 
   dtr -r runner1,runner2 --setup "rake db:test:prepare"
 
-You also can specify DTR_AGENT_ENV_SETUP_CMD in your master process environment to let all agents
-that are not started with '--setup' option specified to setup all agents environment.
-
-At last, you need define a DTR::TestTask:
+If you installed dtr as your Rails project plugin, you don't need the following steps.
+Otherwise, you need define a DTR::TestTask in your rake file:
 
-  require 'dtr'
+  require 'dtr/raketasks'
 
-  ENV['DTR_AGENT_ENV_SETUP_CMD'] = 'rake db:test:prepare'
+  # Uncomment the following code to specify DTR_AGENT_ENV_SETUP_CMD here to get all agents
+  # that are not started with '--setup' option specified to setup test environment.
+  # ENV['DTR_AGENT_ENV_SETUP_CMD'] = 'rake db:test:prepare'
   DTR.broadcast_list = ['broadcast_ip']
-  DTR.group = 'mysql-firefox'
+  DTR.group = 'my_project_name'
 
   DTR::TestTask.new do |t|
     t.test_files = FileList['test/**/*_test.rb']
@@ -126,12 +137,13 @@ More details about DTR::TestTask and DTR::PackageTask, see the API doc:
   ri DTR::PackageTask
 
 Notes: 
-  * DTR broadcast_list and group configuration would be cached in the directory. Name of configuration file is '.dtr_env_pstore'.
-  * For packaging codebase, DTR::PackageTask creates tasks: dtr_package, dtr_clobber_package and dtr_repackage; DTR Master just simply run a command 'rake dtr_repackage' to create the package and run 'rake dtr_clobber_package' to clean package. So make sure there are those tasks under root namespace in your rake tasks.
+
+* DTR broadcast_list and group configuration would be cached in the directory. Name of configuration file is '.dtr_env_pstore'.
+* For packaging codebase, DTR::PackageTask creates tasks: dtr_package, dtr_clobber_package and dtr_repackage; DTR Master just simply run a command 'rake dtr_repackage' to create the package and run 'rake dtr_clobber_package' to clean package. So make sure there are those tasks under root namespace in your rake tasks.
 
 == Run tests in multi-processes on one machine
 
-For running Runner in multi-processes.
+For running agent runners in multi-processes with Master process on same machine and quit with Master process.
 The following is the test task example in the rake file:
 
   require 'dtr/raketasks'
@@ -173,20 +185,19 @@ DTR agent command is invoked from the command line using:
   -r runner1_name,runner2_name     Start DTR test runner agent with unique runner names.
   -a, --broadcast_address ADDRESS  Specify broadcast address for looking up dtr agent service, e.g. 192.168.255.255. Default is 'localhost'. DTR master and monitor would need this.
   -i, --setup COMMAND              Set command for initializing test runner test environment, e.g. 'rake db:test:prepare'. Default is do nothing. You also can specify DTR_AGENT_ENV_SETUP_CMD in your master process environment to let all agents setup same environment.
-  -d DIRECTORY                     Specify a directory as agent launching & working directory.
-      --working_directory
   -m, --monitor                    Monitor the status of the dtr agents and master processes. Used for testing your dtr grid environment. CAUTION! monitoring agents causes all idle agents hang on by the monitor process.
   -v, --version                    Show version
   -h, --help                       Show this help doc
 
-  Notes: 
-    * DTR would always add 'localhost' into broadcast list.
-    * Agent start by specifying runners by -r option, every runner would be started in different process for running test. e.g. dtr -r runner1,runner2
-    * DTR master environment options:
-      * DTR_MASTER_ENV: this variable would be copied into agent process for sharing info between master and agents. Normally used in agent setup environment command.
-      * DTR_AGENT_ENV_SETUP_CMD: this variable would be applied as agent setup environment command when agent have no setup environment command specified by --setup option.
-      * DTR_LOG_LEVEL: master process logger level, e.g. ENV['DTR_LOG_LEVEL'] = Logger::DEBUG. Agent process logs would be output in master process log file, so setting this option also changes agent logger level.
-      * DTR_RUNNER_NAME: this environment variable would be provided in runner process for test environment to get the runner name to setup, e.g. setting up database configuration.
+Notes: 
+
+* DTR would always add 'localhost' into broadcast list.
+* Agent start by specifying runners by -r option, every runner would be started in different process for running test. e.g. dtr -r runner1,runner2
+* DTR master environment options:
+  * DTR_MASTER_ENV: this variable would be copied into agent process for sharing info between master and agents. Normally used in agent setup environment command.
+  * DTR_AGENT_ENV_SETUP_CMD: this variable would be applied as agent setup environment command when agent have no setup environment command specified by --setup option.
+  * DTR_LOG_LEVEL: master process logger level, e.g. ENV['DTR_LOG_LEVEL'] = Logger::DEBUG. Agent process logs would be output in master process log file, so setting this option also changes agent logger level.
+  * DTR_RUNNER_NAME: this environment variable would be provided in runner process for test environment to get the runner name, e.g. setting up database configuration.
 
 Type "dtr --help" for an up-to-date option summary.
 

data/Rakefile

@@ -101,7 +101,7 @@ else
   gem_content = <<-GEM
 Gem::Specification.new do |spec|
   spec.name = 'dtr'
-  spec.version = "1.0.0"
+  spec.version = "1.1.0"
   spec.summary = "DTR is a distributed test runner to run tests on distributed computers for decreasing build time."
 
   #### Dependencies and requirements.

data/TODO

@@ -3,12 +3,10 @@
 Send suggestions for this list to iam@li-xiao.com
 
 === To Do
+  Add teardown command for agent runners to run after all test finished and before runner quits. This and setup test environment command features provide ability to start & stop selenium proxy server, or start & stop test server.
 
 === TBD for next release
   should trigger started notification on test runner in agent Herald, for some tools are using this hook(e.g. inbrowser_test) to setup test environment.
   agent should wakeup when worker is dead, so that it could be picked up by another master process
-  auto prepare database for each runner, so that run tests in multi-processes could be easy setup.
-  provide a way to monitor agent runner efficiency
-  use system instead of fork to launch sub process in agent, which makes dtr support windows and jruby
   
 (moved DONE list to CHANGES file)

data/bin/dtr

@@ -1,6 +1,11 @@
 #!/usr/bin/env ruby
 require 'optparse'
-require 'logger'
+
+#add lib for test environment
+if File.exists?(File.expand_path(File.dirname(__FILE__) + '/../lib/dtr.rb'))
+  $LOAD_PATH.unshift File.expand_path(File.dirname(__FILE__) + '/../lib')
+end
+
 begin
   require 'dtr'
 rescue LoadError
@@ -17,7 +22,7 @@ Notes:
     * DTR_MASTER_ENV: this variable would be copied into agent process for sharing info between master and agents. Normally used in agent setup environment command.
     * DTR_AGENT_ENV_SETUP_CMD: this variable would be applied as agent setup environment command when agent have no setup environment command specified by --setup option.
     * DTR_LOG_LEVEL: master process logger level, e.g. ENV['DTR_LOG_LEVEL'] = Logger::DEBUG. Agent process logs would be output in master process log file, so setting this option also changes agent logger level.
-    * DTR_RUNNER_NAME: this environment variable would be provided in runner process for test environment to get the runner name to setup, e.g. setting up database configuration.
+    * DTR_RUNNER_NAME: this environment variable would be provided in runner process for test environment to get the runner name, e.g. setting up database configuration.
 
 DTR is a distributed test runner program for decreasing time of running ruby tests based on ruby 'test/unit' package.
 For additional information, see http://dtr.rubyforge.org/
@@ -45,7 +50,7 @@ opts = OptionParser.new do |opts|
   end
   
   opts.on("-r runner1_name,runner2_name", Array, "Start DTR test runner agent with unique runner names.") do |names|
-    DTR_AGENT_OPTIONS[:runners] = names.collect{|name| name.untaint}
+    DTR.agent_runners = names.collect{|name| name.untaint}
   end
   
   opts.on("-a", "--broadcast_address ADDRESS", "Specify broadcast address for looking up dtr agent service, e.g. 192.168.255.255. Default is 'localhost'. DTR master and monitor would need this.") do |address|
@@ -55,20 +60,22 @@ opts = OptionParser.new do |opts|
   end
 
   opts.on("-i", "--setup COMMAND", "Set command for initializing test runner test environment, e.g. 'rake db:test:prepare'. Default is do nothing. You also can specify DTR_AGENT_ENV_SETUP_CMD in your master process environment to let all agents setup same environment.") do |command|
-    DTR_AGENT_OPTIONS[:agent_env_setup_cmd] = command.untaint
+    DTR.agent_env_setup_cmd = command.untaint
   end
 
-  opts.on("-d", "--working_directory DIRECTORY", "Specify a directory as agent launching & working directory.") do |dir|
-    DTR_AGENT_OPTIONS[:launching_dir] = dir
+  opts.on("-e", "--execute RUBY_SCRIPT", "Execute a ruby script with 'dtr/agent' loaded.") do |script|
+    $agent_script = script
   end
-  
+
   opts.on_tail("-v", "--version", "Show version") do
     puts "dtr, version " + DTRVERSION
+    exit
   end
 
   opts.on_tail("-h", "--help", "Show this help doc") do 
     puts opts
     puts NOTES
+    exit
   end
 end
 
@@ -81,11 +88,11 @@ if no_argv
   puts NOTES
 end
 
-if $monitor
+if defined?($monitor)
   DTR.monitor
-end
-if DTR_AGENT_OPTIONS[:runners] && !DTR_AGENT_OPTIONS[:runners].empty?
-  Dir.chdir(DTR_AGENT_OPTIONS[:launching_dir] || '.') do
-    DTR.start_agent
-  end
+elsif defined?($agent_script)
+  require 'dtr/agent'
+  eval($agent_script)
+elsif DTR.agent_runners && !DTR.agent_runners.empty?
+  DTR.start_agent
 end

data/dtr.gemspec

@@ -1,12 +1,12 @@
 Gem::Specification.new do |spec|
   spec.name = 'dtr'
-  spec.version = "1.0.0"
+  spec.version = "1.1.0"
   spec.summary = "DTR is a distributed test runner to run tests on distributed computers for decreasing build time."
 
   #### Dependencies and requirements.
-  spec.files = ["lib/dtr/agent/brain.rb", "lib/dtr/agent/herald.rb", "lib/dtr/agent/runner.rb", "lib/dtr/agent/sync_codebase.rb", "lib/dtr/agent/sync_logger.rb", "lib/dtr/agent/test_case.rb", "lib/dtr/agent/test_unit.rb", "lib/dtr/agent/worker.rb", "lib/dtr/agent/working_env_ext.rb", "lib/dtr/agent.rb", "lib/dtr/facade.rb", "lib/dtr/master.rb", "lib/dtr/monitor.rb", "lib/dtr/raketasks.rb", "lib/dtr/shared/adapter.rb", "lib/dtr/shared/configuration.rb", "lib/dtr/shared/message_decorator.rb", "lib/dtr/shared/ruby_ext.rb", "lib/dtr/shared/service/agent.rb", "lib/dtr/shared/service/file.rb", "lib/dtr/shared/service/rinda.rb", "lib/dtr/shared/service/runner.rb", "lib/dtr/shared/service/working_env.rb", "lib/dtr/shared/service.rb", "lib/dtr/shared/sync_codebase/copiable_package.rb", "lib/dtr/shared/sync_codebase/master_ext.rb", "lib/dtr/shared/sync_codebase/package.rb", "lib/dtr/shared/sync_codebase/sync_service.rb", "lib/dtr/shared/sync_codebase.rb", "lib/dtr/shared/sync_logger.rb", "lib/dtr/shared/utils/cmd.rb", "lib/dtr/shared/utils/env_store.rb", "lib/dtr/shared/utils/logger.rb", "lib/dtr/shared/utils.rb", "lib/dtr/shared/working_env.rb", "lib/dtr/shared.rb", "lib/dtr/test_unit/drb_test_runner.rb", "lib/dtr/test_unit/injection.rb", "lib/dtr/test_unit/test_case_injection.rb", "lib/dtr/test_unit/test_suite_injection.rb", "lib/dtr/test_unit/testrunnermediator_injection.rb", "lib/dtr/test_unit/thread_safe_test_result.rb", "lib/dtr/test_unit/worker_club.rb", "lib/dtr/test_unit.rb", "lib/dtr/test_unit_injection.rb", "lib/dtr.rb", "bin/dtr", "CHANGES", "dtr.gemspec", "lib", "LICENSE.TXT", "Rakefile", "README.rdoc", "TODO"]
+  spec.files = ["lib/dtr/agent/brain.rb", "lib/dtr/agent/herald.rb", "lib/dtr/agent/process_root.rb", "lib/dtr/agent/rails_ext.rb", "lib/dtr/agent/runner.rb", "lib/dtr/agent/sync_codebase.rb", "lib/dtr/agent/sync_logger.rb", "lib/dtr/agent/test_case.rb", "lib/dtr/agent/test_unit.rb", "lib/dtr/agent/worker.rb", "lib/dtr/agent/working_env_ext.rb", "lib/dtr/agent/working_status.rb", "lib/dtr/agent.rb", "lib/dtr/facade.rb", "lib/dtr/master.rb", "lib/dtr/monitor.rb", "lib/dtr/raketasks.rb", "lib/dtr/shared/adapter/base.rb", "lib/dtr/shared/adapter/mortality.rb", "lib/dtr/shared/adapter.rb", "lib/dtr/shared/configuration.rb", "lib/dtr/shared/message_decorator.rb", "lib/dtr/shared/root.rb", "lib/dtr/shared/ruby_ext.rb", "lib/dtr/shared/service/agent.rb", "lib/dtr/shared/service/file.rb", "lib/dtr/shared/service/rinda.rb", "lib/dtr/shared/service/runner.rb", "lib/dtr/shared/service/working_env.rb", "lib/dtr/shared/service.rb", "lib/dtr/shared/sync_codebase/copiable_package.rb", "lib/dtr/shared/sync_codebase/master_ext.rb", "lib/dtr/shared/sync_codebase/package.rb", "lib/dtr/shared/sync_codebase/sync_service.rb", "lib/dtr/shared/sync_codebase.rb", "lib/dtr/shared/sync_logger.rb", "lib/dtr/shared/utils/cmd.rb", "lib/dtr/shared/utils/env_store.rb", "lib/dtr/shared/utils/logger.rb", "lib/dtr/shared/utils.rb", "lib/dtr/shared/working_env.rb", "lib/dtr/shared.rb", "lib/dtr/test_unit/drb_test_runner.rb", "lib/dtr/test_unit/injection.rb", "lib/dtr/test_unit/test_case_injection.rb", "lib/dtr/test_unit/test_suite_injection.rb", "lib/dtr/test_unit/testrunnermediator_injection.rb", "lib/dtr/test_unit/thread_safe_test_result.rb", "lib/dtr/test_unit/worker_club.rb", "lib/dtr/test_unit.rb", "lib/dtr/test_unit_injection.rb", "lib/dtr.rb", "bin/dtr", "CHANGES", "dtr.gemspec", "lib", "LICENSE.TXT", "Rakefile", "README.rdoc", "TODO"]
 
-  spec.test_files = ["test/acceptance/agent_working_env_test.rb", "test/acceptance/dtr_package_task_test.rb", "test/acceptance/general_test.rb", "test/acceptance/raketasks_test.rb", "test/acceptance/sync_codebase_test.rb", "test/acceptance/sync_logger_test.rb", "test/agent_helper.rb", "test/logger_stub.rb", "test/test_helper.rb", "test/unit/adapter_test.rb", "test/unit/configuration_test.rb", "test/unit/facade_test.rb", "test/unit/logger_test.rb", "test/unit/test_unit_test.rb", "test/unit/working_env_test.rb", "testdata/a_failed_test_case.rb", "testdata/a_file_system_test_case.rb", "testdata/a_test_case.rb", "testdata/a_test_case2.rb", "testdata/an_error_test_case.rb", "testdata/another_project", "testdata/another_project/passed_test_case.rb", "testdata/another_project/Rakefile", "testdata/hacked_run_method_test_case.rb", "testdata/is_required_by_a_test.rb", "testdata/lib", "testdata/lib/lib_test_case.rb", "testdata/package_task_test_rakefile", "testdata/Rakefile", "testdata/raketasks", "testdata/raketasks/Rakefile", "testdata/raketasks/success_test_case.rb", "testdata/scenario_test_case.rb", "testdata/setup_agent_env_test_case.rb", "testdata/sleep_3_secs_test_case.rb", "testdata/verify_dir_pwd", "testdata/verify_dir_pwd/Rakefile", "testdata/verify_dir_pwd/verify_dir_pwd_test_case.rb"]
+  spec.test_files = ["test/acceptance/agent_working_env_test.rb", "test/acceptance/dtr_package_task_test.rb", "test/acceptance/general_test.rb", "test/acceptance/rails_ext_test.rb", "test/acceptance/raketasks_test.rb", "test/acceptance/sync_codebase_test.rb", "test/acceptance/sync_logger_test.rb", "test/agent_helper.rb", "test/logger_stub.rb", "test/test_helper.rb", "test/unit/adapter_test.rb", "test/unit/configuration_test.rb", "test/unit/database_initializer_test.rb", "test/unit/facade_test.rb", "test/unit/logger_test.rb", "test/unit/test_unit_test.rb", "test/unit/thread_safe_test_result_test.rb", "test/unit/working_env_test.rb", "testdata/a_failed_test_case.rb", "testdata/a_file_system_test_case.rb", "testdata/a_test_case.rb", "testdata/a_test_case2.rb", "testdata/an_error_test_case.rb", "testdata/another_project", "testdata/another_project/passed_test_case.rb", "testdata/another_project/Rakefile", "testdata/hacked_run_method_test_case.rb", "testdata/is_required_by_a_test.rb", "testdata/lib", "testdata/lib/lib_test_case.rb", "testdata/package_task_test_rakefile", "testdata/rails_projects", "testdata/rails_projects/simple_project", "testdata/rails_projects/simple_project/app", "testdata/rails_projects/simple_project/app/controllers", "testdata/rails_projects/simple_project/app/controllers/application.rb", "testdata/rails_projects/simple_project/app/controllers/products_controller.rb", "testdata/rails_projects/simple_project/app/helpers", "testdata/rails_projects/simple_project/app/helpers/application_helper.rb", "testdata/rails_projects/simple_project/app/helpers/products_helper.rb", "testdata/rails_projects/simple_project/app/models", "testdata/rails_projects/simple_project/app/models/product.rb", "testdata/rails_projects/simple_project/app/views", "testdata/rails_projects/simple_project/app/views/products", "testdata/rails_projects/simple_project/app/views/products/index.rhtml", "testdata/rails_projects/simple_project/config", "testdata/rails_projects/simple_project/config/boot.rb", "testdata/rails_projects/simple_project/config/database.yml", "testdata/rails_projects/simple_project/config/database.yml.mysql", "testdata/rails_projects/simple_project/config/environment.rb", "testdata/rails_projects/simple_project/config/environments", "testdata/rails_projects/simple_project/config/environments/development.rb", "testdata/rails_projects/simple_project/config/environments/production.rb", "testdata/rails_projects/simple_project/config/environments/test.rb", "testdata/rails_projects/simple_project/config/initializers", "testdata/rails_projects/simple_project/config/initializers/inflections.rb", "testdata/rails_projects/simple_project/config/initializers/mime_types.rb", "testdata/rails_projects/simple_project/config/initializers/new_rails_defaults.rb", "testdata/rails_projects/simple_project/config/routes.rb", "testdata/rails_projects/simple_project/db", "testdata/rails_projects/simple_project/db/migrate", "testdata/rails_projects/simple_project/db/migrate/20081027133744_create_products.rb", "testdata/rails_projects/simple_project/db/schema.rb", "testdata/rails_projects/simple_project/doc", "testdata/rails_projects/simple_project/doc/README_FOR_APP", "testdata/rails_projects/simple_project/public", "testdata/rails_projects/simple_project/public/404.html", "testdata/rails_projects/simple_project/public/422.html", "testdata/rails_projects/simple_project/public/500.html", "testdata/rails_projects/simple_project/public/dispatch.cgi", "testdata/rails_projects/simple_project/public/dispatch.fcgi", "testdata/rails_projects/simple_project/public/dispatch.rb", "testdata/rails_projects/simple_project/public/favicon.ico", "testdata/rails_projects/simple_project/public/images", "testdata/rails_projects/simple_project/public/images/rails.png", "testdata/rails_projects/simple_project/public/index.html", "testdata/rails_projects/simple_project/public/javascripts", "testdata/rails_projects/simple_project/public/javascripts/application.js", "testdata/rails_projects/simple_project/public/javascripts/controls.js", "testdata/rails_projects/simple_project/public/javascripts/dragdrop.js", "testdata/rails_projects/simple_project/public/javascripts/effects.js", "testdata/rails_projects/simple_project/public/javascripts/prototype.js", "testdata/rails_projects/simple_project/public/robots.txt", "testdata/rails_projects/simple_project/Rakefile", "testdata/rails_projects/simple_project/README", "testdata/rails_projects/simple_project/script", "testdata/rails_projects/simple_project/script/about", "testdata/rails_projects/simple_project/script/console", "testdata/rails_projects/simple_project/script/dbconsole", "testdata/rails_projects/simple_project/script/destroy", "testdata/rails_projects/simple_project/script/generate", "testdata/rails_projects/simple_project/script/performance", "testdata/rails_projects/simple_project/script/performance/benchmarker", "testdata/rails_projects/simple_project/script/performance/profiler", "testdata/rails_projects/simple_project/script/performance/request", "testdata/rails_projects/simple_project/script/plugin", "testdata/rails_projects/simple_project/script/process", "testdata/rails_projects/simple_project/script/process/inspector", "testdata/rails_projects/simple_project/script/process/reaper", "testdata/rails_projects/simple_project/script/process/spawner", "testdata/rails_projects/simple_project/script/runner", "testdata/rails_projects/simple_project/script/server", "testdata/rails_projects/simple_project/test", "testdata/rails_projects/simple_project/test/fixtures", "testdata/rails_projects/simple_project/test/fixtures/products.yml", "testdata/rails_projects/simple_project/test/functional", "testdata/rails_projects/simple_project/test/functional/products_controller_test.rb", "testdata/rails_projects/simple_project/test/test_helper.rb", "testdata/rails_projects/simple_project/test/unit", "testdata/rails_projects/simple_project/test/unit/product_test.rb", "testdata/Rakefile", "testdata/raketasks", "testdata/raketasks/Rakefile", "testdata/raketasks/success_test_case.rb", "testdata/scenario_test_case.rb", "testdata/setup_agent_env_test_case.rb", "testdata/sleep_3_secs_test_case.rb", "testdata/verify_dir_pwd"]
 
   #### Load-time details: library and application (you will need one or both).
 

data/lib/dtr/agent/brain.rb

@@ -15,42 +15,68 @@
 module DTR
   module Agent
 
-    class Brain
+    class Brain < ProcessRoot
       include Adapter::Follower
       
-      def initialize(runner_names, agent_env_setup_cmd)
-        raise 'No runner? What can I do for you?' if runner_names.blank?
-        @runner_names = runner_names
-        @agent_env_setup_cmd = agent_env_setup_cmd
+      def initialize
+        super
+        raise 'No runner? What can I do for you?' if DTR.configuration.agent_runners.blank?
         DTR.info {""}
         DTR.info {"--------------------beautiful line--------------------------"}
-        DTR.info {"=> Agent environment setup command: #{@agent_env_setup_cmd}"}
-        DTR.info {"=> Runner names: #{@runner_names.join(', ')}"}
-        DTR.info {"=> Broadcast list: #{DTR.configuration.broadcast_list.inspect}"}
+        DTR.info {"=> Agent environment setup command: #{DTR.configuration.agent_env_setup_cmd}"}
+        DTR.info {"=> Runners: #{DTR.configuration.agent_runners.join(', ')}"}
         DTR.info {"=> Listening port: #{DTR.configuration.agent_listen_port}"}
         DTR.info {"=> Group: #{DTR.configuration.group}"}
+        DTR.info {"=> Dir.pwd: #{Dir.pwd}"}
+        DTR.info {"=> Follower listen heartbeat timeout: #{DTR.configuration.follower_listen_heartbeat_timeout}"}
       end
 
       def hypnotize
-        loop do
-          if wakeup?
-            DTR.info {"Agent brain wakes up"}
-            work(DTR.fork_process { Worker.new(@runner_names, @agent_env_setup_cmd).launch })
-            DTR.info {"Agent brain is going to sleep"}
+        with_relaxation do
+          loop do
+            do_once
           end
         end
+      end
+
+      def hypnotize_once
+        with_relaxation do
+          do_once
+        end
+      end
+
+      private
+      def do_once
+        if wakeup?
+          DTR.info {"Agent brain wakes up"}
+
+          if DTR.run_script("DTR::Agent::Herald.new")
+            fang_gou
+          else
+            DTR.info {"=> No runner started."}
+          end
+
+          DTR.info {"Agent brain is going to sleep"}
+        end
+      end
+
+      def with_relaxation
+        yield
       rescue Interrupt, SystemExit, SignalException
+        DTR.info {$!.message}
       ensure
         relax
+        DTR.info {"Agent brain is dieing."}
       end
 
-      def work(worker)
-        until sleep?
-          #keep worker working :D
+      def fang_gou
+        DTR.configuration.runners_should_be_working
+        Worker.new.watch_runners do
+          until sleep?
+            DTR.configuration.runners_should_be_working
+          end
+          DTR.configuration.agent_is_going_to_sleep
         end
-      ensure
-        DTR.info {"Killing worker"}
-        DTR.kill_process worker
       end
     end
   end

data/lib/dtr/agent/herald.rb

@@ -15,18 +15,15 @@
 module DTR
 
   module Agent
-    class Herald
+    class Herald < ProcessRoot
       class WorkingEnvError < StandardError
       end
 
       include Service::WorkingEnv
       include Service::Agent
 
-      def initialize(working_env_key, agent_env_setup_cmd, runners)
-        @working_env_key = working_env_key
-        @agent_env_setup_cmd = agent_env_setup_cmd
-        @runners = runners
-        @env_store = EnvStore.new
+      def initialize
+        super
         start_service
         start_off
       ensure
@@ -34,10 +31,10 @@ module DTR
       end
 
       def start_off
-        DTR.info {"=> Herald starts off..."}
-        provide_agent_info(@agent_env_setup_cmd, @runners)
+        DTR.info {"=> Herald starts off at #{DTR.root}"}
+        provide_agent_info(DTR.configuration.agent_env_setup_cmd, DTR.configuration.agent_runners)
 
-        @env_store[@working_env_key] = fetch_working_env
+        DTR.configuration.working_env = fetch_working_env
       rescue WorkingEnvError
         DTR.error $!.message
         exit(-1)
@@ -52,7 +49,8 @@ module DTR
         DTR.info {"=> Got working environment created at #{working_env[:created_at]} by #{working_env[:host]}"}
 
         raise WorkingEnvError.new("No test files need to load?(working env: #{working_env})") if working_env[:files].blank?
-        raise WorkingEnvError.new('Setup working environment failed, no runner started.') unless working_env.setup_env(@agent_env_setup_cmd)
+
+        working_env.synchronize_for(DTR.configuration.agent_runners)
         working_env
       end
     end

data/lib/dtr/agent/process_root.rb

@@ -0,0 +1,23 @@
+# Copyright (c) 2007-2008 Li Xiao <iam@li-xiao.com>
+# 
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+# 
+#     http://www.apache.org/licenses/LICENSE-2.0
+# 
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+module DTR
+  module Agent
+    class ProcessRoot
+      def initialize(*args)
+        DTR.root = Dir.pwd
+      end
+    end
+  end
+end

data/lib/dtr/agent/rails_ext.rb

@@ -0,0 +1,57 @@
+# Copyright (c) 2007-2008 Li Xiao <iam@li-xiao.com>
+# 
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+# 
+#     http://www.apache.org/licenses/LICENSE-2.0
+# 
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+module DTR
+  module Agent
+    module RailsExt
+      module DatabaseInitializer
+        def preparing_database_command
+          dtr_database_config_exists = File.exist?('config/database.yml.dtr')
+          default_database_config_exists = File.exist?('config/database.yml')
+
+          if !dtr_database_config_exists && !default_database_config_exists
+            DTR.info("No config/database.yml.dtr and config/database.yml exists, bypass database initialization.")
+            return
+          end
+
+          if dtr_database_config_exists
+            DTR.info("Found config/database.yml.dtr, use it as database configuration")
+            FileUtils.cp('config/database.yml.dtr', 'config/database.yml')
+          end
+          DTR.info("Clean databases")
+          Cmd.execute("rake --trace db:drop DTR_RUNNER_NAME=#{ENV['DTR_RUNNER_NAME']}", :error_output_log_level => :debug)
+
+          # Counldn't add --trace here, for Test::Unit detected --trace as a invalid option, don't know why
+          "rake db:create db:migrate db:test:prepare DTR_RUNNER_NAME=#{ENV['DTR_RUNNER_NAME']}"
+        end
+      end
+
+      module WorkingEnvExt
+        include DatabaseInitializer
+
+        def self.included(base)
+          base.alias_method_chain :setup_environment, :preparing_database
+        end
+
+        def setup_environment_with_preparing_database
+          if setup_environment_command.blank? && File.directory?('config')
+            DTR.debug("No setup environment command found but found 'config' directory, try default preparing database command")
+            self[:agent_env_setup_cmd] = preparing_database_command
+          end
+          setup_environment_without_preparing_database
+        end
+      end
+    end
+  end
+end

data/lib/dtr/agent/runner.rb

@@ -14,65 +14,45 @@
 
 module DTR
   module Agent
-    class Runner
+    class Runner < ProcessRoot
       include DRbUndumped
       include Service::Runner
 
-      def self.start(name, env)
-        self.new(name, env).start
-        DRb.thread.join if DRb.thread
-      end
-
-      attr_reader :name, :identifier
+      attr_reader :name
 
-      def initialize(name, env)
+      def initialize(name)
+        super
         @name = name
-        @identifier = env[:identifier]
-        @env = env
       end
       
       def start
         #start service first, so that all logs can be sync with master process
         start_service
-        DTR.info("=> Starting runner #{name} at #{Dir.pwd}, pid: #{Process.pid}")
-        init_environment
-        provide
-        DTR.info {"=> Runner #{name} provided"}
-      rescue Exception
-        DTR.error($!.message)
-        DTR.error($!.backtrace.join("\n"))
-      end
 
-      def init_environment
-        DTR.info {"#{name}: Initialize working environment..."}
         ENV['DTR_RUNNER_NAME'] = name
-
-        @env[:libs].select{ |lib| !$LOAD_PATH.include?(lib) && File.exists?(lib) }.each do |lib|
-          $LOAD_PATH << lib
-          DTR.debug {"#{name}: appended lib: #{lib}"}
-        end
-        DTR.info {"#{name}: libs loaded"}
-        DTR.debug {"#{name}: $LOAD_PATH: #{$LOAD_PATH.inspect}"}
-
-        @env[:files].each do |f|
-          begin
-            load f unless f =~ /^-/
-            DTR.debug {"#{name}: loaded #{f}"}
-          rescue LoadError => e
-            DTR.error {"#{name}: No such file to load -- #{f}"}
-            DTR.debug {"Environment: #{@env}"}
+        DTR.info "Start #{self} at #{DTR.root}, pid: #{Process.pid}"
+        DTR.configuration.working_env.load_environment do
+          provide
+          DTR.info {"=> Runner provided"}
+          while DTR.configuration.runners_should_be_working?
+            sleep(1)
           end
         end
-        DTR.info {"#{name}: test files loaded"}
+      rescue
+        DTR.error($!.message)
+        DTR.error($!.backtrace.join("\n"))
+      ensure
+        #make sure exit process for drb may cause this process hang on
+        exit!(0)
       end
 
       def run(test, result, &progress_block)
-        DTR.debug {"#{name}: running #{test}..."}
+        DTR.debug {"running #{test}..."}
         Agent::TestCase.new(test, result, &progress_block).run
-        DTR.debug {"#{name}: done #{test}"}
+        DTR.debug {"done #{test}"}
       ensure
         provide
-        DTR.debug {"=> Runner #{name} provided"}
+        DTR.debug {"=> Runner provided"}
       end
 
       def provide