test-loop 6.0.0 → 7.0.0

data/README.md

@@ -39,11 +39,14 @@ Features
 
 * Executes test files in parallel, making full use of multiple processors.
 
+* Logs the output from your tests into separate files: one log per test file.
+  The path to a log file is simply the path of its test file plus ".log".
+
 * Generally I/O bound, so you can have it always running without CPU slowdown.
 
 * Configurable through a `.test-loop` file in your current working directory.
 
-* Implemented in less than 100 lines (SLOC) of pure Ruby code! :-)
+* Implemented in less than 150 lines (SLOC) of pure Ruby code! :-)
 
 
 Installation
@@ -82,23 +85,25 @@ Operation
 
 * Press Control-C or send the SIGINT signal to quit the test loop.
 
+* Send the SIGUSR1 signal to terminate workers and their subprocesses.
+
 
 Configuration
 -------------
 
 test-loop looks for a configuration file named `.test-loop` in the current
-working directory.  This configuration file is a normal Ruby script, in which
-you can query and modify the `$test_loop_config` OpenStruct object as follows:
+working directory.  This configuration file is a Ruby script in which you
+can query and modify the `Test::Loop::Config` OpenStruct object as follows:
 
-* `$test_loop_config.overhead_file_globs` is an array of file globbing
+* `Test::Loop::Config.overhead_file_globs` is an array of file globbing
   patterns that describe a set of Ruby scripts that are loaded into the main
   Ruby process as overhead.
 
-* `$test_loop_config.reabsorb_file_globs` is an array of file globbing
+* `Test::Loop::Config.reabsorb_file_globs` is an array of file globbing
   patterns that describe a set of files which cause the overhead to be
   reabsorbed whenever they change.
 
-* `$test_loop_config.test_file_matchers` is a hash that maps a file globbing
+* `Test::Loop::Config.test_file_matchers` is a hash that maps a file globbing
   pattern describing a set of source files to a lambda function yielding a
   file globbing pattern describing a set of test files that need to be run.
   In other words, whenever the source files (the hash key; left-hand side of
@@ -113,51 +118,60 @@ you can query and modify the `$test_loop_config` OpenStruct object as follows:
 
   Then you would add the following to your configuration file:
 
-      $test_loop_config.test_file_matchers['{lib,app}/**/*.rb'] = lambda do |path|
+      Test::Loop::Config.test_file_matchers['{lib,app}/**/*.rb'] = lambda do |path|
         extn = File.extname(path)
         name = File.basename(path, extn)
         "{test,spec}/**/#{name.reverse}#{extn}" # <== notice the reverse()
       end
 
-* `$test_loop_config.test_name_parser` is a lambda function that is passed a
+* `Test::Loop::Config.test_name_parser` is a lambda function that is passed a
   line of source code to determine whether that line can be considered as a
   test definition, in which case it must return the name of the test being
   defined.
 
-* `$test_loop_config.before_each_test` is a lambda function that is executed
+* `Test::Loop::Config.before_each_test` is a lambda function that is executed
   inside the worker process before loading the test file.  It is passed (1)
-  the path to the test file and (2) the names of tests (which were identified
-  by `$test_loop_config.test_name_parser`) inside the test file that have
-  changed since the last time the test file was run.
+  the path to the test file, (2) the path to the log file containing the live
+  output of running the test file, and (3) an array containing the names of
+  tests (which were identified by `Test::Loop::Config.test_name_parser`)
+  inside the test file that have changed since the last run of the test file.
 
   These test names should be passed down to your chosen testing library,
   instructing it to skip all other tests except those passed down to it.  This
   accelerates your test-driven development cycle and improves productivity!
 
-* `$test_loop_config.after_all_tests` is a lambda function that is executed
-  inside the master process after all tests have finished running.  It is
-  passed (1) a list of passing test files along with the exit statuses of the
-  worker processes that ran them, (2) a list of failing test files along with
-  the exit statuses of the worker processes that ran them, (3) the time when
-  test execution began, and (4) how many seconds it took for the overall test
-  execution to complete.
+* `Test::Loop::Config.after_each_test` is a lambda function that is executed
+  inside the master process after a test has finished running.  It is passed
+  (1) the path to the test file, (2) the path to the log file containing the
+  output of running the test file, (3) a `Process::Status` object describing
+  the exit status of the worker process that ran the test file, (4) the time
+  when test execution began, and (5) how many seconds it took for the overall
+  test execution to complete.
 
-  For example, to display a summary of the test execution results as an OSD
-  notification, add the following to your configuration file:
+  For example, to display a summary of the test execution results as an
+  on-screen-display notification while also displaying the log file if the
+  test failed, add the following to your configuration file:
 
-      $test_loop_config.after_all_tests = lambda do |passes, fails, started_at, elapsed_time|
-        success = fails.empty?
+      Test::Loop::Config.after_each_test = lambda do |test_file, log_file, run_status, started_at, elapsed_time|
+        success = run_status.success?
 
-        title = started_at.strftime("#{success ? 'PASS' : 'FAIL'} at %X on %x")
+        title = '%s at %s in %0.1fs' %
+          [success ? 'PASS' : 'FAIL', started_at.strftime('%X'), elapsed_time]
 
-        message = '%d ran, %d passed, %d failed in %0.1f seconds' %
-          [passes.length + fails.length, passes.length, fails.length, elapsed_time]
+        message = test_file
 
-        Thread.new do # launch in background
+        Thread.new do # run in background
           system 'notify-send', '-i', "dialog-#{success ? 'information' : 'error'}", title, message or
           system 'growlnotify', '-a', 'Xcode', '-m', message, title or
           system 'xmessage', '-timeout', '5', '-title', title, message or
           puts title, message
+
+          unless success # show failure log
+            system 'xdg-open', log_file or
+            system 'open', log_file or
+            system 'xmessage', '-title', log_file, '-file', log_file or
+            puts log_file, File.read(log_file)
+          end
         end
       end
 

data/bin/test-loop

@@ -22,151 +22,210 @@
 # OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
 #
 
-process_invocation_vector = [$0, *ARGV].map! {|s| s.dup }
-
 require 'ostruct'
 require 'diff/lcs'
 
-begin
-  notify = lambda {|message| puts "test-loop: #{message}" }
-
-  # supply default configuration
-  config = $test_loop_config = OpenStruct.new
-
-  config.overhead_file_globs = ['{test,spec}/{test,spec}_helper.rb']
+module Test
+  module Loop
+    CONFIG_FILE = File.join(Dir.pwd, '.test-loop')
+
+    Config = OpenStruct.new(
+      :overhead_file_globs => ['{test,spec}/{test,spec}_helper.rb'],
+
+      :reabsorb_file_globs => ['{test,spec}/{test,spec}_helper.rb',
+                               'config/*.{rb,yml}', 'Gemfile.lock',
+                               File.basename(CONFIG_FILE)],
+
+      :test_file_matchers => {
+        # source files that correspond to test files
+        '{lib,app}/**/*.rb' => lambda do |path|
+          extn = File.extname(path)
+          name = File.basename(path, extn)
+          "{test,spec}/**/#{name}_{test,spec}#{extn}"
+        end,
+
+        # the actual test files themselves
+        '{test,spec}/**/*_{test,spec}.rb' => lambda {|path| path }
+      },
+
+      :test_name_parser => lambda do |line|
+        case line
+        when /^\s*def\s+test_(\w+)/ then $1
+        when /^\s*(test|context|should|describe|it)\b.+?(['"])(.*?)\2/ then $3
+        end
+      end,
+
+      :before_each_test => lambda do |test_file, log_file, test_names|
+        unless test_names.empty?
+          test_name_pattern = test_names.map do |name|
+            # sanitize string interpolation and non-method-name characters
+            name.gsub(/\#\{.*?\}/, ' ').strip.gsub(/\W+/, '.*')
+          end.join('|')
+
+          case File.basename(test_file)
+          when /(\b|_)test(\b|_)/ # Test::Unit
+            ARGV.push '--name', "/#{test_name_pattern}/"
+          when /(\b|_)spec(\b|_)/ # RSpec
+            ARGV.push '--example', test_name_pattern
+          end
+        end
+      end,
+
+      :after_each_test =>
+        lambda {|test_file, log_file, run_status, started_at, elapsed_time|}
+    )
+
+    def self.run
+      register_signals
+      load_user_config
+      absorb_overhead
+      run_test_loop
+    rescue Exception => error
+      STDERR.puts error.inspect, error.backtrace
+      sleep 1
+      reload_master_process
+    end
 
-  config.reabsorb_file_globs = config.overhead_file_globs +
-    ['config/*.{rb,yml}', 'Gemfile.lock', '.test-loop']
+    class << self; private
+      SCRIPT_NAME = File.basename($0)
+      EXEC_VECTOR = [$0, *ARGV].map {|s| s.dup.freeze }.freeze
 
-  config.test_file_matchers = {
-    # source files that correspond to test files
-    '{lib,app}/**/*.rb' => lambda do |path|
-      extn = File.extname(path)
-      name = File.basename(path, extn)
-      "{test,spec}/**/#{name}_{test,spec}#{extn}"
-    end,
+      def notify message
+        # using print() because puts() is not an atomic operation
+        print "#{SCRIPT_NAME}: #{message}\n"
+      end
 
-    # the actual test files themselves
-    '{test,spec}/**/*_{test,spec}.rb' => lambda {|path| path }
-  }
+      def register_signals
+        trap(:INT) { destroy_process_group }
+        master_pid = $$ # kill only the workers, not the master
+        trap(:USR1) { destroy_process_group unless $$ == master_pid }
+        trap(:QUIT) { reload_master_process }
+        trap(:TSTP) {} # ignore until ready for testing in run_test_loop()
+      end
 
-  config.test_name_parser = lambda do |line|
-    case line
-    when /^\s*def\s+test_(\w+)/ then $1
-    when /^\s*(test|context|should|describe|it)\b.+?(['"])(.*?)\2/ then $3
-    end
-  end
+      def destroy_process_group
+        Process.kill :KILL, -$$
+      end
 
-  config.before_each_test = lambda do |test_file, test_names|
-    unless test_names.empty?
-      test_name_pattern = test_names.map do |name|
-        # sanitize string interpolation and non-method-name characters
-        name.gsub(/\#\{.*?\}/, ' ').strip.gsub(/\W+/, '.*')
-      end.join('|')
-
-      case File.basename(test_file)
-      when /(\b|_)test(\b|_)/ # Test::Unit
-        ARGV.push '--name', "/#{test_name_pattern}/"
-      when /(\b|_)spec(\b|_)/ # RSpec
-        ARGV.push '--example', test_name_pattern
+      def reload_master_process
+        notify 'Restarting loop...'
+        Process.kill :USR1, -$$
+        exec(*EXEC_VECTOR)
       end
-    end
-  end
 
-  config.after_all_tests = lambda {|passes, fails, started_at, elapsed_time|}
+      def load_user_config
+        if File.exist? CONFIG_FILE
+          notify 'Loading configuration...'
+          load CONFIG_FILE
+        end
+      end
 
-  # load user's configuration overrides
-  if File.exist? config_file = File.join(Dir.pwd, '.test-loop')
-    notify.call 'Loading configuration...'
-    load config_file
-  end
+      def absorb_overhead
+        notify 'Absorbing overhead...'
+        $LOAD_PATH.unshift 'lib', 'test', 'spec'
+        Dir[*Config.overhead_file_globs].each do |file|
+          require File.basename(file, File.extname(file))
+        end
+      end
 
-  # absorb test execution overhead into master process
-  $LOAD_PATH.unshift 'lib', 'test', 'spec'
+      def run_test_loop
+        @running_files = []
+        @running_files_lock = Mutex.new
+
+        @lines_by_file = {} # path => readlines
+        @last_ran_at = @started_at = Time.now
+        trap(:TSTP) { @last_ran_at = Time.at(0); @lines_by_file.clear }
+
+        notify 'Ready for testing!'
+        loop do
+          # figure out what test files need to be run
+          test_files = Config.test_file_matchers.map \
+          do |source_glob, test_matcher|
+            Dir[source_glob].select {|file| File.mtime(file) > @last_ran_at }.
+            map {|path| Dir[test_matcher.call path] }
+          end.flatten.uniq
+
+          test_files = @running_files_lock.
+            synchronize { test_files - @running_files }
+
+          # fork worker processes to run the test files in parallel
+          @last_ran_at = Time.now
+          test_files.each {|f| run_test_file f }
+
+          # reabsorb test execution overhead as necessary
+          if Dir[*Config.reabsorb_file_globs].
+             any? {|file| File.mtime(file) > @started_at }
+          then
+            Process.kill :QUIT, $$
+          end
+
+          sleep 1
+        end
+      end
 
-  notify.call 'Absorbing overhead...'
-  Dir[*config.overhead_file_globs].each do |file|
-    require File.basename(file, File.extname(file))
-  end
+      def run_test_file test_file
+        @running_files_lock.synchronize { @running_files.push test_file }
+        log_file = test_file + '.log'
 
-  # continuously watch for and test changed code
-  test_file_cache = {} # path => readlines
-  started_at = last_ran_at = Time.now
-  trap(:QUIT) { started_at = Time.at(0) }
-  trap(:TSTP) { last_ran_at = Time.at(0); test_file_cache.clear }
-
-  notify.call 'Ready for testing!'
-  loop do
-    # figure out what test files need to be run
-    test_files = config.test_file_matchers.map do |source_glob, test_matcher|
-      Dir[source_glob].select {|file| File.mtime(file) > last_ran_at }.
-      map {|path| Dir[test_matcher.call path] }
-    end.flatten.uniq
-
-    # fork worker processes to run the test files in parallel
-    unless test_files.empty?
-      notify.call 'Running tests...'
-      last_ran_at = Time.now
-
-      test_files.each do |test_file|
         # cache the contents of the test file for diffing below
         new_lines = File.readlines(test_file)
-        old_lines = test_file_cache[test_file] || new_lines
-        test_file_cache[test_file] = new_lines
+        old_lines = @lines_by_file[test_file] || new_lines
+        @lines_by_file[test_file] = new_lines
+
+        worker_pid = fork do
+          # capture test output in log file because tests are run in parallel
+          # which makes it difficult to understand interleaved output thereof
+          $stdout.reopen(log_file, 'w').sync = true
+          $stderr.reopen($stdout)
 
-        fork do
           # determine which test blocks have changed inside the test file
-          test_names = Diff::LCS.diff(old_lines, new_lines).flatten.map do |change|
+          test_names = Diff::LCS.diff(old_lines, new_lines).flatten.map \
+          do |change|
             catch :found do
               # search backwards from the line that changed up to
               # the first line in the file for test definitions
               change.position.downto(0) do |i|
-                if test_name = config.test_name_parser.call(new_lines[i])
+                if test_name = Config.test_name_parser.call(new_lines[i])
                   throw :found, test_name
                 end
               end; nil # prevent unsuccessful search from returning an integer
             end
           end.compact.uniq
 
-          config.before_each_test.call test_file, test_names
+          # tell the testing framework to run only the changed test blocks
+          Config.before_each_test.call test_file, log_file, test_names
 
-          load test_file
-
-          # at this point, the at_exit() hook of the testing framework used by
-          # the user's test suite will take care of running all tests defined
-          # by the test file loaded above and will also reflect any failures
-          # in the worker's exit status
+          # after loading the user's test file, the at_exit() hook of the
+          # user's testing framework will take care of running the tests and
+          # reflecting any failures in the worker process' exit status
+          load $0 = test_file # set $0 because Test::Unit outputs it
         end
-      end
 
-      # wait for worker processes to finish and report results
-      test_runs = Process.waitall.map {|pid, status| status }
-      elapsed_time = Time.now - last_ran_at
+        # monitor and report on the worker's progress
+        Thread.new do
+          report = lambda do |state|
+            notify [state, worker_pid, test_file].join("\t")
+          end
+          report.call :RUN
 
-      passes, fails = test_files.zip(test_runs).each do |file, run|
-        notify.call "#{run.success? ? 'PASS' : 'FAIL'} #{file}"
-      end.partition {|file, run| run.success? }
+          # wait for worker to finish
+          begin
+            Process.waitpid worker_pid
+          rescue Errno::ECHILD
+            # worker finished and the OS has forgotten about it already
+          end
+          elapsed_time = Time.now - @last_ran_at
 
-      notify.call '%d ran, %d passed, %d failed in %0.1f seconds' %
-        [test_runs.length, passes.length, fails.length, elapsed_time]
+          report.call $?.success? && :PASS || :FAIL
 
-      config.after_all_tests.call passes, fails, last_ran_at, elapsed_time
-    end
+          Config.after_each_test.call \
+            test_file, log_file, $?, @last_ran_at, elapsed_time
 
-    # reabsorb test execution overhead as necessary
-    if Dir[*config.reabsorb_file_globs].any? {|file| File.mtime(file) > started_at }
-      notify.call 'Restarting loop...'
-      exec(*process_invocation_vector)
+          @running_files_lock.synchronize { @running_files.delete test_file }
+        end
+      end
     end
-
-    sleep 1
   end
-
-rescue Interrupt
-  # user wants to quit the loop so terminate all worker processes
-  Process.kill :SIGKILL, -$$ # negative PID propagates signal to children
-
-rescue Exception => error
-  STDERR.puts error.inspect, error.backtrace
-  sleep 1 and exec(*process_invocation_vector)
 end
+
+Test::Loop.run

metadata

@@ -3,10 +3,10 @@ name: test-loop
 version: !ruby/object:Gem::Version 
   prerelease: false
   segments: 
-  - 6
+  - 7
   - 0
   - 0
-  version: 6.0.0
+  version: 7.0.0
 platform: ruby
 authors: 
 - Suraj N. Kurapati
@@ -14,7 +14,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2011-02-09 00:00:00 -08:00
+date: 2011-02-10 00:00:00 -08:00
 default_executable: 
 dependencies: 
 - !ruby/object:Gem::Dependency