tap-test 0.1.0

data/History

@@ -0,0 +1,5 @@
+== 0.1.0 / 2009-05-25
+
+As of the 0.17.0 release, Tap is distributes its test modules independently
+in this gem.  Test modules have been cleaned up and are in many cases not
+compatible with earlier versions distributed with Tap.

data/MIT-LICENSE

@@ -0,0 +1,22 @@
+Copyright (c) 2009, Regents of the University of Colorado.
+
+Permission is hereby granted, free of charge, to any person
+obtaining a copy of this software and associated documentation
+files (the "Software"), to deal in the Software without
+restriction, including without limitation the rights to use,
+copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the
+Software is furnished to do so, subject to the following
+conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
+OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
+HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
+WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+OTHER DEALINGS IN THE SOFTWARE.

data/README

@@ -0,0 +1,123 @@
+= {Tap Test}[http://tap.rubyforge.org/tap-test]
+
+  test v. to reveal the strengths or capabilities of something
+
+Test modules for Tap.
+
+== Description
+
+Provides test modules to simplify testing of common Tap tasks, especially
+tasks that require interaction with files. Modules are also provided to test
+the shell behavior of executables and to define conditions for tests (ex
+*nix/windows only). Tap-Test is not a testing framework. By default Tap-Test
+integrates with Test::Unit and Mini::Test, but it is possible to include the
+test modules into other test frameworks.
+
+{Tap-Test}[http://tap.rubyforge.org/tap-test] is a part of the
+{Tap-Suite}[http://tap.rubyforge.org/tap-suite]. Check out these links for
+documentation, development, and bug tracking.
+
+* Website[http://tap.rubyforge.org] 
+* Lighthouse[http://bahuvrihi.lighthouseapp.com/projects/9908-tap-task-application/tickets]
+* Github[http://github.com/bahuvrihi/tap/tree/master] 
+* {Google Group}[http://groups.google.com/group/ruby-on-tap]
+
+== Usage
+
+The Tap-Test modules are small and targeted. More complete examples can be
+found in the documentation, these are intended to show the point of each
+module. To enable a module in Test::Unit or Mini::Test, require
+'tap/test/unit' and simply call the acts_as_x_test method. Multiple modules
+can be overlaid.
+
+==== {FileTest}[link:classes/Tap/Test/FileTest.html] (acts_as_file_test)
+
+Sets up a test-specific method_root for working with temporary files. Better
+in most cases than using Tempfile because you can flag temporary files to be
+saved on a failure (using ENV['KEEP_OUTPUTS']='true'). Also provides a new
+assertion to test that all the files in an expected directory are equal to
+those in an output directory.
+
+  require 'tap/test/unit'
+  class FileTestTest < Test::Unit::TestCase
+    acts_as_file_test
+
+    def test_method_root
+      assert_equal "test_method_root", File.basename(method_root.root)
+      
+      path = method_root.prepare(:tmp, 'file.txt') {|io| io << 'content' }
+      assert_equal "content", File.read(path)
+    end
+  end
+  
+==== {ShellTest}[link:classes/Tap/Test/ShellTest.html] (acts_as_shell_test)
+
+Simple testing of shell commands.
+
+  require 'tap/test/unit'
+  class ShellTestTest < Test::Unit::TestCase
+    acts_as_shell_test :cmd_pattern => '% alias', :cmd => 'echo'
+
+    def test_echo
+      assert_equal "goodnight moon", sh("echo goodnight moon").strip
+    end
+    
+    def test_echo_with_an_alias
+      sh_test %q{
+  % alias goodnight moon
+  goodnight moon
+  }
+    end
+  end
+    
+==== {SubsetTest}[link:classes/Tap/Test/SubsetTest.html] (acts_as_subset_test)
+
+Allows in-file subsetting of tests into groups. Easy to get carried away with
+this one, but handy, especially for platform-specific tests. Turn on a subset
+or all tests using an ENV variable (ex ENV['ALL']='true').
+
+  require 'tap/test/unit'
+  class SubsetTestTest < Test::Unit::TestCase
+    acts_as_subset_test
+    
+    condition(:windows) { match_platform?('mswin') }
+    
+    def test_something_for_windows_only
+      condition_test(:windows) do
+        # ...
+      end
+    end
+    
+    def test_something_that_takes_forever
+      extended_test do
+        # ...
+      end
+    end
+  end
+
+==== {TapTest}[link:classes/Tap/Test/TapTest.html]  (acts_as_tap_test)
+
+Sets up Tap::App.instance for testing tasks.
+
+  require 'tap/test/unit'
+  class TapTestTest < Test::Unit::TestCase
+    acts_as_tap_test
+
+    def test_task
+      t = Tap::Task.intern {|task| "result" }
+      assert_equal "result", t.process
+    end
+  end
+
+== Installation
+
+Tap-Test is available as a gem on RubyForge[http://rubyforge.org/projects/tap].  Use:
+
+  % gem install tap-test
+  
+== Info
+
+Copyright (c) 2009, Regents of the University of Colorado.
+Developer:: {Simon Chiang}[http://bahuvrihi.wordpress.com], {Biomolecular Structure Program}[http://biomol.uchsc.edu/], {Hansen Lab}[http://hsc-proteomics.uchsc.edu/hansenlab/] 
+Support:: CU Denver School of Medicine Deans Academic Enrichment Fund
+License:: {MIT-Style}[link:files/MIT-LICENSE.html]

data/lib/tap/test/file_test/class_methods.rb

@@ -0,0 +1,17 @@
+module Tap
+  module Test
+    module FileTest
+      
+      # Class methods extending tests which include FileTest.
+      module ClassMethods
+      
+        # The class-level test root (a Tap::Root)
+        attr_accessor :class_test_root
+      
+        # An array of directories to be cleaned up by cleanup
+        attr_accessor :cleanup_dirs
+        
+      end
+    end
+  end
+end

data/lib/tap/test/file_test.rb

@@ -0,0 +1,387 @@
+require 'tap/test/file_test/class_methods'
+require 'tap/test/utils'
+
+module Tap
+  module Test  
+    
+    # FileTest facilitates access and utilization of test-specific files and
+    # directories. FileTest provides each test method with a Tap::Root 
+    # (method_root) specific for the method, and defines a new assertion method 
+    # (assert_files) to facilitate tests which involve the production and/or 
+    # modification of files.
+    #
+    #   [file_test_doc_test.rb]
+    #   class FileTestDocTest < Test::Unit::TestCase
+    #     acts_as_file_test
+    # 
+    #     def test_something
+    #       # each test class has a class test root (ctr)
+    #       # and each test method has a method-specific
+    #       # root (method_root)
+    #
+    #       ctr.root                        # => File.expand_path(__FILE__.chomp('_test.rb'))
+    #       method_root.root                # => File.join(ctr.root, "/test_something")
+    #       method_root[:input]             # => File.join(ctr.root, "/test_something/input")
+    #
+    #       # files in the :output and :tmp directories are cleared
+    #       # before and after each test; this passes each time the
+    #       # test is run with no additional cleanup:
+    #
+    #       assert !File.exists?(method_root[:tmp])
+    #
+    #       tmp_file = method_root.prepare(:tmp, 'sample.txt') {|file| file << "content" }
+    #       assert_equal "content", File.read(tmp_file)
+    #
+    #       # the assert_files method compares files produced
+    #       # by the block the expected files, ensuring they
+    #       # are the same (see the documentation for the 
+    #       # simplest use of assert_files)
+    #       
+    #       expected_file = method_root.prepare(:expected, 'output.txt') {|file| file << 'expected output' }
+    #
+    #       # passes
+    #       assert_files do 
+    #         method_root.prepare(:output, 'output.txt') {|file| file << 'expected output' }
+    #       end 
+    #     end
+    #   end
+    #
+    # FileTest requires that a method_name method is provided by the including
+    # class, in order to properly set the directory for method_root.
+    module FileTest
+      
+      def self.included(base) # :nodoc:
+        super
+        base.extend FileTest::ClassMethods
+        base.cleanup_dirs = [:output, :tmp]
+      end
+      
+      # The test-method-specific Tap::Root which may be used to
+      # access test files.  method_root is a duplicate of ctr
+      # reconfigured so that method_root.root is ctr[method_name.to_sym]
+      attr_reader :method_root
+      
+      # Sets up method_root and calls cleanup.  Be sure to call super when
+      # overriding this method.
+      def setup
+        super
+        @method_root = ctr.dup.reconfigure(:root => ctr[method_name.to_sym])
+        cleanup
+      end
+      
+      # Cleans up the method_root.root directory by removing the class
+      # cleanup_dirs (by default :tmp and :output). The root directory
+      # will also be removed if it is empty.
+      # 
+      # Override as necessary in subclasses.
+      def cleanup
+        self.class.cleanup_dirs.each do |dir|
+          clear_dir(method_root[dir])
+        end
+        try_remove_dir(method_root.root)
+      end
+    
+      # Calls cleanup unless flagged otherwise by an ENV variable. To prevent
+      # cleanup (when debugging for example), set the 'KEEP_OUTPUTS' or 
+      # 'KEEP_FAILURES' ENV variables:
+      #
+      #   % rap test KEEP_OUTPUTS=true
+      #   % rap test KEEP_FAILURES=true
+      #
+      # Cleanup is only suppressed for failing tests when KEEP_FAILURES is
+      # specified.  Be sure to call super when overriding this method.
+      def teardown
+        # check that method_root still exists (nil may
+        # indicate setup was overridden without super)
+        unless method_root
+          raise "teardown failure: method_root is nil (does setup call super?)"
+        end
+        
+        # clear out the output folder if it exists, unless flagged otherwise
+        unless ENV["KEEP_OUTPUTS"] == "true" || (!passed? && ENV["KEEP_FAILURES"] == "true")
+          begin
+            cleanup
+          rescue
+            raise("cleanup failure: #{$!.message}")
+          end
+        end
+        
+        try_remove_dir(ctr.root)
+      end 
+      
+      # Convenience method to access the class_test_root.
+      def ctr
+        self.class.class_test_root or raise "setup failure: no class_test_root has been set for #{self.class}"
+      end
+      
+      # Runs a file-based test that compares files created by the block with
+      # files in an expected directory.  The block receives files from the
+      # input directory, and should return a list of files relative to the 
+      # output directory.  Only the files returned by the block are compared;
+      # additional files in the output directory are effectively ignored.
+      #
+      # === Example
+      # Lets define a test that transforms input files into output files in a
+      # trivial way, simply by replacing 'input' with 'output' in the file.
+      #
+      #   class FileTestDocTest < Test::Unit::TestCase
+      #     acts_as_file_test
+      # 
+      #     def test_assert_files
+      #       assert_files do |input_files|
+      #         input_files.collect do |path|
+      #           input = File.read(path)
+      #           output_file = method_root.path(:output, File.basename(path))
+      #
+      #           File.open(output_file, "w") do |f|
+      #             f << input.gsub(/input/, "output")
+      #           end 
+      #
+      #           output_file
+      #         end
+      #       end
+      #     end
+      #   end
+      #
+      # Now say you had some input and expected files for test_assert_files:
+      #
+      #   file_test_doc/test_assert_files
+      #   |- expected
+      #   |   |- one.txt
+      #   |   `- two.txt
+      #   `- input
+      #       |- one.txt
+      #       `- two.txt
+      # 
+      #   [input/one.txt]
+      #   test input 1
+      #
+      #   [input/two.txt]
+      #   test input 2
+      #
+      #   [expected/one.txt]
+      #   test output 1
+      #
+      #   [expected/two.txt]
+      #   test output 2
+      #
+      # When you run the test, the assert_files passes the input files to the 
+      # block.  When the block completes, assert_files compares the output
+      # files returned by the block with the files in the expected directory.
+      # In this case, the files are equal and the test passes.
+      #
+      # Say you changed the content of one of the expected files:
+      #
+      #   [expected/one.txt]
+      #   test flunk 1
+      #
+      # Now the test fails because the output files aren't equal to the
+      # expected files.  The test also fails if there are missing or extra
+      # files. 
+      #
+      # === Options
+      # A variety of options adjust the behavior of assert_files:
+      # 
+      #   :input_dir                      specify the directory to glob for input files
+      #                                     (default method_root[:input])
+      #   :output_dir                     specify the output directory
+      #                                     (default method_root[:output])
+      #   :expected_dir                   specify the directory to glob for expected files
+      #                                     (default method_root[:expected])
+      #   :input_files                    directly specify the input files for the block
+      #   :expected_files                 directly specify the expected files for comparison
+      #   :include_input_directories      specifies directories to be included in the 
+      #                                     input_files array (by default dirs are excluded)
+      #   :include_expected_directories   specifies directories to be included in the
+      #                                     expected-output file list comparison 
+      #                                     (by default dirs are excluded)  
+      #                  
+      # assert_files will fail if <tt>:expected_files</tt> was not specified
+      # in the options and no files were found in <tt>:expected_dir</tt>.  This
+      # check tries to prevent silent false-positive results when you forget to
+      # put expected files in their place.
+      #
+      # === File References
+      #
+      # Sometimes the same files will get used across multiple tests.  To allow
+      # separate management of test files and prevent duplication, file 
+      # references can be provided in place of test files.  For instance, with a
+      # test directory like:
+      #
+      #   method_root
+      #   |- expected
+      #   |   |- one.txt.ref
+      #   |   `- two.txt.ref
+      #   |- input
+      #   |   |- one.txt.ref
+      #   |   `- two.txt.ref
+      #   `- ref
+      #       |- one.txt
+      #       `- two.txt
+      #   
+      # The input and expected files (all references in this case) can be
+      # dereferenced to the 'ref' paths like so:
+      #
+      #   assert_files :reference_dir => method_root[:ref] do |input_files|
+      #     input_files # => ['method_root/ref/one.txt', 'method_root/ref/two.txt']
+      #
+      #     input_files.collect do |input_file|
+      #       output_file = method_root.path(:output, File.basename(input_file)
+      #       FileUtils.cp(input_file, output_file)
+      #       output_file
+      #     end
+      #   end
+      #
+      # Dereferencing occurs relative to the input_dir/expected_dir
+      # configurations; a reference_dir must be specified for dereferencing to
+      # occur (see Utils.dereference for more details).
+      #
+      # === Keeping Outputs
+      #
+      # By default FileTest cleans up everything under method_root except the
+      # input and expected directories. For ease in debugging, ENV variable
+      # flags can be specified to prevent cleanup for all tests (KEEP_OUTPUTS)
+      # or just tests that fail (KEEP_FAILURES).  These flags can be specified
+      # from the command line if you're running the tests with rake or rap:
+      #
+      #   % rake test keep_outputs=true
+      #   % rap test keep_failures=true
+      #
+      #--
+      # TODO:
+      # * add debugging information to indicate, for instance,  
+      #   when dereferencing is going on.
+      def assert_files(options={}, &block) # :yields: input_files
+        transform_test(block, options) do |expected_file, output_file|
+          unless FileUtils.cmp(expected_file, output_file) 
+            flunk "<#{expected_file}> not equal to\n<#{output_file}>"
+          end
+        end
+      end
+      
+      # The default assert_files options
+      def assert_files_options
+        {
+          :input_dir => method_root[:input],
+          :output_dir => method_root[:output],
+          :expected_dir => method_root[:expected],
+          
+          :input_files => nil,
+          :expected_files => nil,
+          :include_input_directories => false,
+          :include_expected_directories => false,
+          
+          :reference_dir => nil,
+          :reference_extname => '.ref'
+        }
+      end
+      
+      private
+      
+      # Yields to the input block for each pair of entries in the input 
+      # arrays.  An error is raised if the input arrays do not have equal 
+      # numbers of entries.
+      def each_pair(a, b, &block) # :yields: entry_a, entry_b,
+        each_pair_with_index(a,b) do |entry_a, entry_b, index|
+          yield(entry_a, entry_b)
+        end
+      end
+
+      # Same as each_pair but yields the index of the entries as well.
+      def each_pair_with_index(a, b, error_msg=nil, &block) # :yields: entry_a, entry_b, index
+        a = [a] unless a.kind_of?(Array)
+        b = [b] unless b.kind_of?(Array)
+        
+        unless a.length == b.length
+          raise ArgumentError, (error_msg || "The input arrays must have an equal number of entries.")
+        end
+        
+        0.upto(a.length-1) do |index|
+          yield(a[index], b[index], index)
+        end
+      end
+      
+      # Attempts to recursively remove the specified method directory and all 
+      # files within it.  Raises an error if the removal does not succeed.
+      def clear_dir(dir)
+        # clear out the folder if it exists
+        FileUtils.rm_r(dir) if File.exists?(dir)
+      end
+      
+      # Attempts to remove the specified directory.  The root 
+      # will not be removed if the directory does not exist, or
+      # is not empty.  
+      def try_remove_dir(dir)
+        # Remove the directory if possible
+        begin
+          FileUtils.rmdir(dir) if File.exists?(dir) && Dir.glob(File.join(dir, "*")).empty?
+        rescue
+          # rescue cases where there is a hidden file, for example .svn
+        end
+      end
+      
+      def transform_test(block, options={}) # :yields: expected_files, output_files
+        options = assert_files_options.merge(options)
+        input_dir = options[:input_dir]
+        output_dir = options[:output_dir] 
+        expected_dir = options[:expected_dir]
+        
+        reference_dir = options[:reference_dir]
+        reference_pattern = options[:reference_pattern]
+        
+        Utils.dereference([input_dir, expected_dir], reference_dir, reference_pattern || '**/*.ref') do
+
+          # Get the input and expected files in this manner:
+          # - look for manually specified files
+          # - glob for files if none were specified
+          # - expand paths and sort
+          # - remove directories unless specified not to do so
+          input_files, expected_files = [:input, :expected].collect do |key|
+            files = options["#{key}_files".to_sym]
+            if files.nil?
+              pattern = File.join(options["#{key}_dir".to_sym], "**/*")
+              files = Dir.glob(pattern)
+            end
+            files = [files].flatten.collect {|file| File.expand_path(file) }.sort
+
+            unless options["include_#{key}_directories".to_sym]
+              files.delete_if {|file| File.directory?(file)} 
+            end
+          
+            files
+          end
+        
+          # check at least one expected file was found
+          if expected_files.empty? && options[:expected_files] == nil
+            flunk "No expected files specified."
+          end
+        
+          # get output files from the block, expand and sort
+          output_files = [*block.call(input_files)].collect do |output_file| 
+            File.expand_path(output_file)
+          end.sort
+        
+          # check that the expected and output paths are the same
+          translated_expected_files = expected_files.collect do |expected_file|
+            Tap::Root::Utils.translate(expected_file, expected_dir, output_dir)
+          end
+          assert_equal translated_expected_files, output_files, "Missing, extra, or unexpected output files"
+        
+          # check that the expected and output file contents are equal
+          errors = []
+          each_pair(expected_files, output_files) do |expected_file, output_file|
+            unless (File.directory?(expected_file) && File.directory?(output_file)) || FileUtils.cmp(expected_file, output_file)
+              begin
+                yield(expected_file, output_file)
+              rescue
+                errors << $!
+              end
+            end
+          end
+          flunk "File compare failed:\n" + errors.join("\n") unless errors.empty?
+        end
+      end
+      
+    end
+  end
+end

data/lib/tap/test/shell_test/class_methods.rb

@@ -0,0 +1,24 @@
+module Tap
+  module Test
+    module ShellTest
+      # Class methods extending tests which include ShellTest.
+      module ClassMethods
+      
+        # Sets the default sh_test_options
+        attr_writer :sh_test_options
+        
+        # Returns a hash of the default sh_test options.
+        def sh_test_options
+          @sh_test_options ||= {}
+        end
+
+        private
+
+        # helper to retrieve class constants
+        def class_const(const_name) # :nodoc:
+          const_defined?(const_name) ? const_get(const_name) : nil
+        end
+      end
+    end
+  end
+end