baretest 0.2.4 → 0.4.0.pre1

data/doc/news/news-0.3.0.rdoc

@@ -0,0 +1,7 @@
+NEWS
+
+For baretest version 0.3.0
+* Setup-variations
+* skip-method and Skip < StandardError
+* Persist data (independently of the test directory) over multiple runs
+* Formatter specific options

data/doc/quickref.rdoc

@@ -14,8 +14,10 @@ Also look into the examples and baretests own test directory.
     1. git clone git://github.com/apeiros/baretest.git
     2. cd baretest
     3. rake gem:install
+    If 3. fails, try deleting the MANIFEST.txt and do `rake manifest:create`.
+    Then try step 3 again. Also you must have nokogiri and rdoc >=2.3 installed.
 2. Change into the project directory
-3. `baretest --init` to create the basic test-directory layout
+3. `baretest init` to create the basic test-directory layout
 
 
 
@@ -25,11 +27,17 @@ Your tests for PROJECT/lib/foo.rb belong into PROJECT/test/suite/lib/foo.rb.
 Your tests for PROJECT/bin/bar belong into PROJECT/test/suite/bin/bar.
 In other words, for every path, insert /test/suite after PROJECT to get the
 path to the corresponding testfile.
+Besides 'suite', baretest also recognizes 'integration', 'unit', and 'system'
+as directories. You can use these to separate different concerns of your suite.
 
 
 
 == Writing tests
 
+One way to to learn baretest is to simply look at the examples in
+'examples/tests'. They should provide you with enough information for a quick
+start into writing tests using baretest.
+
 A testfile commonly looks like this:
 
   BareTest.suite "module ModuleName" do
@@ -73,7 +81,7 @@ A testfile commonly looks like this:
   end
 
 This layout makes it easy to figure where tests for something are, and thus
-makes maintaining the test-code base easier.
+makes maintaining the testcodebase easier.
 
 Setup callbacks are invoked from outermost suite to innermost suite, and
 within the same suite, in the order of definition.
@@ -118,9 +126,40 @@ Running this suite will print:
 
 
 
-== Skipping Tests and Suites
+== Skipping and ignoring Suites and Assertions
+
+A suite is pending if it either has no block or a block which contains no
+assertions and suites.
+
+You can skip a suite by:
+* Creating the suite with a :skip option, like
+  `suite "MySuite", :skip => "I want to skip this suite" do`
+* Calling skip in the suite, like
+  suite "MySuite" do
+    skip "I want to skip this suite"
+  end
+  You can call skip at any point in the suite.
+
+You can ignore a suite by:
+* Adding one or more tags to the suite, like
+  `suite "MySuite", :tags => [:set_x, :set_y]`
+  And then specify any of those tags to be ignored, like:
+  `baretest -- -:set_x`
+
+An assertion is pending if it has no block.
+
+You can skip an assertion by:
+* Having it in a suite that is skipped
+* Creating the assertion with a :skip option, like
+  `assert "Something", :skip => "I want to skip this assertion" do`
+* Calling skip in the assertion, like
+  assert "Something" do
+    skip "I want to skip this assertion"
+  end
+  You can call skip at any point in the assertion.
 
-A test is skipped if it does not have a block or calls 'skip'
+You can ignore an assertion by:
+* Having it in a suite that is ignored
 
 == Assertion helper methods
 
@@ -226,13 +265,15 @@ can alternatively be used with named arguments:
 
 baretest's test-cycle is:
 
-1. Create Run instance and the toplevel suite
-2. Load everything as required by the command line flags
-3. Load PROJECT/test/setup.rb
-4. Find PROJECT/test/suite/**/*.rb
-5. Load each file found in 2., but for every file, see whether
-   PROJECT/test/helpers/suite/**/*.rb exists and load that first if it does
-6. Invoke run on the Run instance
+1. Create the toplevel suite
+2. Load PROJECT/test/setup.rb
+3. Find every file as required by the command line flags (defaults to
+   PROJECT/test/{suite,unit,integration,system}/**/*.rb)
+4. Load each file found in 3., but for every file, see whether
+   PROJECT/test/helpers/**/*.rb exists and load that first if it does
+   (e.g. for PROJECT/test/suite/foo.rb load PROJECT/test/helpers/suite/foo.rb)
+5. Create a BareTest::Run instance with the toplevel suite and the passed options
+6. Invoke run_all on that Run instance.
 
 From there on it depends on the loaded formatters and extenders, what really
 will happen. But the norm is, that suites and assertions will be executed in
@@ -242,20 +283,25 @@ order of definition.
 
 == Debugging tests
 
-Use `baretest -i` to run baretest in interactive mode. When a failure or an
-error occurs, you have access to the following commands:
-
-* s!          - original assertions' status
-* e!          - error message and full backtrace
-* em!         - error message
-* bt!         - full backtrace
-* iv!         - all available instance variables
-* cv!         - all available class variables
-* gv!         - all available global variables
-* file        - file this assertion was defined in
-* line        - line number in the file where this assertion's definition starts
-* nesting     - a '>'-separated list of suite descriptions this assertion is
-                nested in
-* description - this assertion's description
-* code        - code of this assertion
-* help        - overview over all the commands
+Use `baretest -i` to run baretest in interactive mode. Try it with
+`baretest -i examples/irb_mode/failures.rb`
+When a failure or an error occurs, you have access to the following commands:
+
+help!::        overview over all the commands
+s!::           the assertions' original status
+sc!::          the assertions' original status code
+e!::           prints the error message and full backtrace
+em!::          prints the error message
+bt!::          prints the full backtrace
+lv!::          lists all available local variables
+iv!::          lists all available instance variables
+cv!::          lists all available class variables
+gv!::          lists all available global variables, per default dropping rubys
+               standard globals (use gv!(false) to avoid that)
+file!::        the file this assertion was defined in
+line!::        the line number in the file where this assertion's definition
+               starts
+nesting!::     a '>'-separated list of suite descriptions this assertion is
+               nested in
+description!:: this assertion's description
+code!::        code of this assertion

data/doc/whats_going_on.rdoc

@@ -1,3 +1,8 @@
+= What's going on
+
+This document intends to show you what happens behind the scene and help you
+understand the mechanics of baretest.
+
 With the following test definition:
   BareTest.suite "A"
     setup do

data/doc/writing_tests.rdoc

@@ -25,7 +25,7 @@ The standard directory layout looks like this:
   |-- rake (contains rake relevant stuff)
   `-- Rakefile
 
-In your project directory, you can invoke `baretest --init`, this will
+In your project directory, you can invoke `baretest init`, this will
 create the 'test' directory. It will mirror your project directory. That is,
 it will recreate all directories nested in bin and lib within test/suite.
 The directory layout of 'test' is as follows:
@@ -118,31 +118,43 @@ In this simplistic example, this may seem like wasted time. The more complex the
 setup becomes and the more assertions need the same setup, the more time a
 separate setup phase saves.
 It additionally helps in making intent clear: this is setup, and this is test.
+The setup and teardown is run (via instance_eval) in the same context as the
+assertion. Local variables however are NOT shared. To pass data from the setup
+to the assertion, use instance variables as shown here.
+
 
 
 == 5. When troubles strike
 
 If one of your assertions fails or errors, you can use `baretest -i` to
-investigate the issue. It will throw you into an irb session, with
-self being the failing/erroring assertion and with several helpful
-methods (use `help` in the irb session to get a list of those).
+investigate the issue. It will throw you into an irb session, with self being
+the failing/erroring assertion context and with several helpful methods (use
+`help` in the irb session to get a list of those).
 
 == Things left to be written out
+
+This is not part of the tutorial but things the author of this tutorial should
+still add to the tutorial itself.
+
 * toplevel suite may have a name/description too, it'll act the same as if
   there was a suite in an unnamed toplevel suite
 * [setup] They will also be run for every nested suite's assertion,
   where the outermost setup is run first, the innermost last.
 * using stubs & mocks
-* However, suites with the same name are considered the same.
+* Suites with the same name and nesting are considered the same and become
+  merged
   
-  For example, this code:
+  Example:
   
-    BareTest.suite "class Foo" do
-      suite "class Bar" do
-        assert "foo"
+    BareTest.suite "Foo" do
+      suite "Bar" do
+        assert "x"
+      end
+    end
+    BareTest.suite "Foo" do
+      suite "Bar" do
+        assert "y"
+      end
     end
   
-  
-  Just like ruby's
-  namespacing works. That is, if you twice do `module X; class Y; ...; end; end`,
-  it will both times open the same class X::Y.
+  Now the suite "Foo" > "Bar" has two assertions, "x" and "y"

data/examples/components/rack-test.rb

@@ -0,0 +1,17 @@
+MiniApp = proc { |*a| [200, {'Content-Type' => 'text/plain'}, "hello world"] }
+
+BareTest.suite "Rack-Test", :use => :rack_test do
+  setup do
+    @app = MiniApp
+  end
+
+  assert "Requesting '/' is 200 OK" do
+    get '/'
+    last_response.ok?
+  end
+
+  assert "Requesting / gets the body 'hello world'" do
+    get '/'
+    equal 'hello world', last_response.body
+  end
+end

data/examples/rake/test.rake

@@ -0,0 +1,40 @@
+#--
+# Copyright 2009-2010 by Stefan Rusterholz.
+# All rights reserved.
+# See LICENSE.txt for permissions.
+#++
+
+
+
+# This rake task collects some ENV variables and then delegates to duty to
+# BareTest::CommandLine#run
+
+namespace :test do
+  desc "Information about how your test directory should look."
+  task :structure do
+    project_dir = File.expand_path(Dir.getwd)
+
+    puts "rake test:run expects to the directory test (#{File.expand_path(project_dir)}/test) to exist."
+  end
+
+  desc "Run testsuite. Set FORMAT env variable to change the formatter used, INTERACTIVE to 'true' to have irb mode."
+  task :run do
+    begin
+      require 'baretest'
+    rescue LoadError => e
+      puts "Could not run tests: #{e}"
+    else
+      # Options can only be supplied via ENV
+      options     = {
+        :format      => (ENV["FORMAT"] || 'cli'),
+        :interactive => (ENV["INTERACTIVE"] =~ /^[ty]/i), # true, TRUE, yes, YES or abbreviated
+      }
+
+      # Run all tests
+      BareTest::CommandLine.run([], options)
+    end
+  end
+end
+
+desc 'Alias for test:run'
+task :test => 'test:run'

data/examples/tests/01_basics_I.rb

@@ -0,0 +1,34 @@
+# To start the test definition you do `BareTest.suite do ...`, see
+# BareTest::Suite::new for more
+BareTest.suite do
+
+  # The individual tests can be grouped into suites
+  suite "Success" do
+
+    # A test is defined via
+    #   assert "description of what we assert" do
+    #     (the assertion itself)
+    #   end
+    # Where the return value (and/or whether it raises or throws something)
+    # defines its status
+    assert "Returning a trueish value (non nil/false) is a success" do
+      true
+    end
+  end
+
+  suite "Failure" do
+    assert "Returning a falsish value (nil/false) is a failure" do
+      false
+    end
+  end
+
+  suite "Pending" do
+    assert "Without a block is pending"
+  end
+
+  suite "Error" do
+    assert "Uncaught exceptions are an error" do
+      raise "Error!"
+    end
+  end
+end

data/examples/tests/02_basics_II_helpers.rb

@@ -0,0 +1,25 @@
+BareTest.suite do
+  assert "The given block raises" do
+    raises do
+      raise "If this raises then the assertion is a success"
+    end
+  end
+
+  assert "The given block raises a specific exception" do
+    raises ArgumentError do # if you want to use {} instead of do/end, you must use parens: raises(ArgumentError) { ... }
+      raise ArgumentError, "If this raises then the assertion is a success"
+    end
+  end
+
+  assert "Assert a float to be close to another" do
+    a = 0.18 - 0.01
+    b = 0.17
+    within_delta a, b, 0.001 # using a == b would be false, because a == 0.169... and b == 1.70...
+  end
+
+  assert "Assert two randomly ordered arrays to contain the same values" do
+    a = [*"A".."Z"] # an array with values from A to Z
+    b = a.sort_by { rand }
+    equal_unordered(a, b) # can be used with any Enumerable, uses hash-key identity
+  end
+end

data/examples/tests/03_basics_III_setup_and_teardown.rb

@@ -0,0 +1,53 @@
+BareTest.suite do
+  suite "Setup & Teardown" do
+    # All setup blocks are executed, so adding another setup block will not
+    # replace an existing one. They are executed in order of definition and
+    # before *each* execution of an assert.
+    setup do
+      @foo = "foo"
+    end
+
+    assert "@foo should be set" do
+      equal("foo", @foo)
+    end
+
+    # All teardown blocks are executed, so adding another teardown block will
+    # not replace an existing one. They are executed in order of definition and
+    # after *each* execution of an assert.
+    teardown do
+      @foo = nil # setting an instance variable to nil isn't really necessary,
+                 # but f.ex. closing an open file handle or similar is a good
+                 # idea.
+    end
+  end
+
+  suite "Nested Setup & Teardown" do
+    setup do
+      @outer_setup = "outer foo"
+      @bar         = "outer bar"
+    end
+
+    suite "Nested suite" do
+      setup do
+        @inner_setup = "inner foo"
+        @bar         = "inner bar"
+      end
+
+      assert "@outer_setup is inherited" do
+        equal("outer foo", @outer_setup)
+      end
+
+      assert "@inner_setup is defined" do
+        equal("inner foo", @inner_setup)
+      end
+
+      assert "@bar is overridden" do
+        equal(@bar, "inner bar")
+      end
+    end
+
+    assert "@inner_setup is not defined in outer suite" do
+      !defined?(@inner_setup)
+    end
+  end
+end

data/examples/tests/04_advanced_I_dependencies.rb

@@ -0,0 +1,31 @@
+BareTest.suite do
+  suite "Dependencies" do
+    suite "Existing dependency" do
+      suite "A", :provides => :a do
+        assert "This assertion will succeed, due to that, :a will be provided" do
+          true
+        end
+      end
+
+      suite "B, depending on A", :depends_on => :a do
+        assert "This assertion will succeed, because the dependency ':a' is provided" do
+          true
+        end
+      end
+    end
+
+    suite "Missing dependency" do
+      suite "C", :provides => :c do
+        assert "This assertion will fail, due to that, :c will NOT be provided" do
+          failure 'Intentional failure'
+        end
+      end
+
+      suite "D, depending on C", :depends_on => :c do
+        assert "This assertion will be skipped, because the dependency ':c' is not provided" do
+          true
+        end
+      end
+    end
+  end
+end

data/examples/tests/05_advanced_II_tags.rb

@@ -0,0 +1,12 @@
+BareTest.suite do
+  suite "focus", :tags => :focus do
+    assert "a" do
+      true
+    end
+  end
+  suite "hocus", :tags => :hocus do
+    assert "b" do
+      true
+    end
+  end
+end