attest 0.1.0 → 0.2.0

data/doodle.txt

@@ -93,50 +93,94 @@ most basic
 - don't print summaries after every context, only a totals summary at the end DONE
 - make it so that tests can take an options hash and one of the options should be to not run setup or teardown for the test, perhaps make it not part of the opts hash DONE
 - should be able to specify a test without a body for it to appear in the output as pending DONE
-- refactor all example code so it is a bit nicer and more representative
-- once all the above are done, create some RDOC documentation to describe the basic features and usage of the framework
-- also fix up the description and extended description, make sure everything looks proper as far as doco is concerned on github etc.
-- if everything is ok at this point, do a first public release of the gem (give it an appropriately betaish version number)
-- make sure the gem is available through rubyforge as well as wherever else is appropriate, e.g. gemcutter
+- refactor all example code so it is a bit nicer and more representative DONE
+- once all the above are done, create some RDOC documentation to describe the basic features and usage of the framework DONE
+- also fix up the description and extended description, make sure everything looks proper as far as doco is concerned on github etc. DONE
+- if everything is ok at this point, do a first public release of the gem (give it an appropriately betaish version number) DONE
+- make sure the gem is available through rubyforge as well as wherever else is appropriate, e.g. gemcutter DONE
 
 
-- when committing need to make sure there is a valid gemspec
-- when used with bundler dependencies like trollop don't come trhough so can't execute attest properly get an error instead need to detect if being used via bundler and do a bundle setup etc.
-
 next release
 
-- integrate mocha so that can mock using mocha 
-- output calculations of expectations not just tests
-- allow expectations to be specified directly as tests (perhaps failures should contain the block that has failed)
-- create a failures only output writer and allow it to be configured
-- create a test/unit style output writer and allow it to be configured
-- test out with method missing defined on the class, does it still work ok
-- make sure if method missing is defined it is tried out first before the new_method_missing on kernel takes over
-- should be able to disable a test so that it is not executed, but is listed as disabled in the output
+$ test out with method missing defined on the class, does it still work ok, essentially call the original method missing first, catch appropriate exception out of there and then run my method missing code and if still no good then propagate the original exception DONE
+$ make sure if method missing is defined it is tried out first before the new_method_missing on kernel takes over (this is very similar to the one above), need a test here
+$ integrate mocha so that can mock using mocha, this should be relatively straight forward DONE
+$ mocha should be set up as a configurable parameter even though it will be the only test double capability available DONE
+$ try out using mocha within the framework on another project totally separate, to nut out how it should be not made a runtime dependency but can be used when pulled in to the other project as a development dependency or a default dependency, also if not using bundler at all and it is not installed, what happens DONE
+$ refactor how mocha gets included into the project  DONE
+$ make sure the binary is only used to read the command line and calls down to other methods to actually load the test files and begin execution of all the tests including outputting the summary at the end DONE
+$ output calculations of expectations not just tests, relatively straight forward and is a good idea DONE
+$ allow expectations to be specified directly as tests (perhaps failures should contain the block that has failed), the test passes and fails and the floating expectation passes and fails will probably need to be reported separately DONE
+$ if we get a test failure, perhaps more of a message regarding where/why the failure occured i.e. which expectation actually failed DONE
+$ create a failures only output writer DONE
+$ create a test/unit style output writer the one with the dots DONE
+$ allow the different types of output writer to be configured as the one to use via command line parameter DONE
+$ should be able to disable a test so that it is not executed, but is listed as disabled in the output DONE
+$ refactor the command line handling to split out actual cli stuff from the attest code that acts on the cli params DONE
+$ should be able to configure attest to be used without any test double framework at all DONE
+$ create a better examples directory more consistent test naming etc DONE
+$ combine assertion reporting and test reporting, we only really care about assertion reporting, no need to print out separate line for both DONE
+$ ability to provide multiple directories, files from which to pull the possible tests DONE
+$ ability to provide multiple directories, file which should be excluded from possible tests DONE
+$ need to refactor my execution context for less duplication and more consistency DONE
+$ rename before_all to be before each DONE
+$ need a before_each and a before_all which gets executed before all the test in the group, currently have a before_all which really acts like a before_each not good DONE
+$ possible bug or unexpected feature, if test contains no assertions in the output then it is treated by the output writer as having an implicit assertion/expectation (a success expectation) this is ok I think DONE
+$ currently unable to specify expectations in before_all and after_all and have them be picked up, this should work, expectations should belong to the test container rather than to any specific test object, on second thoughts maybe shouldn't have expectations on before and after all DONE
+$ should be able to specify expectations on the before_each and after_each DONE
+$ more different types of expectations, should_equal(5){5}, should_not_equal, should_be_same(obj), should_be_same_as, should_not_be_same, should_not_be_same_as, should_not_fail, should_pass, should_be_around(6){5}.plus_minus(1), should_be_a/an(some_lambda){"hello"}, should_match/should_match_a/should_match_an, more thought regarding how to specify a custom matcher, something for arrays and hashes, should_include(some_array), perhaps ability to do blocks and do without blocks i.e. should_include(5, some_array) DONE
+$ refactor expectations so that they can be used without a block but simply with parameters DONE
+$ need to refactor my output writers to extract all common functionality so that there isn't so much code duplication (perhaps using modules or something) DONE
+$ create a rake task that can be easily configured into a rake file with all the relevant configuration values so that attest can be run through rake instead of just the command line DONE
+$ timing for how long it took to complete the whole test suite DONE
+$ better failure reporting DONE
+$ modify the readme, mention the problems wil inline test definition, mention the new expectations, mentiont about the new output writers, mention how output reporting works, disabled tests, test double framework configuration, inclusion and exclusion, before_all and before_each, mention the rake task DONE
+
+
+$ BUG: I think I am now counting expectations passes rather than test passes, since i am getting ran 6 test 7 successful for mocka tests, need to look into this further DONE
+
+$ POC perhaps hold off until next release, ability to actually convert a block to a string to be able to provide more information regarding what actually failed, essentially will need proc.to_s code DONE
+$ POC remove trollop as a dependency to minimize dependecies, either that, or pull the trollop code in directly to minimize on dependecies, see if it's possible to easily acomplish this with git, if not then just manual DONE
+- POC once test double functionality is in, see what we can do about writing some tests for this framework using itself, even a minimal test suite will be better than nothing
+$ POC rake task that can easily be configured in a project to configure all the moving parts and run attest via rake (e.g. rake attest) DONE
+- POC expectations on objects themselves, see if this can be done in such a way as to make it possible to include that bit separately via a separate require or command line paramter or rake config or all of the above
+- POC hook in autotest somehow
+- POC write some smarts about requiring things maybe even do this as a separate gem similar to require_all but with even more smarts and potential configurations call it require_bot
+- POC own test double framework first only for ability to simply provide stubs, also ability to do mocks that verify expected behaviour, possibly two modes, one that complains when unexpected methods are called within test methods, the other mode is the opposite, is fine with unexpected methods, but complains when expected methods are not called, mock and stub class and instance methods, mock on just one object or all objects of the type, probably should be its own gem as well
+$ POC support to allow more traditional i.e. non-inline creating of test in which case perhaps the if env block should not be necessary, tests will live in an attest directory, whatever once again should work via executable as well as rake task, all the smarts that we need to get this to happen DONE
 
 next release 2
 
+- figure out if mocha works properly when using bundler, my guess it will not
+- need to be able to define helper methods within tests somehow that can be called from within tests and setups etc.
+- timing for each test as well as each container should be recorded even if not outputted
+- refactor the duplication between the attest executable in bin and the attest rake task
+- figure out a better way to read/execute tests if they are defined inline
+- as part of the spec test suite create a set of blackbox tests that replicate the functionality in the examples directory to test that actual functionality is working correctly, perhaps create some sort of testing output writer and use that for asserting stuff possibly, these will esssentially be acceptance tests
+- create a better set of unit tests that are not as black box
+- perhaps keep track of timings for how long it took to complete each of the test containers as well as each of the individual tests so that can track stats for test execution speed
+- still more matchers, should_be_around(6, :plus_minus => 1){5} that should work for floats also, maybe give ability to do predicate matchers, i.e if objects responds to a question mark method like include? should be able to do should_include 5, some_array, also think about a good way to create more flexible matchers, more flexible than lambda based ones so that can supply parameters to matchers
 - write and integrate own test double framework (look at rr for syntax suggestions)
 - create some tests using the framework itself around the framework classes (minimal test suite)
 - allow expectations to be made on objects themselves
 - allow for tests to be given ids as well as descriptions
 - allow for multiple before and after blocks should specify which test to run this block for using ids of tests as parameters
+- think of even more different types of expectations
+- specify your own expectations
+- some way to share the expectations that you have created yourself amongs all the different tests in different files
+- pull in tests from other files into current file easily
+- ability to run tests in parallel in multiple different ruby processes (depending on how many processors you have, configurable) then aggregate all the results, unit test parallelisation, ability to declare some test classes as non-parallelisable in which case they execute in a traditional fashion after all the parallelised tests have been run
+- ability to only run unsuccessful tests from a previous run, only the fails and the errors
+- ability to run only the tests that were run on the previous run
+- print out how long it took to complete the current test run
 
 next release 3
 - come up with some more useful expectations, both on object and as standalone methods of test
 - look into rails stuff regarding expectations and idioms etc
 - should somehow be able to require all the necessary ruby files that are needed when a test is run for the test not to fail, e.g. because ruby files depend on other ruby files
+- hook into autotest somehow
 
-
-
-what to say in readme
-give an example of usage
-talk about how to define tests inline
-talk about how to execute the tests in one file or directory etc
-list the currently supported expectations
-talk about pendind tests
-talk about the fact that other output writers are coming
-talk about the fact that you can disable setup for particular tests
-talk about how to test modules within an included class
-no nested context as yet, but can define multiple top level ones, no shared tests or contexts as yet
-mention the examples directory but it is currently crappy
+bug 
+- when requiring ruby files in another ruby file that one seems to inherit the tests from the other one DONE 
+- have a look at doing a similar thing for load method, i.e. myself call new load with an introduced new paramter, and only in that case should the env var for attest be set otherwise even regular load should ignore the if attest block, only if it look slike being an issue
+- BUG when nosetup is defined on test when the test is parsed it is parsed in the context of the parser and therefore hits method missing the only reason it works right now is because there is an instance var on parser called nosetup

data/examples/{magic_calculator.rb → basic_functionality_example.rb}

@@ -1,4 +1,4 @@
-class MagicCalculator 
+class ACalculator 
   def remember_value(value)
     @value_in_memory = value
   end
@@ -22,19 +22,19 @@ class MagicCalculator
 end
 
 if ENV["attest"]
-  this_tests MagicCalculator do
-    before_all{@calculator = MagicCalculator.new}
-    after_all{@calculator = nil}
+  this_tests ACalculator do
+    before_each{@calculator = ACalculator.new}
+    after_each{@calculator = nil}
 
     test("a pending test")
     test("deliberately fail the test"){should_fail}
     test("a successful empty test"){}
     test("should NOT raise an error") {should_not_raise{@calculator.increment 4}}
     test("it should raise an error, don't care what kind") {should_raise {@calculator.divide 5, 0}}
-    test("it should raise a ZeroDivisionError error with a message"){should_raise(ZeroDivisionError){@calculator.divide 5, 0}.with_message(/divided by.*/)}
-    test("adding 5 and 2 does not equal 8") { should_not_be_true{ @calculator.add(5,2) == 8 } } 
+    test("it should raise a ZeroDivisionError error with a message"){should_raise(ZeroDivisionError, :with_message => /divided by.*/){@calculator.divide 5, 0}}
+    test("adding 5 and 2 does not equal 8") { should_not_be_true(@calculator.add(5,2) == 8) } 
     test("this test will be an error when non-existant method called") {should_be_true{ @calculator.non_existant }}
-    test("should be able to call a private method like it was public"){should_be_true{@calculator.multiply(2,2) == 4}}
+    test("should be able to call a private method like it was public"){should_be_true(@calculator.multiply(2,2) == 4)}
 
     test "access an instance variable without explicitly exposing it" do
       @calculator.remember_value(5)
@@ -45,6 +45,40 @@ if ENV["attest"]
       should_not_raise{@calculator.increment 4}
       should_raise{ @calculator.non_existant }
     end
-    test("should not have access to calculator instance since run without setup", nosetup){should_be_true{@calculator == nil}}
+
+    nosetup 
+    test("should not have access to calculator instance since run without setup"){should_be_true(@calculator == nil)}
+
+    test("should_equal expectations with and without block") do
+      should_equal 5, 5
+      should_equal(5){5}
+      should_not_equal 7, 8
+      should_not_be_equal(8){9}
+    end
+
+    test("should_be_same and should_not_be_same with and without block") do
+      should_be_same 5, 5
+      should_not_be_same(5){5.0}
+      string = "a"
+      should_be_same_as(string, string)
+      should_not_be_same_as("a"){string}
+    end
+
+    test("should_be_true without block") {should_be_true 5 == 5.0}
+    test("should_not_be_true without block") {should_be_false 5 == 6}
+
+    test("should_succeed test") do 
+      should_succeed
+    end
+
+    test("should_be a lambda based matcher test") do 
+      great = lambda{"great"}
+      should_be(great){"great"}
+      should_not_be(great){"bad"}
+
+      joke = lambda{"knock, knock"}
+      should_be_a(joke){"knock, knock"}
+      should_not_be_a(joke){"hello"}
+    end
   end
 end

data/examples/mocha_example.rb

@@ -0,0 +1,43 @@
+class Product 
+end
+
+if ENV["attest"]
+  this_tests "testing of basic mocha integration" do
+    test("mocking a class method") do 
+      product = Product.new
+      Product.expects(:find).with(1).returns(product)
+      should_be_true {product == Product.find(1)}
+    end
+
+    test("mocking an instance method") do 
+      product = Product.new
+      product.expects(:save).returns(true)
+      should_be_true{product.save}
+    end
+
+    test("stubbing an instance method") do 
+      prices = [stub(:pence => 1000), stub(:pence => 2000)]
+      product = Product.new
+      product.stubs(:prices).returns(prices)
+      should_be_true{  [1000, 2000] ==  product.prices.collect {|p| p.pence}}
+    end
+
+    test("stubbing an all instances") do 
+      Product.any_instance.stubs(:name).returns('stubbed_name')
+      product = Product.new
+      should_be_true{ 'stubbed_name' == product.name }
+    end
+
+    test("traditional mocking") do
+      object = mock()
+      object.expects(:expected_method).with(:p1, :p2).returns(:result)
+      should_be_true { :result == object.expected_method(:p1, :p2) }
+    end
+
+    test("shortcuts") do 
+      object = stub(:method1 => :result1, :method2 => :result2)
+      should_be_true{ :result1 == object.method1 }
+      should_be_true{ :result2 == object.method2 }
+    end
+  end
+end

data/examples/module_example.rb

@@ -0,0 +1,49 @@
+class StandardCalculator
+  def self.plus(x, y)
+    x + y
+  end
+
+  def self.minus(x, y)
+    x - y
+  end
+end
+
+module CalcModule
+  def double(x)
+    2 * x
+  end
+end
+
+if ENV["attest"]
+  this_tests "another class with calculations" do
+    test("adding two numbers") {should_be_true{StandardCalculator.plus(5,11) == 16}}
+    test("subtracting two numbers"){should_not_be_true{StandardCalculator.minus(10,5) == 4}}
+  end
+
+  this_tests CalcModule do
+    before_all do 
+      @hello = [1,2,3]
+    end
+
+    before_each { @module_class = create_and_include(CalcModule) }
+
+    test("magically instance of a class that will include the module"){should_be_true{@module_class.double(5)==10}}
+
+    nosetup
+    test("another test without setup"){should_be_true{true}}
+
+    disabled
+    test("a disabled test"){should_be_true{true}}
+
+    test("accessing object which was created in a before_all block") do 
+      should_be_true{@hello.size == 3}
+      @hello << 5
+      should_be_true{@hello.size == 4}
+    end
+
+    test("accessing object from before_all block again to make sure it is the same object") do 
+      should_be_true{@hello.last == 5}
+      @hello << 10
+    end
+  end
+end

data/examples/more/nesting/expectations_as_tests_example.rb

@@ -0,0 +1,20 @@
+class AssertionsAsTests
+  class << self
+    def five
+      5
+    end
+
+    def error
+      raise
+    end
+  end
+end
+
+if ENV["attest"]
+  this_tests "the fact that assertions can be specified as tests" do
+    should_be_true{5 == AssertionsAsTests.five}
+    should_raise do 
+      AssertionsAsTests.error
+    end
+  end
+end

data/lib/attest.rb

@@ -1,22 +1,6 @@
-require "rubygems"
-require "bundler"
-Bundler.setup(:default)
+$:.unshift(File.expand_path(File.dirname(__FILE__))) unless $:.include?(File.expand_path(File.dirname(__FILE__)))
 
-require "trollop"
-require "singleton"
 require 'attest/config'
-require 'attest/core_ext/kernel'
-require 'attest/core_ext/object'
-
-require 'attest/expectation_result'
-require 'attest/test_container'
-require 'attest/test_object'
-require 'attest/execution_context'
-require 'attest/test_parser'
-require 'attest/itself'
-require 'attest/attest_error'
-
-require 'attest/output/basic_output_writer'
 
 module Attest
   class << self
@@ -29,9 +13,9 @@ module Attest
 
     Attest::Config.public_instance_methods(false).each do |name|
       self.class_eval <<-EOT
-        def #{name}(*args)
-          configure.send("#{name}", *args)
-        end
+       def #{name}(*args)
+         configure.send("#{name}", *args)
+       end
       EOT
     end
   end

data/lib/attest/config.rb

@@ -1,12 +1,9 @@
+require 'singleton'
+
 module Attest
   class Config
     include Singleton
 
-    attr_accessor :output_writer, :current_file
-
-    def initialize
-      @output_writer = Attest::Output::BasicOutputWriter.new
-    end
-
+    attr_accessor :output_writer, :current_file, :testdouble, :possible_tests
   end
 end

data/lib/attest/core_ext/kernel.rb

@@ -1,5 +1,14 @@
+require 'attest/test_parser'
+
 module Kernel
   def new_method_missing(name, *args, &block)
+    original_error = nil
+    begin
+        old_method_missing(name, *args, &block) 
+      return
+    rescue NoMethodError => e
+      original_error = e
+    end
     private_method = false
     instance_variable = false
     private_methods.each do |meth|
@@ -14,12 +23,21 @@ module Kernel
       end
       send(name, *args, &block)
     else
-      old_method_missing(name, *args, &block) 
+      raise original_error
     end
   end
   alias_method :old_method_missing, :method_missing
   alias_method :method_missing, :new_method_missing
 
+  def new_require(filename)
+    current_attest_value = ENV["attest"]
+    ENV["attest"] = nil
+    old_require(filename)
+    ENV["attest"] = current_attest_value
+  end
+  alias_method :old_require, :require
+  alias_method :require, :new_require
+
   private
   def this_tests(description="anonymous", &block)
     container = Attest::TestParser.new(description, block).parse

data/lib/attest/core_ext/object.rb

@@ -1,5 +1,4 @@
 class Object
-  def itself 
-    Attest::Itself.new(self)
+  def hello 
   end
 end

data/lib/attest/core_ext/proc.rb

@@ -0,0 +1,35 @@
+current_dir = File.expand_path(File.dirname(__FILE__))
+require "#{current_dir}/../proc/proc_source_reader"
+
+class Proc
+  def to_string
+    @source ||= Attest::ProcSourceReader.find(*source_descriptor)
+  end
+
+  private
+  def source_descriptor
+    unless @file && @line
+      if md = /^#<Proc:0x[0-9A-Fa-f]+@(.+):(\d+)(.+?)?>$/.match(inspect)
+        @file, @line = md.captures
+      end
+    end
+    [@file, @line.to_i]
+  end
+end
+
+if __FILE__ == $0
+  simple_proc = Proc.new() { |a|
+    puts  "Hello Rudy2" 
+  }
+  puts simple_proc.to_string
+
+  another_simple_proc = Proc.new() do |x|
+    #printing
+    puts "Printing #{x}"
+    puts "printing more stuff"
+  end
+  puts another_simple_proc.to_string
+
+  third_proc = eval "Proc.new do puts 'blah'; end"
+  puts third_proc.to_string
+end