any-spec 0.0.0

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2010 Aaron Gough (http://thingsaaronmade.com/)
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.rdoc

@@ -0,0 +1,177 @@
+= AnySpec
+
+AnySpec is a framework for writing executable language specifications for programming languages. AnySpec makes it easy to write an automated black-box test suite for a target language. AnySpec test-cases are completely language agnostic, though the assertions associated with a test-case are written in Ruby.
+
+To maintain its independence from the language under test AnySpec expects that a test-case will run some arbitrary text file against an executable, and then the output from that executable will be assessed using the assertions defined in the test-case. This method of operation has the disadvantage that introspection into the internals of the executable is not possible, but has the advantage that the target language does not have to be advanced in its development to be targeted, unlike an in-language specification like RubySpec.
+
+=== Example Tests
+
+Test cases written using AnySpec combine the program code under test and the test assertions into a single file. The program code is defined first, then a separator is inserted, then the assertions are defined:
+
+  (define test 1)  
+  (display test)
+  
+  ----
+  
+  assert_output "1"
+  assert_execution_success
+  
+Writing a 'dirty' test is also easy:
+
+  (define 1_invalid_identifier 1)
+  
+  ----
+  
+  assert_execution_failure  
+
+Having a test-case that does not contain 'assert_execution_failure' implies that the test should execute successfully. Therefore:
+
+  puts 1
+  
+  ---- 
+  
+  assert_output "1"
+
+Is equivalent to:
+
+  puts 1
+  
+  ---- 
+  
+  assert_output "1"
+  assert_execution_success
+
+=== Installation
+
+For convenience AnySpec is packaged as a RubyGem. If you already have Ruby and RubyGems installed, then installing AnySpec is as easy as entering this at your command line:
+
+  > gem install any-spec
+  
+Mac OS X and most flavours of Linux come with Ruby and RubyGems preinstalled. If you do not have them installed please check the {installation instructions on the Ruby website}[http://www.ruby-lang.org/en/downloads/]
+
+=== Usage
+
+AnySpec expects that you define a small specification file in YAML format telling it where your test-cases are and what their file-extension is, for instance:
+
+  # in file: r5rs-spec.yaml
+  specification_root: test_cases
+  specification_extension: .scm
+  
+AnySpec will look for test cases recursively inside the 'specification_root' folder, so with this specification file, the directory structure for your specification might look like this:
+
+  r5rs-spec
+    |
+    |- r5rs-spec.yaml
+    |- test_cases
+         |
+         |- 01-output-tests
+         |    |
+         |    |- 01-display-test.scm
+         |    |- 02-stderr-test.scm
+         |
+         |- 02-variable-tests
+              |
+              |- 01-define-test.scm
+              |- 02-set-test.scm
+              
+Any files that have a different extension to the one specified using 'specification_extension' will be ignored. Any matching files will be executed in directory/file order, which allows you to specify which order the test cases should be run in.
+
+Once you have your specification file and test-cases in order you can run them using the AnySpec command line interface:
+
+  > any-spec executable_to_target path_to_specification_file
+  
+This makes it very easy to run a specification against multiple runtimes for a given language, and can also help a lot when trying to initially bootstrap a language.
+
+=== API
+
+AnySpec currently defines a small set of assertions:
+
+  # Make the test-case always fail, good as a reminder that you haven't yet fully
+  # implemented a particular test-case
+  flunk
+
+  # Assert that the test-case runs and exits successfully (returns exit status 0)
+  assert_execution_success
+  
+  # Assert that the test-case does not execute cleanly (returns an exit status other than 0)
+  assert_execution_failure
+  
+  # Assert that the test-case outputs a given string to either STDOUT or STDERR
+  assert_output "YOUR STRING HERE" 
+  
+AnySpec assertions are all written in Ruby and executed in the context of the test-case. You can build your own custom assertions using the following generic assertions:
+
+  # Assert that 'expression' evaluates to true
+  assert expression
+  
+  # Assert that the result of a block of custom code is true
+  assert_block("Your custom failure message here") do
+    ...
+    your_custom_code_here
+    ...
+    return result # true or false
+  end
+  
+For example:
+
+  assert 1 == 2   # "false is not true"
+  assert nil      # "nil is not true"
+  assert 1 == 1   # passes
+  
+  assert_block("There is something wrong with the universe!") do
+    true == true
+  end
+  
+If necessary you can also write completely custom Ruby code that has access to the data from the test-case. Accessor methods provided by the test case are:
+
+  # The result of the last assertion, if it is 'false' no further assertions should be run
+  @test_case.last_assertion_result
+  
+  # The captured output from the test-case (AnySpec captures output to STDOUT and STDERR)
+  @test_case.test_output
+  
+  # The number of assertions that have already been run against this test-case.
+  # If you write custom assertions (not using 'assert' or 'assert_block') then
+  # you will need to increment this counter manually for each assertion you run
+  @test_case.assertions
+  
+  # The message that will be output if the assertions fail, this is set automatically
+  # by the built-in assertions, but you will need to set it manually if you write your own
+  @test_case.message
+  
+An example of a custom assertion:
+
+  module AnySpec
+    class Assertions
+    
+      # Lets create a custom assertion
+      def always_fail
+        # Avoid running this assertion if a previous assertion has already failed
+        # all the built-in assertions do this, and all custom ones should too
+        return if(@test_case.last_assertion_result == false)
+        
+        # Set the assertion result to false because we're always failing
+        @test_case.last_assertion_result = false
+        
+        # Set the failure message, this will be displayed to whoever is running the test suite
+        @test_case.message = "This assertion always fails, the output of the test-case was:\n"
+        
+        # Add the test-case output to the failure message so the user can see it
+        @test_case.message += @test_case.test_output + "\n\n"
+      end
+    
+    end
+  end
+  
+  
+=== Contributing and Feedback
+
+If you have an issue with AnySpec or would like to see specific functionality added to it please email me at: aaron@aarongough.com
+
+If you would like to provide a patch please fork AnySpec, write your patches and then send me a pull request. Make sure you write tests for your new functionality so I don't accidentally break it in the future. Please do not touch VERSION or Rakefile.  
+
+=== Author & Credits
+
+Author:: {Aaron Gough}[mailto:aaron@aarongough.com]
+
+Copyright (c) 2010 {Aaron Gough}[http://thingsaaronmade.com/] ({thingsaaronmade.com}[http://thingsaaronmade.com/]), released under the MIT license

data/Rakefile

@@ -0,0 +1,43 @@
+require 'rake'
+require 'rake/testtask'
+require 'rake/rdoctask'
+
+desc 'Default: run unit tests.'
+task :default => :test
+
+begin
+  require 'jeweler'
+  Jeweler::Tasks.new do |gemspec|
+    gemspec.name = "any-spec"
+    gemspec.summary = "A framework for writing executable language specifications"
+    gemspec.description = "AnySpec is a framework for writing executable language specifications and automated black-box functional tests for programming languages."
+    gemspec.email = "aaron@aarongough.com"
+    gemspec.homepage = "http://github.com/aarongough/any-spec"
+    gemspec.authors = ["Aaron Gough"]
+    gemspec.rdoc_options << '--line-numbers' << '--inline-source'
+    gemspec.extra_rdoc_files = ['README.rdoc', 'MIT-LICENSE']
+    gemspec.executables << "any-spec"
+  end
+rescue LoadError
+  puts "Jeweler not available. Install it with: gem install jeweler"
+end
+
+
+desc 'Test AnySpec.'
+Rake::TestTask.new(:test) do |t|
+  t.libs << 'lib/*.rb'
+  t.libs << 'test'
+  t.pattern = 'test/**/*_test.rb'
+  t.verbose = true
+end
+
+
+desc 'Generate documentation for AnySpec.'
+Rake::RDocTask.new(:rdoc) do |rdoc|
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.title    = 'AnySpec'
+  rdoc.options << '--line-numbers' << '--inline-source'
+  rdoc.rdoc_files.include('README.rdoc')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+  rdoc.rdoc_files.include('app/**/*.rb')
+end

data/VERSION

@@ -0,0 +1 @@
+0.0.0

data/bin/any-spec

@@ -0,0 +1,6 @@
+#! /usr/bin/env ruby
+
+require File.expand_path(File.join(File.dirname(__FILE__), '..', 'lib', 'any-spec.rb'))
+
+test_runner = AnySpec::TestRunner.new(ARGV[0], ARGV[1])
+test_runner.run_tests

data/lib/any-spec.rb

@@ -0,0 +1 @@
+Dir[File.join(File.dirname(__FILE__), 'any-spec', '**',  '*.rb')].each {|file| require File.expand_path(file) }

data/lib/any-spec/assertions.rb

@@ -0,0 +1,45 @@
+module AnySpec
+  class Assertions
+  
+    def initialize(test_case_instance)
+      @test_case = test_case_instance
+    end
+  
+    def flunk
+      message = "Flunked."
+      assert_block(message) { false }
+    end
+  
+    def assert( expression )
+      message = "#{expression.inspect} is not true"
+      assert_block(message) { expression == true }
+    end
+    
+    def assert_output(expected, output = @test_case.test_output)
+      message = "Expected output to be:\n<#{expected.inspect}> but was:\n<#{output.inspect}>"
+      assert_block(message) { output == expected }
+    end
+    
+    def assert_execution_success
+      message = "Execution of test case failed when it was expected to succeed:\n"
+      message += @test_case.test_output
+      assert_block(message) { @test_case.exit_status == 0 }
+    end
+    
+    def assert_execution_failure
+      message = "Execution of test case succeeded when it was expected to fail:\n"
+      message +=  @test_case.test_output
+      assert_block(message) { @test_case.exit_status != 0 }
+    end
+    
+    def assert_block(message = "assert_block failed")
+      return if(@test_case.last_assertion_result == false)
+      @test_case.assertions += 1
+      unless( yield )
+        @test_case.last_assertion_result = false
+        @test_case.message = message
+      end
+    end
+  
+  end
+end

data/lib/any-spec/test-case.rb

@@ -0,0 +1,53 @@
+module AnySpec
+  class TestCase
+  
+    attr_accessor :test_code,
+                  :assertion_code,
+                  :target_executable,
+                  :path,
+                  :last_assertion_result,
+                  :message,
+                  :test_output,
+                  :exit_status,
+                  :assertions     
+  
+    def initialize(path, target_executable)
+      raise Exception, "\n\nNo test case exists at path: #{path}" unless(File.exist?(path))
+      raise Exception, "\n\nTest case is empty: #{path}" if(File.size(path) == 0)
+      raw_test = File.open(path).read
+      test_parts = raw_test.split("----")
+      raise Exception, "\n\nTest case formatted incorrectly: #{path}" unless(test_parts.length == 2)
+      @test_code = test_parts[0].strip
+      @assertion_code = test_parts[1].strip
+      @path = path
+      @target_executable = target_executable
+    end
+    
+    def run
+      temporary_filename = File.join( File.split(@path)[0], "#{Time.now.to_i}-any-spec" + File.extname(@path) )
+      tmp = File.open(temporary_filename, "w")
+      tmp.write(@test_code)
+      tmp.close
+      @test_output = `#{@target_executable} #{temporary_filename} 2>&1`
+      @exit_status = $?.exitstatus
+      @last_assertion_result = true
+      @message = ""
+      @assertions = 0
+      unless(@assertion_code.include?("assert_execution_success") || @assertion_code.include?("assert_execution_failure"))
+        @assertion_code = "\nassert_execution_success\n" + @assertion_code
+      end
+      begin
+        AnySpec::Assertions.new(self).instance_eval(@assertion_code, @path)
+      rescue Exception => e
+        @message = "Error in assertion code:\n"
+        @message += e.message
+        @message += "\n" + e.backtrace.join("\n")
+        @last_assertion_result = false
+      end
+      return @last_assertion_result
+    ensure
+      File.delete(temporary_filename)
+    end
+    
+  end
+end

data/lib/any-spec/test-runner.rb

@@ -0,0 +1,69 @@
+module AnySpec
+  class TestRunner
+  
+    require 'yaml'
+    
+    attr_accessor :target_executable,
+                  :specification_root,
+                  :specification_extension,
+                  :test_case_paths,
+                  :test_cases,
+                  :report
+  
+    def initialize( target_executable, test_specification_file )
+      # Verify that the target executable exists and is in the current PATH
+      @target_executable = `which #{target_executable}`.strip
+      raise Exception, "The target executable you specified does not exist!" if(@target_executable.empty?)
+      # Load the test specification file
+      @test_specification_file = File.expand_path(test_specification_file)
+      raise Exception, "The test specification file you supplied does not exist." unless(File.exist? @test_specification_file)
+      test_spec = YAML::load_file(@test_specification_file)
+      @specification_root = test_spec["specification_root"]
+      @specification_extension = test_spec["specification_extension"]
+      # Find and load test-case file paths
+      @test_case_paths = Dir[File.join(File.split(@test_specification_file)[0], @specification_root, '**',  "*" + @specification_extension)].sort
+      # Instantiate the test cases
+      @test_cases = @test_case_paths.map do |test_case|
+        AnySpec::TestCase.new(test_case, @target_executable)
+      end
+    end
+    
+    def run_tests(silence = false)
+      @report = ""
+      @silence = silence
+      message "\nLoaded suite: #{@test_specification_file}\n"
+      message "Targeting: #{@target_executable}\n"
+      message "\nStarted\n"
+      start_time = Time.now
+      assertions = 0
+      failed_tests = []
+      @test_cases.each do |test_case|
+        result = test_case.run
+        message "." if(result)
+        message "F" if(!result)
+        assertions += test_case.assertions
+        failed_tests << test_case if(!result)
+      end
+      message "\nFinished in #{(Time.now - start_time).to_f} seconds.\n\n"
+      failed_tests.each_index do |x|
+        test_case = failed_tests[x]
+        message "  #{x + 1}) Failure:\n"
+        message "In file: " + test_case.path.gsub(File.split(@test_specification_file)[0], "") + "\n"
+        message test_case.message + "\n\n"
+      end
+      pass_rate = format("%.2f",((failed_tests.length.to_f) / @test_cases.length) * 100)
+      message "#{@test_cases.length} tests, #{assertions} assertions, #{failed_tests.count} failures, #{pass_rate}% pass rate\n\n"
+      return @test_cases
+    end
+    
+    def message(string)
+      if(@silence)
+        @report << string
+      else
+        print string
+        $stdout.flush
+      end
+    end
+    
+  end
+end

data/test/fixtures/example_test_specification.yml

@@ -0,0 +1,2 @@
+specification_root: test_cases
+specification_extension: .rb

data/test/fixtures/test_cases/error.rb

@@ -0,0 +1,4 @@
+puts 1 * "10"
+
+----
+

data/test/fixtures/test_cases/failing.rb

@@ -0,0 +1,5 @@
+print "foo"
+
+----
+
+assert_output "bar"

data/test/fixtures/test_cases/incorrect.blah

@@ -0,0 +1 @@
+blah

data/test/fixtures/test_cases/test1.rb

@@ -0,0 +1,5 @@
+print "this is a test"
+
+----
+
+assert_output "this is a test"

data/test/fixtures/test_cases/test2.rb

@@ -0,0 +1,5 @@
+print :foo
+
+----
+
+assert_output "foo"

data/test/setup/test_unit_extensions.rb

@@ -0,0 +1,21 @@
+module Test::Unit
+  # Used to fix a minor minitest/unit incompatibility in flexmock
+  AssertionFailedError = Class.new(StandardError)
+  
+  class TestCase
+   
+    def self.test(name, &block)
+      test_name = "test_#{name.gsub(/\s+/,'_')}".to_sym
+      defined = instance_method(test_name) rescue false
+      raise "#{test_name} is already defined in #{self}" if defined
+      if block_given?
+        define_method(test_name, &block)
+      else
+        define_method(test_name) do
+          flunk "No implementation provided for #{name}"
+        end
+      end
+    end
+
+  end
+end

data/test/test_helper.rb

@@ -0,0 +1,13 @@
+$test = true
+
+require 'rubygems'
+require 'test/unit'
+require 'yaml'
+
+require_files = []
+require_files << File.join(File.dirname(__FILE__), '..', 'lib',  'any-spec.rb')
+require_files.concat Dir[File.join(File.dirname(__FILE__), 'setup', '*.rb')]
+
+require_files.each do |file|
+  require File.expand_path(file)
+end

data/test/unit/assertions_test.rb

@@ -0,0 +1,103 @@
+require File.expand_path(File.join(File.dirname(__FILE__), '..',  'test_helper.rb'))
+
+class AssertionsTest < Test::Unit::TestCase
+  
+  require 'ostruct'
+  
+  def setup
+    @test_case = OpenStruct.new
+    @test_case.last_assertion_result = true
+    @test_case.message = ""
+    @test_case.assertions = 0
+  end
+  
+  test "assert_block should pass when block returns true" do
+    AnySpec::Assertions.new(@test_case).assert_block {true}
+    assert_equal true, @test_case.last_assertion_result
+    assert_equal "", @test_case.message
+  end
+  
+  test "assert_block should increment assertion counter" do
+    AnySpec::Assertions.new(@test_case).assert_block {true}
+    assert_equal 1, @test_case.assertions
+    AnySpec::Assertions.new(@test_case).assert_block {true}
+    assert_equal 2, @test_case.assertions
+  end
+  
+  test "assert_block should fail when block returns false" do
+    AnySpec::Assertions.new(@test_case).assert_block {false}
+    assert_equal false, @test_case.last_assertion_result
+    assert_equal "assert_block failed", @test_case.message
+  end
+  
+  test "flunk should flunk" do
+    AnySpec::Assertions.new(@test_case).flunk
+    assert_equal false, @test_case.last_assertion_result
+    assert_equal "Flunked.", @test_case.message
+  end
+  
+  test "assert should assert truthiness" do
+    AnySpec::Assertions.new(@test_case).assert(true)
+    assert_equal true, @test_case.last_assertion_result
+    assert_equal "", @test_case.message
+  end
+  
+  test "assert should fail on false" do
+    AnySpec::Assertions.new(@test_case).assert(false)
+    assert_equal false, @test_case.last_assertion_result
+    assert_equal "false is not true", @test_case.message
+  end
+  
+  test "assert should fail on nil" do
+    AnySpec::Assertions.new(@test_case).assert(nil)
+    assert_equal false, @test_case.last_assertion_result
+    assert_equal "nil is not true", @test_case.message
+  end
+  
+  test "assert_output should pass when output matches" do
+    @test_case.test_output = "test"
+    AnySpec::Assertions.new(@test_case).assert_output("test")
+    assert_equal true, @test_case.last_assertion_result
+    assert_equal "", @test_case.message
+  end
+  
+  test "assert_output should fail when output does not match" do
+    @test_case.test_output = "test2"
+    AnySpec::Assertions.new(@test_case).assert_output("test")
+    assert_equal false, @test_case.last_assertion_result
+    assert_equal "Expected output to be:\n<\"test\"> but was:\n<\"test2\">", @test_case.message
+  end
+  
+  test "assert_execution_success should pass when exit_status == 0" do
+    @test_case.exit_status = 0
+    @test_case.test_output = ""
+    AnySpec::Assertions.new(@test_case).assert_execution_success
+    assert_equal true, @test_case.last_assertion_result
+    assert_equal '', @test_case.message
+  end
+  
+  test "assert_execution_success should fail when exit_status != 0" do
+    @test_case.test_output = "test2"
+    @test_case.exit_status = 1
+    AnySpec::Assertions.new(@test_case).assert_execution_success
+    assert_equal false, @test_case.last_assertion_result
+    assert_equal "Execution of test case failed when it was expected to succeed:\ntest2", @test_case.message
+  end
+  
+  test "assert_execution_failure should fail when exit_status == 0" do
+    @test_case.test_output = "test2"
+    @test_case.exit_status = 0
+    AnySpec::Assertions.new(@test_case).assert_execution_failure
+    assert_equal false, @test_case.last_assertion_result
+    assert_equal "Execution of test case succeeded when it was expected to fail:\ntest2", @test_case.message
+  end
+  
+  test "assert_execution_failure should pass when exit_status != 0" do
+    @test_case.exit_status = 1
+    @test_case.test_output = ""
+    AnySpec::Assertions.new(@test_case).assert_execution_failure
+    assert_equal true, @test_case.last_assertion_result
+    assert_equal '', @test_case.message
+  end
+  
+end

data/test/unit/test_case_test.rb

@@ -0,0 +1,44 @@
+require File.expand_path(File.join(File.dirname(__FILE__), '..',  'test_helper.rb'))
+
+class TestCaseTest < Test::Unit::TestCase
+  
+  def setup
+    @test_file = File.expand_path(File.join(File.dirname(__FILE__), '..', 'fixtures', 'test_cases', 'test1.rb'))
+  end
+  
+  test "should load test case" do
+    test_case = AnySpec::TestCase.new(@test_file, "ruby")
+    assert_equal test_case.test_code, 'print "this is a test"'
+    assert_equal test_case.assertion_code, 'assert_output "this is a test"'
+    assert_equal "ruby", test_case.target_executable
+    assert_equal @test_file, test_case.path
+  end
+  
+  test "should run test case" do
+    test_case = AnySpec::TestCase.new(@test_file, "ruby")
+    result = test_case.run
+    assert_equal true, result
+  end
+  
+  test "test case that is not formatted correctly should raise exception" do
+    assert_raises Exception do
+      @test_file = File.expand_path(File.join(File.dirname(__FILE__), '..', 'fixtures', 'test_cases', 'incorrect.blah'))
+      test_case = AnySpec::TestCase.new(@test_file, "ruby")
+    end
+  end
+  
+  test "empty test case should raise exception" do
+    assert_raises Exception do
+      @test_file = File.expand_path(File.join(File.dirname(__FILE__), '..', 'fixtures', 'test_cases', 'empty.blah'))
+      test_case = AnySpec::TestCase.new(@test_file, "ruby")
+    end
+  end
+  
+  test "test case that does not exist should raise exception" do
+    assert_raises Exception do
+      @test_file = File.expand_path(File.join(File.dirname(__FILE__), '..', 'fixtures', 'test_cases', 'does_not_exist.blah'))
+      test_case = AnySpec::TestCase.new(@test_file, "ruby")
+    end
+  end
+  
+end

data/test/unit/test_runner_test.rb

@@ -0,0 +1,37 @@
+require File.expand_path(File.join(File.dirname(__FILE__), '..',  'test_helper.rb'))
+
+class TestRunnerTest < Test::Unit::TestCase
+  
+  def setup
+    @spec_file = File.expand_path(File.join(File.dirname(__FILE__), '..', 'fixtures', 'example_test_specification.yml'))
+    @spec = YAML::load_file(@spec_file)
+  end
+  
+  test "should create new TestRunner instance" do
+    test_runner = AnySpec::TestRunner.new("ruby", @spec_file)
+    assert_equal `which ruby`.strip, test_runner.target_executable
+    assert_equal @spec["specification_root"], test_runner.specification_root
+    assert_equal @spec["specification_extension"], test_runner.specification_extension
+    assert_equal 4, test_runner.test_case_paths.length
+    assert_equal 4, test_runner.test_cases.length
+  end
+  
+  test "should raise error when target executable does not exist" do
+    assert_raises Exception do
+      test_runner = AnySpec::TestRunner.new("blagitz", @spec_file)
+    end
+  end
+  
+  test "should raise error when specification file does not exist" do
+    assert_raises Exception do
+      test_runner = AnySpec::TestRunner.new("ruby", "~/blah/foo/blag.bar")
+    end
+  end
+  
+  test "should run test cases" do
+    test_runner = AnySpec::TestRunner.new("ruby", @spec_file)
+    results = test_runner.run_tests(true)
+    assert_equal 4, results.length
+  end
+  
+end

metadata

@@ -0,0 +1,96 @@
+--- !ruby/object:Gem::Specification 
+name: any-spec
+version: !ruby/object:Gem::Version 
+  prerelease: false
+  segments: 
+  - 0
+  - 0
+  - 0
+  version: 0.0.0
+platform: ruby
+authors: 
+- Aaron Gough
+autorequire: 
+bindir: bin
+cert_chain: []
+
+date: 2010-10-08 00:00:00 -04:00
+default_executable: 
+dependencies: []
+
+description: AnySpec is a framework for writing executable language specifications and automated black-box functional tests for programming languages.
+email: aaron@aarongough.com
+executables: 
+- any-spec
+- any-spec
+extensions: []
+
+extra_rdoc_files: 
+- MIT-LICENSE
+- README.rdoc
+files: 
+- MIT-LICENSE
+- README.rdoc
+- Rakefile
+- VERSION
+- bin/any-spec
+- lib/any-spec.rb
+- lib/any-spec/assertions.rb
+- lib/any-spec/test-case.rb
+- lib/any-spec/test-runner.rb
+- test/fixtures/example_test_specification.yml
+- test/fixtures/test_cases/empty.blah
+- test/fixtures/test_cases/error.rb
+- test/fixtures/test_cases/failing.rb
+- test/fixtures/test_cases/incorrect.blah
+- test/fixtures/test_cases/test1.rb
+- test/fixtures/test_cases/test2.rb
+- test/setup/test_unit_extensions.rb
+- test/test_helper.rb
+- test/unit/assertions_test.rb
+- test/unit/test_case_test.rb
+- test/unit/test_runner_test.rb
+has_rdoc: true
+homepage: http://github.com/aarongough/any-spec
+licenses: []
+
+post_install_message: 
+rdoc_options: 
+- --charset=UTF-8
+- --line-numbers
+- --inline-source
+require_paths: 
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement 
+  none: false
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      segments: 
+      - 0
+      version: "0"
+required_rubygems_version: !ruby/object:Gem::Requirement 
+  none: false
+  requirements: 
+  - - ">="
+    - !ruby/object:Gem::Version 
+      segments: 
+      - 0
+      version: "0"
+requirements: []
+
+rubyforge_project: 
+rubygems_version: 1.3.7
+signing_key: 
+specification_version: 3
+summary: A framework for writing executable language specifications
+test_files: 
+- test/fixtures/test_cases/error.rb
+- test/fixtures/test_cases/failing.rb
+- test/fixtures/test_cases/test1.rb
+- test/fixtures/test_cases/test2.rb
+- test/setup/test_unit_extensions.rb
+- test/test_helper.rb
+- test/unit/assertions_test.rb
+- test/unit/test_case_test.rb
+- test/unit/test_runner_test.rb