delano-tryouts 0.5.1 → 0.6.0

data/CHANGES.txt

@@ -1,5 +1,20 @@
 TRYOUTS, CHANGES
 
+
+#### 0.6.0 (2009-06-24) ###############################
+
+NOTE: External dreams files are no longer supported.
+NOTE: command testing (:cli) is disabled. 
+
+* CHANGE: Removed all mention of external dreams files. 
+* CHANGE: dream method is not available inside drill block. 
+* CHANGE: rcode, emsg, backtrace renamed to ecode, error, trace
+* ADDED: dream is now available above drill blocks. 
+* ADDED: Better verbose and debugging output
+* ADDED: Cleaner test output
+* ADDED: drill success is now solely based on output. 
+
+
 #### 0.5.1 (2009-06-13) ###############################
 
 * ADDED: dream method is now available inside drill block

data/README.rdoc

@@ -1,7 +1,8 @@
-= Tryouts - v0.5 BETA
+= Tryouts - v0.6 BETA
 
 Tryouts is a high-level testing library for your command-line applications and Ruby codes.
 
+NOTE: The DSL syntax has changed significantly from 0.5 and command-line app tests are currently disabled. 
 
 == Terminology
 
@@ -11,41 +12,6 @@ Tryouts is a bit different than other testing libraries. Test definitions are or
 * Drill: a test. 
 * Dream: the expected outcome of a drill. 
 
-
-== Testing a command-line application (a CLI)
-
-Tryouts tests command-line applications by comparing expected output with the actual output. Let's say we have an executable called mockout and we want to test the following commands:
-
-    $ bin/executable
-    $ bin/executable -f yaml
-    
-The tryout definition would look like this:
-
-    command :executable, "path/2/executable"
-    
-    tryout "Common Usage" do
-      drill  "No Command"
-      drill "YAML Output", :f, 'yaml'
-    end
-    
-And the expected output would be defined like this:
-
-    dream "No Command" do
-      output inline(%Q{
-            Date:                                   2009-02-16
-         Players:            d-bam, alberta, birds, condor man
-          Owners:            greg, rupaul, telly, prince kinko
-      })
-    end
-    dream "YAML Output" do
-      format :yaml
-      output ({
-        "Date" => "2009-02-16",
-        "Players" => ["d-bam", "alberta", "birds", "condor man"],
-        "Owners" => ["greg", "rupaul", "telly", "prince kinko"]
-      })
-    end
-    
 == Testing Ruby codes (an API)
 
 Tryouts employs the same approach for testing Ruby codes. The return value of the drill block is compared to the expectation defined by the dream. Here is an example of including dreams inside the tryout definition. 
@@ -53,17 +19,22 @@ Tryouts employs the same approach for testing Ruby codes. The return value of th
     library :caesars, LIBRARY_PATH
 
     tryout "Common Usage" do
-      dream "Some Maths", 3 
-      drill "Some Maths" do
+      dream 3
+      drill "Check the return value" do
         12 / 4
       end
         
-      dream "A Block", Proc, :class  # Test the class type instead of the value
-      drill "A Block" do
+      dream :class, :Proc
+      drill "Check the return value class" do
         Proc.new do
           :anything
         end
       end
+      
+      dream :exception, NameError
+      drill "A test can pass based on the exception" do
+        raise NameError
+      end
     end
 
 
@@ -75,23 +46,9 @@ This library is very new (est. 2009-05-19) and has not been vetted by the scruti
 * Unexpected errors. 
 
 
-== 3 Ways to define tryouts
-
-There are three ways to define an instance of this class:
-* In +_tryouts.rb+ files using the DSL syntax. One file per Tryouts object. 
-  * See: http://github.com/delano/tryouts/blob/master/tryouts/mockoutcli_tryouts.rb
-* In standalone ruby files using a hybrid DSL/OO syntax. Supports multiple 
-  Tryouts objects per file. 
-  * See: http://github.com/delano/tryouts/blob/master/tryouts/standalone_test.rb
-* With regular object-oriented syntax.
-  * See: http://tryouts.rubyforge.org/
-
 == On Threads
 
-Tryouts does some funky stuff to make it simple to write tests. This "funky 
-stuff" means that this library is *not thread-safe at definition-time*. However, once 
-all tryouts files are parsed (or in OO-syntax, once all objects are created), this
-class should be *thread-safe at drill-time*. 
+Tryouts does some funky stuff to make it simple to write tests. This "funky stuff" means that this library is *not thread-safe at definition-time*. However, once all tryouts files are parsed (or in OO-syntax, once all objects are created), this class should be *thread-safe at drill-time*. 
 
 == More Info
 
@@ -101,7 +58,7 @@ class should be *thread-safe at drill-time*.
 
 == Thanks
 
-* Everyone at Utrecht.rb 
+* Everyone at Utrecht.rb and Amsterdam.rb
 
 == Credits
 

data/bin/sergeant

@@ -2,8 +2,8 @@
 
 TRYOUTS_HOME = File.expand_path(File.join(File.dirname(__FILE__), '..'))
 
-local_libs = %w{tryouts net-scp amazon-ec2 aws-s3 caesars drydock rye storable sysinfo annoy}
-local_libs.each { |dir| $:.unshift File.join(TRYOUTS_HOME, '..', dir, 'lib') }
+#local_libs = %w{tryouts net-scp amazon-ec2 aws-s3 caesars drydock rye storable sysinfo annoy}
+#local_libs.each { |dir| $:.unshift File.join(TRYOUTS_HOME, '..', dir, 'lib') }
 
 require 'drydock'
 require 'tryouts'
@@ -15,7 +15,7 @@ require 'tryouts/cli'
 module TryoutsCLI
   extend Drydock
   
-  debug :on
+  debug :off
   default :run, :with_args
   
   global :q, :quiet, "Decrease output"

data/lib/tryouts.rb

@@ -4,7 +4,7 @@ require 'rye'
 require 'yaml'
 begin; require 'json'; rescue LoadError; end   # json may not be installed
 
-GYMNASIUM_HOME = File.join(Dir.pwd, 'tryouts')
+GYMNASIUM_HOME = File.join(Dir.pwd, 'tryouts')  ## also check try (for rye)
 GYMNASIUM_GLOB = File.join(GYMNASIUM_HOME, '**', '*_tryouts.rb')
 
 
@@ -29,7 +29,7 @@ class Tryouts
   # Raised when there is a problem loading or parsing a Tryouts::Drill::Dream object
   class BadDreams < Exception; end
   
-  VERSION = "0.5.1"
+  VERSION = "0.6.0"
   
   require 'tryouts/mixins'
   require 'tryouts/tryout'
@@ -49,6 +49,17 @@ class Tryouts
     # An instance of SysInfo
   @@sysinfo = SysInfo.new
   
+  @@debug = false
+  @@verbose = 0
+  
+  def self.debug?; @@debug; end
+  def self.enable_debug; @@debug = true; end
+  def self.disable_debug; @@debug = false; end
+  
+  def self.verbose; @@verbose; end
+  def self.verbose=(v); @@verbose = (v == true) ? 1 : v; end
+  
+  
   # Returns +@@instances+
   def self.instances; @@instances; end
   # Returns +@@sysinfo+
@@ -60,26 +71,18 @@ class Tryouts
   attr_accessor :dtype
     # An Array of file paths which populated this instance of Tryouts
   attr_accessor :paths
-    # A Hash of dreams for all tryouts in this class. The keys should
-    # match the names of each tryout. The values are hashes will drill
-    # names as keys and response
-  attr_accessor :dreams
     # An Array of Tryout objects
   attr_accessor :tryouts
     # A Symbol representing the command taking part in the tryouts. For @dtype :cli only. 
   attr_accessor :command
     # A Symbol representing the name of the library taking part in the tryouts. For @dtype :api only.
   attr_accessor :library
-    # The name of the most recent dreams group (see self.dream)
-  attr_accessor :dream_pointer
 
   def initialize(group=nil)
     @group = group || "Default Group"
     @tryouts = HASH_TYPE.new
     @paths = []
     @command = nil
-    @dreams = HASH_TYPE.new
-    @dream_pointer = nil
   end
   
   # Populate this Tryouts from a block. The block should contain calls to 
@@ -101,6 +104,7 @@ class Tryouts
   # Add a shell command to Rye::Cmd and save the command name
   # in @@commands so it can be used as the default for drills
   def command(name=nil, path=nil)
+    raise "command testing is temporarily disabled"
     return @command if name.nil?
     @command = name.to_sym
     @dtype = :cli
@@ -139,9 +143,6 @@ class Tryouts
   def group(name=nil)
     return @group if name.nil?
     @group = name unless name.nil?
-    # Preload dreams if possible
-    dfile = self.class.find_dreams_file(GYMNASIUM_HOME, @group)
-    self.load_dreams_file(dfile) if dfile
     @group
   end
   # Raises a Tryouts::Exception. +group+ is not support in the standalone syntax
@@ -171,8 +172,7 @@ class Tryouts
       to = Tryouts::Tryout.new(name, dtype, command)
       @tryouts[name] = to
     end
-    # Populate the dreams if they've already been loaded
-    to.dreams = @dreams[name] if @dreams.has_key?(name)
+    
     # Process the rest of the DSL
     to.from_block block if block
     to
@@ -201,67 +201,6 @@ class Tryouts
   # NOTE: this is a standalone DSL-syntax method.
   def self.xtryout(*args, &block); end
   
-  # Load dreams from a file or directory or if a block is given
-  # it's processed 
-  # Raises a Tryouts::BadDreams exception when something goes awry. 
-  #
-  # This method is used in two ways:
-  # * In the dreams file DSL
-  # * As a getter method on a Tryouts object
-  def dreams(tryout_name=nil, &definition)
-    return @dreams unless tryout_name
-
-    #
-    # dreams "path/2/dreams"
-    #  OR
-    # dreams "path/2/file_of_dreams.rb"
-    #
-    if File.exists?(tryout_name)
-      dfile = tryout_name
-      # If we're given a directory we'll build the filename using the class name
-      if File.directory?(tryout_name)
-        dfile = self.class.find_dreams_file(tryout_name, @group) 
-      end
-      raise BadDreams, "Cannot find dreams file (#{tryout_name})" unless dfile
-      @dreams = load_dreams_file( dfile) || {}
-    
-    #
-    # dreams "Tryout Name" do
-    #   dream "drill name" ...
-    # end
-    #
-    elsif tryout_name.kind_of?(String) && definition  
-      to = find_tryout(tryout_name, @dtype)
-      
-      if to.nil?
-        @dream_pointer = tryout_name  # Used in Tryouts.dream
-        @dreams[ @dream_pointer ] ||= {}
-        definition.call
-      else
-        to.from_block &definition
-      end
-    else
-      raise BadDreams, tryout_name
-    end
-    @dreams
-  end
-  # Without arguments, returns a Hash of all known dreams.
-  # With arguments, it calls Tryouts#dreams on the current instance of Tryouts. 
-  #
-  # NOTE: this is a standalone DSL-syntax method.
-  def self.dreams(*args, &block)
-    if args.empty? && block.nil?
-      dreams = {}
-      @@instances.each_pair do |name,inst|
-        dreams[name] = inst.dreams
-      end
-      return dreams
-    else
-      # Call the Tryouts#dreams instance method
-      @@instances.last.dreams(*args, &block)
-    end
-  end
-  
   # Returns +@tryouts+.
   #
   # Also acts as a stub for Tryouts#tryout in case someone 
@@ -275,66 +214,6 @@ class Tryouts
     tryout(args, &block)
   end
   
-  # +name+ of the Drill associated to this Dream
-  # +output+ A String or Array of expected output. A Dream object will be created using this value (optional)
-  # +definition+ is a block which will be run on an instance of Dream
-  #
-  # This method is different than Tryout#dream because this one stores
-  # dreams inside an instance variable of the current Tryouts object.
-  # This allows for the situation where the dreams block appears before
-  # the tryout block. See Tryouts#tryout
-  #
-  # NOTE: This method is DSL-only. It's not intended to be used in OO syntax. 
-  def dream(name, output=nil, format=nil, rcode=0, emsg=nil, &definition)
-    to = find_tryout(@dream_pointer, @dtype)
-    if to.nil?
-      if output.nil?
-        dobj = Tryouts::Drill::Dream.from_block definition
-      else
-        dobj = Tryouts::Drill::Dream.new(output)
-        dobj.format, dobj.rcode, dobj.emsg = format, rcode, emsg
-      end
-      @dreams[@dream_pointer][name] = dobj
-    else
-      # Let the Tryout object process the dream DSL.
-      # We'll get here if the dream is placed after 
-      # the drill with the same name in the same block.
-      to.dream name, output, format, rcode, emsg, &definition
-    end
-  end
-  # Calls Tryouts#dream on the current instance of Tryouts
-  #
-  # NOTE: this is a standalone DSL-syntax method.
-  def self.dream(*args, &block)
-    @@instances.last.dream(*args, &block)
-  end
-  
-  # This method does nothing. It provides a quick way to disable a dream.
-  #
-  # NOTE: This is a DSL-only method and is not intended for OO use.
-  def xdream(*args, &block); end
-  # This method does nothing. It provides a quick way to disable a dream.
-  #
-  # NOTE: this is a standalone DSL-syntax method.
-  def self.xdream(*args, &block); end
-  
-  # Populate @@dreams with the content of the file +dpath+. 
-  #
-  # NOTE: this is an OO syntax method
-  def load_dreams_file(dpath)
-    type = File.extname dpath
-    if type == ".yaml" || type == ".yml"
-      @dreams = YAML.load_file dpath
-    elsif type == ".json" || type == ".js"
-      @dreams = JSON.load_file dpath
-    elsif type == ".rb"
-      @dreams = instance_eval File.read(dpath)
-    else
-      raise BadDreams, "Unknown kind of dream: #{dpath}"
-    end
-    @dreams
-  end
-  
   # Parse a +_tryouts.rb+ file. See Tryouts::CLI::Run for an example. 
   #
   # NOTE: this is an OO syntax method
@@ -387,27 +266,5 @@ class Tryouts
   ## end
   ##+++
   
-  # Find a dreams file in the directory +dir+ based on the current group name.
-  # The expected filename format is: groupname_dreams.ext where "groupname" is
-  # the lowercase name of the Tryouts group (spaces removed) and "ext" is one 
-  # of: yaml, js, json, rb. 
-  #
-  #     e.g.
-  #     Tryouts.find_dreams_file "dirpath"   # => dirpath/tryouts_dreams.rb
-  #
-  def self.find_dreams_file(dir, group=nil)
-    dpath = nil
-    group ||= @@instances.last.group
-    group = group.to_s.downcase.tr(' ', '')
-    [:rb, :yaml].each do |ext|
-      tmp = File.join(dir, "#{group}_dreams.#{ext}")
-      if File.exists?(tmp)
-        dpath = tmp
-        break
-      end
-    end
-    dpath
-  end
-  
 
 end

data/lib/tryouts/cli/run.rb

@@ -14,7 +14,7 @@ class Run < Drydock::Command
   # Display the dreams from all known tryouts
   def dreams
     load_available_tryouts_files
-    if @global.verbose > 0
+    if Tryouts.verbose > 0
       puts Tryouts.dreams.to_yaml
     else
       Tryouts.dreams.each_pair do |n,dreams|
@@ -32,16 +32,18 @@ class Run < Drydock::Command
   # $ sergeant run [path/2/tryouts]
   # Executes all tryouts that can be found from the current working directory. 
   def run
-    if @global.verbose > 0
-      puts "#{Tryouts.sysinfo.to_s} (#{RUBY_VERSION})"
-    end
+    
+    Tryouts.enable_debug if Drydock.debug?
+    Tryouts.verbose = @global.verbose
+    
+    puts "#{Tryouts.sysinfo.to_s} (#{RUBY_VERSION})" if Tryouts.verbose > 0
     
     load_available_tryouts_files
 
     passed, failed = 0, 0
     Tryouts.instances.each_pair do |group,tryouts_inst|
       puts '', ' %-60s'.att(:reverse) % group
-      puts "  #{tryouts_inst.paths.join("\n  ")}" if @global.verbose > 0
+      puts "  #{tryouts_inst.paths.join("\n  ")}" if Tryouts.verbose > 0
       tryouts_inst.tryouts.each_pair do |name,to|
         to.run
         to.report
@@ -52,14 +54,14 @@ class Run < Drydock::Command
     end
     unless @global.quiet
       if (passed == 0 && failed == 0)
-        puts DEV if @global.verbose > 4
+        puts DEV if Tryouts.verbose > 4
         msg = " You didn't even try to acheive your dreams :[ "
       elsif failed == 0
-        puts PUG if @global.verbose > 4
+        puts PUG if Tryouts.verbose > 4
         msg = " All %s dreams came true ".color(:green)
         msg = msg % [passed+failed]
       else
-        puts BUG if @global.verbose > 4
+        puts BUG if Tryouts.verbose > 4
         score = (passed.to_f / (passed.to_f+failed.to_f)) * 100
         msg = " %s of %s dreams came true (%d%%) ".color(:red)
         msg = msg % [passed, passed+failed, score.to_i]
@@ -74,7 +76,7 @@ class Run < Drydock::Command
     load_available_tryouts_files
     Tryouts.instances.each_pair do |n,tryouts_inst|
       puts n
-      if @global.verbose > 0
+      if Tryouts.verbose > 0
         puts "  #{tryouts_inst.paths.join("\n  ")}"
       end
       tryouts_inst.tryouts.each_pair do |t2,tryout|
@@ -104,7 +106,7 @@ private
       end
     end
     @tryouts_files.uniq!  # Don't load the same file twice
-    @tryouts_files.each { |f| puts "LOADING: #{f}"} if @global.verbose > 0
+    @tryouts_files.each { |f| puts "LOADING: #{f}"} if Tryouts.verbose > 1
     @tryouts_files.each { |file| Tryouts.parse_file file }
   end
 end

data/lib/tryouts/drill.rb

@@ -7,7 +7,8 @@ class Tryouts
   # This class represents a drill. A drill is single test. 
   #
   class Drill
-  
+    
+  require 'tryouts/drill/context'
   require 'tryouts/drill/response'
   require 'tryouts/drill/sergeant/cli'
   require 'tryouts/drill/sergeant/api'
@@ -28,22 +29,22 @@ class Tryouts
     # A Reality object (the actual output of the test)
   attr_reader :reality
       
-  def initialize(name, dtype, *drill_args, &drill)
+  def initialize(name, dtype, *opts, &drill)
     @name, @dtype, @drill = name, dtype, drill
-    @sergeant = hire_sergeant *drill_args
+    @sergeant = hire_sergeant opts
     # For CLI drills, a block takes precedence over inline args. 
     # A block will contain multiple shell commands (see Rye::Box#batch)
     drill_args = [] if dtype == :cli && drill.is_a?(Proc)
     @reality = Tryouts::Drill::Reality.new
   end
   
-  def hire_sergeant(*drill_args)
+  def hire_sergeant(opts={})
     if @dtype == :cli
-      Tryouts::Drill::Sergeant::CLI.new(*drill_args)
+      Tryouts::Drill::Sergeant::CLI.new(*opts)
     elsif @dtype == :api
-      Tryouts::Drill::Sergeant::API.new(drill_args.first)
+      Tryouts::Drill::Sergeant::API.new(opts.first) # should be a hash
     else
-      raise NoSergeant, "What is #{@dtype}?"
+      raise NoSergeant, "Weird drill sergeant: #{@dtype}"
     end
   end
   
@@ -53,15 +54,6 @@ class Tryouts
       @reality = @sergeant.run @drill, context
       # Store the stash from the drill block
       @reality.stash = context.stash if context.respond_to? :stash
-      # Create or overwrite an existing dream if onw was defined in the block
-      if context.respond_to?(:dream) && context.has_dream?
-        @dream = Tryouts::Drill::Dream.new
-        @dream.output = context.dream
-        @dream.format = context.format unless context.format.nil?
-        @dream.rcode = context.rcode unless context.rcode.nil?
-        @dream.emsg = context.emsg unless context.emsg.nil?
-      end
-      
       # If the drill block returned true we assume success if there's no dream
       if @dream.nil? && @reality.output == true
         @dream = Tryouts::Drill::Dream.new
@@ -69,45 +61,31 @@ class Tryouts
       end
       process_reality
     rescue => ex
-      @reality.rcode = -2
-      @reality.emsg, @reality.backtrace = ex.message, ex.backtrace
+      @reality.ecode, @reality.etype = -2, ex.class
+      @reality.error, @reality.trace = ex.message, ex.backtrace
     end  
-    note = @dream ? discrepency.join(', ') : 'nodream'
-    puts self.success? ? "PASS".color(:green) : "FAIL (#{note})".color(:red)
     self.success?
   end
   
   def success?
-    return false if @dream.nil? || @reality.nil?
     @dream == @reality
   end
   
-  def discrepency
-    diffs = []
-    if @dream
-      diffs << "rcode" if @dream.rcode != @reality.rcode
-      diffs << "output" if !@dream.compare_output(@reality)
-      diffs << "emsg" if @dream.emsg != @reality.emsg
-    end
-    diffs
-  end
   
   def add_dream(d)
-    @dream = d if d.is_a?(Tryouts::Drill::Dream)
+    @dream = d 
   end
   
   private 
+  
   # Use the :format provided in the dream to convert the output from reality
   def process_reality
-    @reality.normalize!
-    return unless @dream && @dream.format
-    if @dream.format == :to_yaml
-      @reality.output = YAML.load(@reality.output.join("\n"))
-    elsif  @dream.format == :to_json
-      @reality.output = JSON.load(@reality.output.join("\n"))
+    case @dream.format
+    when :class
+      @reality.output = @reality.output.class
+    when :exception
+      @reality.output = @reality.etype
     end
-    
-    #p [:process, @name, @dream.format, @reality.output]
   end
   
 end; end

data/lib/tryouts/drill/context.rb

@@ -0,0 +1,32 @@
+
+class Tryouts
+  class Tryout
+    
+    # All :api Drills are run within this context (not used for :cli). 
+    # Each Drill is executed in a new instance of this class. That means
+    # instance variables are not carried through, but class variables are. 
+    # The before and after blocks are also run in this context.
+    class DrillContext
+        # An ordered Hash of stashed objects. 
+      attr_writer :stash
+
+      def initialize; @stash = Tryouts::HASH_TYPE.new; @has_dream = false; end
+
+      # Set to to true by DrillContext#dream
+      def has_dream?; @has_dream; end
+
+      # If called with no arguments, returns +@stash+. 
+      # If called with arguments, it will add a new value to the +@stash+
+      # and return the new value.  e.g.
+      #
+      #     stash :name, 'some value'   # => 'some value'
+      #
+      def stash(*args)
+        return @stash if args.empty?
+        @stash[args[0]] = args[1] 
+        args[1] 
+      end
+      
+    end
+  end
+end

data/lib/tryouts/drill/response.rb

@@ -5,66 +5,23 @@ class Tryouts::Drill
   # A generic base class for Dream and Reality
   #
   class Response
-    attr_accessor :output, :format, :rcode, :emsg, :backtrace
-    def initialize(output=nil, format=nil, rcode=0)
-      @output, @format, @rcode = output, format, (rcode || 0)
-      @output = nil if @output.nil?
-      normalize!
+    attr_accessor :output, :format
+    def initialize(output=nil, format=nil)
+      @output, @format = output, format
     end
     
-    def ==(other)
-      return false if other.nil?
-      @rcode == other.rcode &&
-      @emsg == other.emsg &&
-      compare_output(other)
-    end
-    
-    def rcode(val=nil); @rcode = val unless val.nil?; normalize!; @rcode; end
-    def output(val=nil); @output = val unless val.nil?; normalize!; @output; end
-    def emsg(val=nil); @emsg = val unless val.nil?; normalize!; @emsg; end
-    def format(val=nil); @format = val unless val.nil?; normalize!; @format; end
-    
-    def output=(val); @output = val; normalize!; @output; end
-    def rcode=(val); @rcode = val; normalize!; @rcode; end
-    def format=(val); @format = val; normalize!; @format; end
-    def emsg=(val); @emsg = val; normalize!; @emsg; end
+    def format(val=nil); @format = val.to_sym unless val.nil?; @format; end
+    def format=(val); @format = val.to_sym; @format; end
     
-    # Enforce the following restrictions on the data fields:
-    # * +rcode+ is an Integer
-    # * +format+ is a Symbol
-    # This method is called automatically any time a field is updated.
-    def normalize!
-      @rcode = @rcode.to_i if @rcode.is_a?(String)
-      @format = @format.to_sym if @format.is_a?(String)
-    end
-    def compare_output(other)
-      #
-      # The dream is always on the left (Does your dream match reality?)
-      # This check is important so we can support both:
-      # @dream == @reality
-      #  AND
-      # @reality == @dream
-      #
-      if self.is_a? Tryouts::Drill::Dream
-        dream, reality = self, other
-      elsif self.is_a? Tryouts::Drill::Reality
-        dream, reality = other, self
-      else
-        # If self isn't a Dream or a Reality, then we have a problem
-        return false
-      end
+    def Response.compare(dream, reality)
+      return false if reality.nil?
       
-      # The matching statement will be the return value.         
-      if dream.format.nil? || (dream.class == reality.class)
-        dream.output == reality.output
-      elsif reality.respond_to? dream.format
-        reality.output.send(dream.format) == dream.output 
-      else
-        false
-      end
+      ## I don't think this check is necessary or useful
+      ##return false unless reality.error.nil? && reality.trace.nil?
+      return true if reality.output == true and dream.nil?
+      reality.output == dream.output
       
     end
-
   end
 
 
@@ -82,18 +39,20 @@ class Tryouts::Drill
     
     def from_block(definition)
       instance_eval &definition
-      self.normalize!
       self
     end
     
-    # Takes a String +val+ and splits the lines into an Array. Each line
-    # has 
+    # Takes a String +val+ and splits the lines into an Array.
     def inline(val=nil)
       lines = (val.split($/) || [])
       lines.shift if lines.first.strip == ""
       lines.pop if lines.last.strip == ""
       lines
     end
+    
+    def ==(reality)
+      Response.compare(self, reality)
+    end
   end
 
   # = Reality 
@@ -103,10 +62,17 @@ class Tryouts::Drill
   class Reality < Tryouts::Drill::Response
       # An ordered hash taken from the DrillContext that created this Reality. 
     attr_accessor :stash
+    attr_accessor :error
+    attr_accessor :trace
+    attr_accessor :ecode
+    attr_accessor :etype
     def initialize
-      @rcode = 0
       @stash = Tryouts::HASH_TYPE.new
     end
+    
+    def ==(dream)
+      Response.compare(dream, self)
+    end
   end
 
 end

data/lib/tryouts/drill/sergeant/api.rb

@@ -9,30 +9,30 @@ class Tryouts; class Drill; module Sergeant
   #
   class API
     
-    # +return_value+ specify a return value. This will be 
+    attr_reader :opts
+    
+    # +opts+ is a Hash with the following optional keys:
+    #
+    # * +:output+ specify a return value. This will be 
     # used if no block is specified for the drill.
-    def initialize(return_value=nil)
-      @return_value = return_value
+    def initialize(opts={})
+      @opts = opts
     end
   
     def run(block, context, &inline)
-      
       # A Proc object takes precedence over an inline block. 
       runtime = (block.nil? ? inline : block)
       response = Tryouts::Drill::Reality.new
       if runtime.nil?
-        response.output = @return_value
+        response.output = @opts[:output]
       else
         begin
-          unless runtime.nil?
-            ret = context.instance_eval &runtime
-            response.output, response.rcode = ret, 0
-          end
-        rescue => ex
-          response.rcode = 1
-          response.output = ret
-          response.emsg = ex.message
-          response.backtrace = ex.backtrace
+          response.output = context.instance_eval &runtime
+        rescue => e
+          puts e.message, e.backtrace if Tryouts.debug? && Tryouts.verbose > 2
+          response.etype = e.class
+          response.error = e.message
+          response.trace = e.backtrace
         end
       end
       response

data/lib/tryouts/drill/sergeant/cli.rb

@@ -25,21 +25,30 @@ class Tryouts; class Drill; module Sergeant
       response = Tryouts::Drill::Reality.new
       begin
         if runtime.nil?
-          ret = @rbox.send *rbox_args
+          ret = @rbox.send *@rbox_args
         else
           ret = @rbox.instance_eval &runtime
         end
-        response.rcode = ret.exit_code
-        response.output = ret.stdout.size == 1 ? ret.stdout.first : Array.new(ret.stdout)  # Cast the Rye::Rap object
-        response.emsg = ret.stderr unless ret.stderr.empty?
+        response.ecode = ret.exit_code
+        if ret.stdout.size == 1
+          response.output = ret.stdout.first 
+        else
+          response.output = Array.new(ret.stdout)  # Cast the Rye::Rap object
+        end
+        response.error = ret.stderr unless ret.stderr.empty?
       rescue Rye::CommandNotFound => ex
-        response.rcode = -2
-        response.emsg = "[#{@rbox.host}] Command not found: #{ex.message}"
-        response.backtrace = ex.backtrace
+        puts ex.message, ex.backtrace if Tryouts.debug? && Tryouts.verbose > 2
+        response.etype = ex.class
+        response.ecode = ex.exit_code
+        response.error = "[#{@rbox.host}] Command not found: #{ex.message}"
+        response.trace = ex.backtrace
       rescue Rye::CommandError => ex
-        response.rcode = ex.exit_code
+        puts ex.message, ex.backtrace if Tryouts.debug? && Tryouts.verbose > 2
+        response.etype = ex.class
+        response.ecode = ex.exit_code
         response.output = ex.stdout
-        response.emsg = ex.stderr
+        response.error = ex.stderr.join $/
+        response.trace = ex.backtrace
       end
       response
     end

data/lib/tryouts/tryout.rb

@@ -26,74 +26,15 @@ class Tryouts
     # For drill type :api, this attribute is ignored. 
   attr_reader :command
     # A Hash of Dream objects for this Tryout. The keys are drill names. 
-  attr_accessor :dreams
+  attr_reader :dream_catcher
   
   @@valid_dtypes = [:cli, :api]
-  
-  # All :api Drills are run within this context (not used for :cli). 
-  # Each Drill is executed in a new instance of this class. That means
-  # instance variables are not carried through, but class variables are. 
-  # The before and after blocks are also run in this context.
-  class DrillContext
-      # An ordered Hash of stashed objects. 
-    attr_writer :stash
-      # A value used as the dream output that will overwrite a predefined dream
-    attr_writer :dream
-    attr_writer :format
-    attr_writer :rcode
-    attr_writer :emsg
-    attr_writer :output
-    
-    def initialize; @stash = Tryouts::HASH_TYPE.new; @has_dream = false; end
-    
-    # Set to to true by DrillContext#dream
-    def has_dream?; @has_dream; end
-    
-    # If called with no arguments, returns +@stash+. 
-    # If called with arguments, it will add a new value to the +@stash+
-    # and return the new value.  e.g.
-    #
-    #     stash :name, 'some value'   # => 'some value'
-    #
-    def stash(*args)
-      return @stash if args.empty?
-      @stash[args[0]] = args[1] 
-      args[1] 
-    end
-    
-    # If called with no arguments, returns +@dream+. 
-    # If called with one argument, it will overwrite +@dream+ with the
-    # first element. If called with two arguments, it will check if
-    # the second argument is a Symbol or Fixnum. If it's a Symbol it 
-    # will assume it's +@format+. If it's a Fixnum, it will assume 
-    # it's +@rcode+. If there's a there's a third argument and it's a
-    # Fixnum, it's assumed to be +@rcode+. In all cases, this method 
-    # returns the value of +@dream+. e.g.
-    #
-    #     dream 'some value'         # => 'some value'
-    #     dream :val1, :class, 1     # => :val1
-    #
-    def dream(*args)
-      return @dream if args.empty?
-      @has_dream = true
-      @dream = args.shift
-      @format = args.shift if args.first.is_a? Symbol
-      @rcode = args.shift if args.first.is_a? Fixnum
-      @emsg = args.shift if args.first.is_a? String
-    end
-    
-    def output(*args); return @output if args.empty?; @output = args.first; end
-    def format(*args); return @format if args.empty?; @format = args.first; end
-    def rcode(*args); return @rcode if args.empty?; @rcode = args.first; end
-    def emsg(*args); return @emsg if args.empty?; @emsg = args.first; end
-    
-  end
      
   def initialize(name, dtype, command=nil, *args)
     raise "Must supply command for dtype :cli" if dtype == :cli && command.nil?
     raise "#{dtype} is not a valid drill type" if !@@valid_dtypes.member?(dtype)
     @name, @dtype, @command = name, dtype, command
-    @drills, @dreams = [], {}
+    @drills, @dream_catcher = [], []
     @passed, @failed = 0, 0
   end
   
@@ -108,13 +49,17 @@ class Tryouts
   
   # Execute all Drill objects
   def run
-    update_drills!   # Ensure all drills have all known dreams
     DrillContext.module_eval &setup if setup.is_a?(Proc)
     puts Tryouts::TRYOUT_MSG.bright % @name
     @drills.each do |drill|
-      drill.run(DrillContext.new)      # Returns true or false
-      drill.reality.stash.each_pair do |n,v|
-        puts '%14s: %s' % [n,v.inspect]
+      drill.run DrillContext.new
+      note = @dream ? '' : '(nodream)'
+      puts drill.success? ? "PASS".color(:green) : "FAIL #{note}".color(:red)
+      puts "      #{drill.reality.output.inspect}" if Tryouts.verbose > 0
+      if Tryouts.verbose > 1
+        drill.reality.stash.each_pair do |n,v|
+          puts '%14s: %s' % [n,v.inspect]
+        end
       end
       drill.success? ? @passed += 1 : @failed += 1
     end
@@ -126,25 +71,23 @@ class Tryouts
     return true if success?
     failed = @drills.select { |d| !d.success? }
     failed.each_with_index do |drill,index|
+      dream, reality = drill.dream, drill.reality
       title = ' %-59s' % %Q{ERROR #{index+1}/#{failed.size} "#{drill.name}"}
       puts $/, ' ' << title.color(:red).att(:reverse)
       
-      if drill.dream
-        puts '%24s: %s (expected %s)' % ["response code", drill.reality.rcode, drill.dream.rcode]
-        puts '%24s: %s' % ["expected output", drill.dream.output.inspect]
-        puts '%24s: %s' % ["actual output", drill.reality.output.inspect]
-        if drill.reality.emsg || (drill.reality.emsg != drill.dream.emsg)
-          puts '%24s: %s' % ["expected error msg", drill.dream.emsg.inspect]
-            puts '%24s: %s' % ["actual error msg", drill.reality.emsg.inspect]
+      if dream
+        puts '%12s: %s' % [ "expected", dream.output.inspect]
+        puts '%12s: %s' % ["returned", reality.output.inspect]
+        unless reality.error.nil?
+          puts '%12s: %s' % ["error", reality.error.inspect]
         end
-        
-        if drill.reality.rcode > 0
-          puts '%24s: ' % ["backtrace"]
-          puts drill.reality.backtrace, $/
+        unless reality.trace.nil?
+          puts '%12s: %s' % ["trace", reality.trace.join($/ + ' '*14)]
+          puts
         end
       else
-        puts '%24s: %s' % ["expected output", "[nodream]"]
-        puts '%24s: %s' % ["actual output", drill.reality.output.inspect]
+        puts '%12s: %s' % ["expected", "[nodream]"]
+        puts '%12s: %s' % ["returned", reality.output.inspect]
       end
       
     end
@@ -158,26 +101,15 @@ class Tryouts
     @success = !(@drills.collect { |r| r.success? }.member?(false))
   end
   
-  # Add a Drill object to the list for this Tryout. If there is a dream
-  # defined with the same name as the Drill, that dream will be given to
-  # the Drill before its added to the list. 
+  # Add a Drill object to the list for this Tryout. If there is one or
+  # more dreams in +@dream_catcher+, it will be added to drill +d+. 
   def add_drill(d)
-    d.add_dream @dreams[d.name] if !@dreams.nil? && @dreams.has_key?(d.name)
-    drills << d if d.is_a?(Tryouts::Drill)
-    d
-  end
-  
-  # Goes through the list of Drill objects (@drills) and gives each 
-  # one its associated Dream object (if available). 
-  # 
-  # This method is called before Tryout#run, but is only necessary in  
-  # the case where dreams where loaded after the drills were defined. 
-  def update_drills!
-    return if @dreams.nil?
-    @drills.each do |drill|
-      next unless @dreams.has_key?(drill.name)
-      drill.add_dream @dreams[drill.name]
+    unless @dream_catcher.empty?
+      d.add_dream @dream_catcher.first
+      @dream_catcher.clear
     end
+    drills << d
+    d
   end
   
   ## ---------------------------------------  EXTERNAL DSL  -----
@@ -199,28 +131,30 @@ class Tryouts
   # +args+ is sent directly to the Drill class. The values are specific on the Sergeant.
   def drill(dname, *args, &definition)
     raise "Empty drill name (#{@name})" if dname.nil? || dname.empty?
-    args.unshift(@command) if @dtype == :cli
-    drill = Tryouts::Drill.new(dname, @dtype, *args, &definition)
-    add_drill drill
+    if definition.nil?
+      drill = Tryouts::Drill.new(dname, @dtype, :output => args.first)
+    else
+      drill = Tryouts::Drill.new(dname, @dtype, args.first, &definition)
+    end
+    self.add_drill drill
   end
   # A quick way to comment out a drill
   def xdrill(*args, &b); end # ignore calls to xdrill
   
-  # +name+ of the Drill associated to this Dream
-  # +output+ A String or Array of expected output. A Dream object will be created using this value (optional)
-  # +definition+ is a block which will be run on an instance of Dream
+  
   #
   # NOTE: This method is DSL-only. It's not intended to be used in OO syntax. 
-  def dream(dname, output=nil, format=nil, rcode=0, emsg=nil, &definition) 
-    raise "Empty dream name (#{@name})" if dname.nil? || dname.empty?
-    if output.nil?
-      raise "No output or block for '#{dname}' (#{@name})" if definition.nil?
+  def dream(*args, &definition) 
+    if args.empty?
       dobj = Tryouts::Drill::Dream.from_block definition
     else
-      dobj = Tryouts::Drill::Dream.new(output)
-      dobj.format, dobj.rcode, dobj.emsg = format, rcode, emsg
+      if args.size == 1
+        dobj = Tryouts::Drill::Dream.new(args.shift)    # dream 'OUTPUT'
+      else
+        dobj = Tryouts::Drill::Dream.new(*args.reverse) # dream :form, 'OUTPUT'
+      end
     end
-    @dreams[dname] = dobj
+    @dream_catcher << dobj
     dobj
   end
   # A quick way to comment out a dream

data/tryouts.gemspec

@@ -0,0 +1,75 @@
+@spec = Gem::Specification.new do |s|
+	s.name = "tryouts"
+  s.rubyforge_project = "tryouts"
+	s.version = "0.6.0"
+	s.summary = "Tryouts are high-level tests for your Ruby code. May all your dreams come true!"
+	s.description = s.summary
+	s.author = "Delano Mandelbaum"
+	s.email = "tryouts@solutious.com"
+	s.homepage = "http://github.com/delano/tryouts"
+  
+  # = EXECUTABLES =
+  # The list of executables in your project (if any). Don't include the path, 
+  # just the base filename.
+  s.executables = %w[sergeant]
+  
+  # Directories to extract rdocs from
+  s.require_paths = %w[lib]  
+  
+  # Specific files to include rdocs from
+  s.extra_rdoc_files = %w[README.rdoc LICENSE.txt]
+  
+  # Update --main to reflect the default page to display
+  s.rdoc_options = ["--line-numbers", "--title", "Tryouts: #{s.summary}", "--main", "README.rdoc"]
+  
+  # = DEPENDENCIES =
+  # Add all gem dependencies
+  s.add_dependency 'drydock', '>= 0.6.5'
+  s.add_dependency 'rye', '>= 0.8.2'
+  s.add_dependency 'sysinfo', '>= 0.5.1'
+  
+  # = MANIFEST =
+  # The complete list of files to be included in the release. When GitHub packages your gem, 
+  # it doesn't allow you to run any command that accesses the filesystem. You will get an
+  # error. You can ask your VCS for the list of versioned files:
+  # git ls-files
+  # svn list -R
+  s.files = %w(
+  CHANGES.txt
+  LICENSE.txt
+  README.rdoc
+  Rakefile
+  bin/mockout
+  bin/sergeant
+  lib/tryouts.rb
+  lib/tryouts/cli.rb
+  lib/tryouts/cli/run.rb
+  lib/tryouts/drill.rb
+  lib/tryouts/drill/context.rb
+  lib/tryouts/drill/response.rb
+  lib/tryouts/drill/sergeant/api.rb
+  lib/tryouts/drill/sergeant/cli.rb
+  lib/tryouts/mixins.rb
+  lib/tryouts/mixins/hash.rb
+  lib/tryouts/orderedhash.rb
+  lib/tryouts/tryout.rb
+  tryouts.gemspec
+  )
+  
+  s.has_rdoc = true
+  s.rubygems_version = '1.3.0'
+
+  if s.respond_to? :specification_version then
+    current_version = Gem::Specification::CURRENT_SPECIFICATION_VERSION
+    s.specification_version = 2
+ 
+    if Gem::Version.new(Gem::RubyGemsVersion) >= Gem::Version.new('1.2.0') then
+      s.add_runtime_dependency(%q<RedCloth>, [">= 4.0.4"])
+    else
+      s.add_dependency(%q<RedCloth>, [">= 4.0.4"])
+    end
+  else
+    s.add_dependency(%q<RedCloth>, [">= 4.0.4"])
+  end
+  
+end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification 
 name: delano-tryouts
 version: !ruby/object:Gem::Version 
-  version: 0.5.1
+  version: 0.6.0
 platform: ruby
 authors: 
 - Delano Mandelbaum
@@ -30,7 +30,7 @@ dependencies:
     requirements: 
     - - ">="
       - !ruby/object:Gem::Version 
-        version: 0.6.6
+        version: 0.8.2
     version: 
 - !ruby/object:Gem::Dependency 
   name: sysinfo
@@ -72,6 +72,7 @@ files:
 - lib/tryouts/cli.rb
 - lib/tryouts/cli/run.rb
 - lib/tryouts/drill.rb
+- lib/tryouts/drill/context.rb
 - lib/tryouts/drill/response.rb
 - lib/tryouts/drill/sergeant/api.rb
 - lib/tryouts/drill/sergeant/cli.rb
@@ -79,6 +80,7 @@ files:
 - lib/tryouts/mixins/hash.rb
 - lib/tryouts/orderedhash.rb
 - lib/tryouts/tryout.rb
+- tryouts.gemspec
 has_rdoc: true
 homepage: http://github.com/delano/tryouts
 post_install_message: