tryouts 0.4.0

data/CHANGES.txt

@@ -0,0 +1,6 @@
+TRYOUTS, CHANGES
+
+
+#### 0.4.0 (2009-06-05) ###############################
+
+NOTE: Initial public release

data/LICENSE.txt

@@ -0,0 +1,19 @@
+Copyright (c) 2009 Solutious Inc, Delano Mandelbaum
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.rdoc

@@ -0,0 +1,122 @@
+= Tryouts - v0.5 ALPHA
+
+Tryouts is a high-level testing library for your command-line applications and Ruby codes.
+
+*WORK IN PROGRESS* 
+
+2009-05-24 - A BBQ started nearby as I was writing this documentation. It's incomplete and I'm sorry about that but I can tell you that this BBQ smells really, really good. 
+
+
+== Terminology
+
+Tryouts is a bit different than other testing libraries. Test definitions are organized in a similar way as Shoulda tests (although the keywords in the syntax are different). 
+
+* Tryout: a set of drills (like basketball tryouts)
+* Drill: a test. 
+* Dream: the expected outcome of a drill. 
+
+
+== Testing a command-line application (a CLI)
+
+Tryouts tests command-line applications by comparing expected output with the actual output. Let's say we have an executable called mockout and we want to test the following commands:
+
+    $ bin/executable
+    $ bin/executable -f yaml
+    
+The tryout definition would look like this:
+
+    command :executable, "path/2/executable"
+    
+    tryout "Common Usage", :cli do
+      drill  "No Command"
+      drill "YAML Output", :f, 'yaml'
+    end
+    
+And the expected output would be defined like this:
+
+    dream "No Command" do
+      output inline(%Q{
+            Date:                                   2009-02-16
+         Players:            d-bam, alberta, birds, condor man
+          Owners:            greg, rupaul, telly, prince kinko
+      })
+    end
+    dream "YAML Output" do
+      format :yaml
+      output ({
+        "Date" => "2009-02-16",
+        "Players" => ["d-bam", "alberta", "birds", "condor man"],
+        "Owners" => ["greg", "rupaul", "telly", "prince kinko"]
+      })
+    end
+    
+== Testing Ruby codes (an API)
+
+Tryouts employs the same approach for testing Ruby codes. The return value of the drill block is compared to the expectation defined by the dream. Here is an example of including dreams inside the tryout definition. 
+
+    library :caesars, LIBRARY_PATH
+
+    tryout "Common Usage", :api do
+      dream "Some Maths", 3 
+      drill "Some Maths" do
+        12 / 4
+      end
+        
+      dream "A Block", Proc, :class  # Test the class type instead of the value
+      drill "A Block" do
+        Proc.new do
+          :anything
+        end
+      end
+    end
+
+
+== ALPHA Notice
+
+This library is very new (est. 2009-05-19) and has not been vetted by the scrutiny of time. In particular you can expect:
+
+* Ugly/awkward output from the tryouts executable. I wrote the core functionality that I needed to start writing tryouts for my other projects. I haven't spent very much time on reporting yet. However, this will change!
+* The test definition syntax may change in future releases. 
+* Unexpected errors. 
+
+== 3 Ways to define tryouts
+There are three ways to define an instance of this class:
+* In +_tryouts.rb+ files using the DSL syntax. One file per Tryouts object. 
+  * See: http://github.com/delano/tryouts/blob/master/tryouts/mockoutcli_tryouts.rb
+* In standalone ruby files using a hybrid DSL/OO syntax. Supports multiple 
+  Tryouts objects per file. 
+  * See: http://github.com/delano/tryouts/blob/master/tryouts/standalone_test.rb
+* With regular object-oriented syntax.
+  * See: http://tryouts.rubyforge.org/
+
+== On Threads
+
+Tryouts does some funky stuff to make it simple to write tests. This "funky 
+stuff" means that this library is *not thread-safe at definition-time*. However, once 
+all tryouts files are parsed (or in OO-syntax, once all objects are created), this
+class should be *thread-safe at drill-time*. 
+
+== More Info
+
+* Check out the sourcecodes[http://github.com/delano/tryouts]
+* Read the rdocs[http://tryouts.rubyforge.org/]
+* About Solutious[http://solutious.com/about/]
+
+== Thanks
+
+* Everyone at Utrecht.rb 
+
+== Credits
+
+* Delano (@solutious.com)
+
+== Related Projects
+
+* Context[http://github.com/jeremymcanally/context/tree/master]
+* Testy[http://github.com/ahoward/testy/tree/master]
+* Expectations[http://expectations.rubyforge.org/]
+* Zebra[http://github.com/giraffesoft/zebra/tree/master]
+
+== License
+
+See: LICENSE.txt

data/Rakefile

@@ -0,0 +1,80 @@
+require 'rubygems'
+require 'rake/clean'
+require 'rake/gempackagetask'
+require 'hanna/rdoctask'
+require 'fileutils'
+include FileUtils
+ 
+task :default => :package
+ 
+# CONFIG =============================================================
+
+# Change the following according to your needs
+README = "README.rdoc"
+CHANGES = "CHANGES.txt"
+LICENSE = "LICENSE.txt"
+
+# Files and directories to be deleted when you run "rake clean"
+CLEAN.include [ 'pkg', '*.gem', '.config', 'doc']
+
+# Virginia assumes your project and gemspec have the same name
+name = (Dir.glob('*.gemspec') || ['tryouts']).first.split('.').first
+load "#{name}.gemspec"
+version = @spec.version
+
+# That's it! The following defaults should allow you to get started
+# on other things. 
+
+
+# TESTS/SPECS =========================================================
+
+
+
+# INSTALL =============================================================
+
+Rake::GemPackageTask.new(@spec) do |p|
+  p.need_tar = true if RUBY_PLATFORM !~ /mswin/
+end
+
+task :release => [ :rdoc, :package ]
+task :install => [ :rdoc, :package ] do
+	sh %{sudo gem install pkg/#{name}-#{version}.gem}
+end
+task :uninstall => [ :clean ] do
+	sh %{sudo gem uninstall #{name}}
+end
+
+
+# RUBYFORGE RELEASE / PUBLISH TASKS ==================================
+
+if @spec.rubyforge_project
+  desc 'Publish website to rubyforge'
+  task 'publish:rdoc' => 'doc/index.html' do
+    sh "scp -rp doc/* rubyforge.org:/var/www/gforge-projects/#{name}/"
+  end
+
+  desc 'Public release to rubyforge'
+  task 'publish:gem' => [:package] do |t|
+    sh <<-end
+      rubyforge add_release -o Any -a #{CHANGES} -f -n #{README} #{name} #{name} #{@spec.version} pkg/#{name}-#{@spec.version}.gem &&
+      rubyforge add_file -o Any -a #{CHANGES} -f -n #{README} #{name} #{name} #{@spec.version} pkg/#{name}-#{@spec.version}.tgz 
+    end
+  end
+end
+
+
+
+# RUBY DOCS TASK ==================================
+
+Rake::RDocTask.new do |t|
+	t.rdoc_dir = 'doc'
+	t.title    = @spec.summary
+	t.options << '--line-numbers' << '-A cattr_accessor=object'
+	t.options << '--charset' << 'utf-8'
+	t.rdoc_files.include(LICENSE)
+	t.rdoc_files.include(README)
+	t.rdoc_files.include(CHANGES)
+	t.rdoc_files.include('bin/tryouts')
+	t.rdoc_files.include('lib/**/*.rb')
+end
+

data/bin/mockout

@@ -0,0 +1,49 @@
+#!/usr/bin/ruby
+
+# = Mockout
+# 
+# This mock is used to generate test output for Tryouts itself.
+# It is otherwise uninteresting!
+#
+
+require 'rubygems'
+require 'drydock'
+require 'yaml'
+
+begin; require 'json'; rescue LoadError; end   # json may not be installed
+
+module DrillCLI 
+  extend Drydock
+  
+  global :f, :format, String, "One of: json, yaml, string (default)"
+  
+  
+  default :info
+  debug :on
+  
+  data = {
+    "Date" => "2009-02-16",
+    "Owners" => ["greg", "rupaul", "telly", "prince kinko"],
+    "Players" => ["d-bam", "alberta", "birds", "condor man"]
+  }
+  
+  option :e, :echo, "Echo the arguments"
+  command :info do |obj|
+    format = obj.global.format.nil? ? nil : "to_#{obj.global.format}"
+    format = nil if obj.global.format == 'string'
+    if obj.option.echo
+      puts obj.argv
+    else
+      if format.nil?
+        data.keys.sort.each do |n|
+          val = data[n]
+          val = val.join(', ') if val.is_a?(Array)
+          puts "%9s %44s" % ["#{n}:", val]
+        end
+      else
+        puts data.send(format)
+      end
+    end
+  end
+  
+end

data/bin/tryouts

@@ -0,0 +1,39 @@
+#!/usr/local/bin/ruby
+
+TRYOUTS_HOME = File.expand_path(File.join(File.dirname(__FILE__), '..'))
+
+local_libs = %w{tryouts net-scp amazon-ec2 aws-s3 caesars drydock rye storable sysinfo annoy}
+local_libs.each { |dir| $:.unshift File.join(TRYOUTS_HOME, '..', dir, 'lib') }
+
+require 'drydock'
+require 'tryouts'
+require 'tryouts/cli'
+
+# = TryoutsCLI
+#
+# This is the Drydock definition for the bin/tryouts executable
+module TryoutsCLI
+  extend Drydock
+  
+  debug :on
+  default :run, :with_args
+  
+  global :q, :quiet, "Decrease output"
+  global :v, :verbose, "Increase output" do 
+    @verbose ||= 0
+    @verbose += 1
+  end
+  
+  about "Run tryouts from current working directory"
+  argv :files
+  command :run => Tryouts::CLI::Run
+  
+  about "Show dreams available from the current working directory"
+  argv :files
+  command :dreams => Tryouts::CLI::Run
+  
+  about "Show tryouts available from the current working directory"
+  argv :files
+  command :list => Tryouts::CLI::Run
+  
+end

data/lib/tryouts/cli/run.rb

@@ -0,0 +1,94 @@
+
+class Tryouts; module CLI
+  
+  # = Run
+  #
+  # The logic bin/tryouts uses for running tryouts. 
+  class Run < Drydock::Command
+
+    def init
+      @tryouts_globs = [GYMNASIUM_GLOB, File.join(Dir.pwd, '*_tryouts.rb')]
+    end
+    
+    def dreams
+      load_available_tryouts_files
+      if @global.verbose > 0
+        puts Tryouts.dreams.to_yaml
+      else
+        Tryouts.dreams.each_pair do |n,dreams|
+          puts n
+          dreams.each_pair do |n, dream|
+            puts "  " << n
+            dream.each_pair do |n, drill|
+              puts "    " << n
+            end
+          end
+        end
+      end
+    end
+    
+    def run
+      if @global.verbose > 0
+        puts "RUBY #{RUBY_VERSION} - #{RUBY_PLATFORM}"
+      end
+      
+      load_available_tryouts_files
+
+      successes = []
+      Tryouts.instances.each_pair do |group,tryouts_inst|
+        puts '', '-'*60 unless @global.quiet
+        puts group
+        puts "  #{tryouts_inst.paths.join("\n  ")}" if @global.verbose > 0
+        tryouts_inst.tryouts.each_pair do |name,to|
+          to.run
+          to.report
+          STDOUT.flush
+          successes << to.success?
+        end
+      end
+      unless successes.member?(false)
+        puts $/, "All your dreams came true" unless @global.quiet
+      end
+    end
+    
+    def list
+      load_available_tryouts_files
+      ##if @global.verbose > 2
+      ##  puts Tryouts.instances.to_yaml   # BUG: Raises "can't dump anonymous class Class"
+      ##else
+        Tryouts.instances.each_pair do |n,tryouts_inst|
+          puts n
+          if @global.verbose > 0
+            puts "  #{tryouts_inst.paths.join("\n  ")}"
+          end
+          tryouts_inst.tryouts.each_pair do |t2,tryout|
+            puts "  " << tryout.name
+            tryout.drills.each do |drill|
+              puts "    " << drill.name
+            end
+          end
+        end
+      ##end
+    end
+    
+  private 
+    def load_available_tryouts_files
+      @tryouts_files = []
+      # If file paths were given, check those only. 
+      unless @argv.empty?
+        @argv.each do |file|
+          file = File.join(file, '**', '*_tryouts.rb') if File.directory?(file)
+          @tryouts_files += Dir.glob file
+        end
+      # Otherwise check the default globs
+      else
+        @tryouts_globs.each do |glob|
+          @tryouts_files += Dir.glob glob
+        end
+      end
+      @tryouts_files.uniq!  # Don't load the same file twice
+      @tryouts_files.each { |file| Tryouts.parse_file file }
+      puts @tryouts_files if @global.verbose > 0
+    end
+  end
+end; end

data/lib/tryouts/cli.rb

@@ -0,0 +1,15 @@
+
+class Tryouts
+  
+  # = CLI
+  # 
+  # A namespace for the tryouts executable (bin/tryouts) logic.
+  #
+  # This convention comes from Drydock (http://github.com/delano/drydock)
+  #
+  module CLI;end
+  
+end
+
+require 'tryouts/cli/run'
+

data/lib/tryouts/drill/response.rb

@@ -0,0 +1,105 @@
+
+class Tryouts::Drill
+  # = Response
+  #
+  # A generic base class for Dream and Reality
+  #
+  class Response
+    attr_accessor :output, :format, :rcode, :emsg, :backtrace
+    def initialize(output=nil, format=nil, rcode=0)
+      @output, @format, @rcode = output, format, (rcode || 0)
+      @output = nil if @output.nil?
+      normalize!
+    end
+    
+    def ==(other)
+      return false if other.nil?
+      @rcode == other.rcode &&
+      @emsg == other.emsg &&
+      compare_output(other)
+    end
+    
+    def rcode(val=nil); @rcode = val unless val.nil?; normalize!; @rcode; end
+    def output(val=nil); @output = val unless val.nil?; normalize!; @output; end
+    def emsg(val=nil); @emsg = val unless val.nil?; normalize!; @emsg; end
+    def format(val=nil); @format = val unless val.nil?; normalize!; @format; end
+    
+    def output=(val); @output = val; normalize!; @output; end
+    def rcode=(val); @rcode = val; normalize!; @rcode; end
+    def format=(val); @format = val; normalize!; @format; end
+    def emsg=(val); @emsg = val; normalize!; @emsg; end
+    
+    # Enforce the following restrictions on the data fields:
+    # * +rcode+ is an Integer
+    # * +format+ is a Symbol
+    # This method is called automatically any time a field is updated.
+    def normalize!
+      @rcode = @rcode.to_i if @rcode.is_a?(String)
+      @format = @format.to_sym if @format.is_a?(String)
+    end
+    def compare_output(other)
+      #
+      # The dream is always on the left (Does your dream match reality?)
+      # This check is important so we can support both:
+      # @dream == @reality
+      #  AND
+      # @reality == @dream
+      #
+      if self.is_a? Tryouts::Drill::Dream
+        dream, reality = self, other
+      elsif self.is_a? Tryouts::Drill::Reality
+        dream, reality = other, self
+      else
+        # If self isn't a Dream or a Reality, then we have a problem
+        return false
+      end
+      
+      # The matching statement will be the return value. 
+      if dream.format.nil?
+        dream.output == reality.output
+      elsif reality.respond_to? dream.format
+        reality.output.send(dream.format) == dream.output 
+      else
+        false
+      end
+      
+    end
+
+  end
+
+
+  # = Dream
+  #
+  # Contains the expected response of a Drill
+  #
+  class Dream < Tryouts::Drill::Response
+    
+    def self.from_block(definition)
+      d = Tryouts::Drill::Dream.new
+      d.from_block definition
+      d
+    end
+    
+    def from_block(definition)
+      instance_eval &definition
+      self.normalize!
+      self
+    end
+    
+    # Takes a String +val+ and splits the lines into an Array. Each line
+    # has 
+    def inline(val=nil)
+      lines = (val.split($/) || [])
+      lines.shift if lines.first.strip == ""
+      lines.pop if lines.last.strip == ""
+      lines
+    end
+  end
+
+  # = Reality 
+  #
+  # Contains the actual response of a Drill
+  #
+  class Reality < Tryouts::Drill::Response; end
+
+end

data/lib/tryouts/drill/sergeant/cli.rb

@@ -0,0 +1,49 @@
+
+
+class Tryouts; class Drill; module Sergeant
+  
+  # = CLI
+  # 
+  # The sergeant responsible for running command-line interface drills.
+  #
+  class CLI
+  
+    attr_reader :rbox
+    
+      # An Array of arguments to be sent to +rbox.send(*rbox_args)+
+    attr_accessor :rbox_args
+    
+    def initialize(*args)
+      @rbox_args = args
+      @rbox = Rye::Box.new
+    end
+  
+    # NOTE: Context is ignored for the CLI Sergeant. 
+    def run(block, context=nil, &inline)
+      # A Proc object takes precedence over an inline block. 
+      runtime = (block.nil? ? inline : block)
+      response = Tryouts::Drill::Reality.new
+      begin
+        if runtime.nil?
+          ret = @rbox.send *rbox_args
+        else
+          ret = @rbox.instance_eval &runtime
+        end
+        response.rcode = ret.exit_code
+        response.output = ret.stdout.size == 1 ? ret.stdout.first : Array.new(ret.stdout)  # Cast the Rye::Rap object
+        response.emsg = ret.stderr unless ret.stderr.empty?
+      rescue Rye::CommandNotFound => ex
+        response.rcode = -2
+        response.emsg = "[#{@rbox.host}] Command not found: #{ex.message}"
+        response.backtrace = ex.backtrace
+      rescue Rye::CommandError => ex
+        response.rcode = ex.exit_code
+        response.output = ex.stdout
+        response.emsg = ex.stderr
+      end
+      response
+    end
+    
+  end
+  
+end; end; end

data/lib/tryouts/drill.rb

@@ -0,0 +1,98 @@
+
+
+class Tryouts
+  
+  # = Drill
+  # 
+  # This class represents a drill. A drill is single test. 
+  #
+  class Drill
+  
+  require 'tryouts/drill/response'
+  require 'tryouts/drill/sergeant/cli'
+  require 'tryouts/drill/sergeant/api'
+  
+  class NoSergeant < Tryouts::Exception; end
+    
+    # A symbol specifying the drill type. One of: :cli, :api
+  attr_reader :dtype
+    # The name of the drill. This should match the name used in the dreams file. 
+  attr_reader :name
+    # A Proc object which contains the drill logic. 
+  attr_reader :drill
+  
+    # A Sergeant object which executes the drill
+  attr_reader :sergeant
+    # A Dream object (the expected output of the test)
+  attr_reader :dream
+    # A Reality object (the actual output of the test)
+  attr_reader :reality
+      
+  def initialize(name, dtype, *drill_args, &drill)
+    @name, @dtype, @drill = name, dtype, drill
+    @sergeant = hire_sergeant *drill_args
+    # For CLI drills, a block takes precedence over inline args. 
+    # A block will contain multiple shell commands (see Rye::Box#batch)
+    drill_args = [] if dtype == :cli && drill.is_a?(Proc)
+    @reality = Tryouts::Drill::Reality.new
+  end
+  
+  def hire_sergeant(*drill_args)
+    if @dtype == :cli
+      Tryouts::Drill::Sergeant::CLI.new(*drill_args)
+    elsif @dtype == :api
+      Tryouts::Drill::Sergeant::API.new(drill_args.first)
+    else
+      raise NoSergeant, "What is #{@dtype}?"
+    end
+  end
+  
+  def run(context=nil)
+    return false if @dream.nil?
+    begin
+      print Tryouts::DRILL_MSG % @name
+      @reality = @sergeant.run @drill, context
+      process_reality
+    rescue => ex
+      @reality.rcode = -2
+      @reality.emsg, @reality.backtrace = ex.message, ex.backtrace
+    end  
+    note = @dream ? discrepency.join(', ') : 'nodream'
+    puts self.success? ? "PASS" : "FAIL (#{note})"
+    self.success?
+  end
+  
+  def success?
+    return false if @dream.nil? || @reality.nil?
+    @dream == @reality
+  end
+  
+  def discrepency
+    diffs = []
+    if @dream
+      diffs << "rcode" if @dream.rcode != @reality.rcode
+      diffs << "output" if !@dream.compare_output(@reality)
+      diffs << "emsg" if @dream.emsg != @reality.emsg
+    end
+    diffs
+  end
+  
+  def add_dream(d)
+    @dream = d if d.is_a?(Tryouts::Drill::Dream)
+  end
+  
+  private 
+  # Use the :format provided in the dream to convert the output from reality
+  def process_reality
+    @reality.normalize!
+    return unless @dream && @dream.format
+    if @dream.format == :to_yaml
+      @reality.output = YAML.load(@reality.output.join("\n"))
+    elsif  @dream.format == :to_json
+      @reality.output = JSON.load(@reality.output.join("\n"))
+    end
+    
+    #p [:process, @name, @dream.format, @reality.output]
+  end
+  
+end; end