delano-tryouts 0.7.2 → 0.7.3

Sign up to get free protection for your applications and to get access to all the features.
data/CHANGES.txt CHANGED
@@ -1,6 +1,14 @@
1
1
  TRYOUTS, CHANGES
2
2
 
3
3
 
4
+ #### 0.7.3 (2009-06-27) ###############################
5
+
6
+ * ADDED: Display Tryouts group name for runtime errors.
7
+ * ADDED: Improved LoadError error message
8
+ * ADDED: Print error messages for drills that PASS.
9
+ * ADDED: Better error handling when supplied tryouts file is not found
10
+
11
+
4
12
  #### 0.7.2 (2009-06-26) ###############################
5
13
 
6
14
  NOTE: You will need to make a syntax change to your tryouts.
data/README.rdoc CHANGED
@@ -27,7 +27,7 @@ This is a return to the original syntax and I think it's the right way to go bec
27
27
  * <tt>drill 'test name', 'return value', :class, String</tt>
28
28
 
29
29
 
30
- == Examples
30
+ == Writing Tests
31
31
 
32
32
  The examples below are a complete overview of Tryouts syntax.
33
33
 
@@ -97,6 +97,45 @@ You can also use Tryouts to run benchmarks. The tryouts method takes a second pa
97
97
 
98
98
  The drill blocks in a benchmark Tryout return Tryouts::Stats objects. See: Tryouts::Drill::Sergeant::Benchmark
99
99
 
100
+ == Running Tests
101
+
102
+ Tryouts comes with an executable called <tt>sergeant</tt>. This is the drill sergeant that tells you what percentage of your dreams come true.
103
+
104
+ $ sergeant
105
+
106
+ Gibbler Gazette
107
+
108
+ Basic syntax with SHA1
109
+ "Object" PASS
110
+ "Class" PASS
111
+ ...
112
+ "Knows when an Hash has changed" PASS
113
+
114
+ All 9 dreams came true
115
+
116
+
117
+ The sergeant looks in the current working directory for a <tt>tryouts</tt> directory and will automatically load all files ending in <tt>_tryouts.rb</tt>.
118
+
119
+ $ ls -l tryouts/
120
+ 01_mixins_tryouts.rb
121
+ 10_syntax_tryouts.rb
122
+ 20_cli_tryouts.rb
123
+ 30_benchmark_tryouts.rb
124
+ 50_class_context_tryouts.rb
125
+
126
+ You can also specify specific files to load:
127
+
128
+ $ sergeant any/file.rb
129
+
130
+ Verbose mode gives you some extra information, including the return values from the drills. The more v's, the more information:
131
+
132
+ $ sergeant -v
133
+
134
+ In quiet mode, the sergeant returns only a PASS or FAIL message (in the case of a FAIL, it will also return a non-zero exit code):
135
+
136
+ $ sergeant -q
137
+ PASS
138
+
100
139
  == Screenshots
101
140
 
102
141
  Here is an example of Tryouts output from a gibbler[http://github.com/delano/gibbler/blob/gibbler-0.2.1/tryouts/gibbler_tryouts.rb] tryout:
data/bin/sergeant CHANGED
@@ -59,4 +59,9 @@ module TryoutsCLI
59
59
 
60
60
  end
61
61
 
62
- Drydock.run!(ARGV, STDIN) if Drydock.run?
62
+ begin
63
+ Drydock.run!(ARGV, STDIN) if Drydock.run?
64
+ rescue Tryouts::Exception => ex
65
+ puts ex.message
66
+ puts ex.backtrace if Tryouts.debug?
67
+ end
data/lib/tryouts.rb CHANGED
@@ -41,7 +41,7 @@ class Tryouts
41
41
  end
42
42
  end
43
43
 
44
- VERSION = "0.7.2"
44
+ VERSION = "0.7.3"
45
45
 
46
46
  require 'tryouts/mixins'
47
47
  require 'tryouts/tryout'
@@ -140,15 +140,23 @@ class Tryouts
140
140
  # Require +name+. If +path+ is supplied, it will "require path".
141
141
  # * +name+ The name of the library in question (required). Stored as a Symbol to +@library+.
142
142
  # * +path+ Add a path to the front of $LOAD_PATH (optional). Use this if you want to load
143
- # a specific copy of the library. Otherwise, it loads from the system path.
144
- def library(name=nil, path=nil)
143
+ # a specific copy of the library. Otherwise, it loads from the system path. If the path
144
+ # in specified in multiple arguments they are joined and expanded.
145
+ #
146
+ # library '/an/absolute/path'
147
+ # library __FILE__, '..', 'lib'
148
+ #
149
+ def library(name=nil, *path)
145
150
  return @library if name.nil?
146
- @library = name.to_sym
147
- @dtype = :api
151
+ @library, @dtype = name.to_sym, :api
152
+ path = File.expand_path(File.join *path)
148
153
  $LOAD_PATH.unshift path unless path.nil?
149
154
  begin
150
155
  require @library.to_s
151
- rescue SyntaxError, LoadError, Exception, TypeError,
156
+ rescue LoadError => ex
157
+ @errors << ex.exception("Cannot load library: #{@library} (#{path})")
158
+ Tryouts.failed = true
159
+ rescue SyntaxError, Exception, TypeError,
152
160
  RuntimeError, NoMethodError, NameError => ex
153
161
  @errors << ex
154
162
  Tryouts.failed = true
@@ -241,6 +249,16 @@ class Tryouts
241
249
  tryout(args, &block)
242
250
  end
243
251
 
252
+ # This method does nothing. It provides a quick way to disable a tryout.
253
+ #
254
+ # NOTE: This is a DSL-only method and is not intended for OO use.
255
+ def xtryouts(*args, &block); end
256
+ # This method does nothing. It provides a quick way to disable a tryout.
257
+ #
258
+ # NOTE: this is a standalone DSL-syntax method.
259
+ def self.xtryouts(*args, &block); end
260
+
261
+
244
262
  # Parse a +_tryouts.rb+ file. See Tryouts::CLI::Run for an example.
245
263
  #
246
264
  # NOTE: this is an OO syntax method
@@ -262,6 +280,12 @@ class Tryouts
262
280
  RuntimeError, NoMethodError, NameError => ex
263
281
  to.errors << ex
264
282
  Tryouts.failed = true
283
+ # It's helpful to display the group name
284
+ file_content.match(/^group (.+?)$/) do |x,t|
285
+ # We use eval as a quick cheat so we don't have
286
+ # to parse all the various kinds of quotes.
287
+ to.group = eval x.captures.first
288
+ end
265
289
  end
266
290
  @@instances[to.group] = to
267
291
  to
@@ -41,13 +41,14 @@ class Run < Drydock::Command
41
41
  print "Tryouts #{Tryouts::VERSION} -- "
42
42
  print "#{Tryouts.sysinfo.to_s} (#{RUBY_VERSION}) -- "
43
43
  puts "#{start.strftime("%Y-%m-%d %H:%M:%S")}"
44
+ puts
44
45
  end
45
46
 
46
47
  load_available_tryouts_files
47
48
 
48
49
  passed, failed = 0, 0
49
50
  Tryouts.instances.each_pair do |group,tryouts_inst|
50
- puts '', ' %-79s'.att(:reverse) % group unless Tryouts.verbose < 0
51
+ puts ' %-79s'.att(:reverse) % group unless Tryouts.verbose < 0
51
52
  puts " #{tryouts_inst.paths.join("\n ")}" if Tryouts.verbose > 0
52
53
  tryouts_inst.tryouts.each_pair do |name,to|
53
54
  begin
@@ -61,16 +62,24 @@ class Run < Drydock::Command
61
62
  passed += to.passed
62
63
  failed += to.failed
63
64
  end
64
-
65
+
65
66
  unless tryouts_inst.errors.empty?
66
67
  title = '%-78s' % " RUNTIME ERRORS !?"
67
68
  puts $/, ' ' << title.color(:red).att(:reverse).bright
68
69
  tryouts_inst.errors.each do |ex|
69
- trace = Tryouts.verbose > 1 ? ex.backtrace : [ex.backtrace.first]
70
+
70
71
  puts '%4s%s: %s' % ['', ex.class, ex.message.to_s.split($/).join($/ + ' '*16)]
71
72
  puts
72
- puts '%14s %s' % ["", trace.join($/ + ' '*16)]
73
- puts
73
+
74
+ if [SyntaxError].member? ex.class
75
+ # don't print anymore.
76
+ else
77
+ unless ex.backtrace.nil?
78
+ trace = Tryouts.verbose > 1 ? ex.backtrace : [ex.backtrace.first]
79
+ puts '%14s %s' % ["", trace.join($/ + ' '*16)]
80
+ puts
81
+ end
82
+ end
74
83
  end
75
84
  end
76
85
  end
@@ -139,6 +148,9 @@ private
139
148
  # If file paths were given, check those only.
140
149
  unless @argv.empty?
141
150
  @argv.each do |file|
151
+ unless File.exists?(file)
152
+ raise Tryouts::Exception, "Not found: #{file}"
153
+ end
142
154
  file = File.join(file, '**', '*_tryouts.rb') if File.directory?(file)
143
155
  @tryouts_files += Dir.glob file
144
156
  end
@@ -190,7 +202,7 @@ class Tryouts::CLI::Run
190
202
  `; .'.'/. ,\`.`. ;'
191
203
  `-=;_-' `-----' `-_;=-' -bodom-
192
204
  }
193
- PUG = %q{
205
+ PUG = %q{
194
206
  __,-----._ ,-.
195
207
  ,' ,-. \`---. ,-----<._/
196
208
  (,.-. o:.` )),"\\\-._ ,' `.
data/lib/tryouts/drill.rb CHANGED
@@ -59,7 +59,11 @@ class Tryouts
59
59
  @dreams << Tryouts::Drill::Dream.new(dream_output, format)
60
60
  end
61
61
  when :benchmark
62
- dream_output, format, reps = *(args.size == 1 ? args.first : [args[1], args[0], args[2]])
62
+ if args.size == 1
63
+ reps = args.first
64
+ else
65
+ dream_output, format, reps = args[1], args[0], args[2]
66
+ end
63
67
  @sergeant = Tryouts::Drill::Sergeant::Benchmark.new reps
64
68
  @dreams << Tryouts::Drill::Dream.new(Tryouts::Stats, :class)
65
69
  unless dream_output.nil?
@@ -112,27 +116,33 @@ class Tryouts
112
116
 
113
117
  def info
114
118
  out = StringIO.new
115
- if Tryouts.verbose > 1
116
- if @dreams.empty?
117
- out.puts '%6s%s'.color(@clr) % ['', @reality.output.inspect]
118
- else
119
- @dreams.each do |dream|
120
- if dream != @reality
121
- out.puts '%6s%s'.color(:red) % ['', @reality.output.inspect]
122
- else
123
- out.puts '%6s%s'.color(:green) % ["", dream.test_to_string(@reality)]
124
- end
119
+ if Tryouts.verbose > 0
120
+ if @dtype == :benchmark
121
+ unless @reality.output.nil?
122
+ mean, sdev, sum = @reality.output.mean, @reality.output.sdev, @reality.output.sum
123
+ out.puts '%6s%.4f (sdev:%.4f sum:%.4f)'.color(@clr) % ['', mean, sdev, sum]
125
124
  end
125
+ else
126
+ out.puts '%6s%s'.color(@clr) % ['', @reality.output.inspect]
126
127
  end
127
- elsif Tryouts.verbose > 0
128
- out.puts '%6s%s'.color(@clr) % ['', @reality.output.inspect]
129
128
  unless @reality.stash.empty?
130
129
  @reality.stash.each_pair do |n,v|
131
130
  out.puts '%18s: %s'.color(@clr) % [n,v.inspect]
132
131
  end
133
- out.puts
134
132
  end
135
133
  end
134
+ if Tryouts.verbose > 1
135
+
136
+ @dreams.each do |dream|
137
+ if dream != @reality
138
+ out.puts '%6s%s'.color(:red) % ['', dream.test_to_string(@reality)]
139
+ else
140
+ out.puts '%6s%s'.color(:green) % ["", dream.test_to_string(@reality)]
141
+ end
142
+ end
143
+ out.puts
144
+
145
+ end
136
146
  out.rewind
137
147
  out.read
138
148
  end
@@ -162,6 +172,10 @@ class Tryouts
162
172
  out.read
163
173
  end
164
174
 
175
+ def has_error?
176
+ !@reality.error.nil?
177
+ end
178
+
165
179
  def success?
166
180
  return false if @dreams.empty? && @reality.output != true
167
181
  begin
@@ -49,7 +49,7 @@ class Tryouts::Drill
49
49
  if dream.format.nil?
50
50
  reality.output == dream.output
51
51
  elsif reality.output.respond_to?(dream.format)
52
- reality.output.send(dream.format) == dream.output
52
+ reality.comparison_value(dream) == dream.output
53
53
  else
54
54
  false
55
55
  end
@@ -96,7 +96,7 @@ class Tryouts::Drill
96
96
  elsif reality.output.respond_to?(dream.format)
97
97
  "#{reality.output.inspect}.#{dream.format} == #{dream.output.inspect}"
98
98
  else
99
- "#{reality.output.inspect}.respond_to?(#{dream.format.inspect}) == false"
99
+ "Unknown method #{dream.format.inspect} for #{reality.output.class} "
100
100
  end
101
101
 
102
102
  end
@@ -195,7 +195,7 @@ class Tryouts::Drill
195
195
  else
196
196
  if @output.nil?
197
197
  @output
198
- elsif @output.respond_to?(dream.format)
198
+ elsif @output.respond_to?(dream.format.to_sym)
199
199
  @output.send(dream.format)
200
200
  else
201
201
  @output
@@ -13,10 +13,10 @@ class Tryouts; class Drill; module Sergeant
13
13
 
14
14
  attr_reader :output
15
15
 
16
- # * +reps+ Number of times to execute drill (>= 0, <= 30). Default: 3
16
+ # * +reps+ Number of times to execute drill (>= 0, <= 1000000). Default: 3
17
17
  #
18
18
  def initialize(reps=nil)
19
- @reps = (1..30).include?(reps) ? reps : 5
19
+ @reps = (1..1000000).include?(reps) ? reps : 5
20
20
  @stats = Tryouts::Stats.new
21
21
  end
22
22
 
@@ -24,6 +24,7 @@ class Tryouts; class Drill; module Sergeant
24
24
  # A Proc object takes precedence over an inline block.
25
25
  runtime = (block.nil? ? inline : block)
26
26
  response = Tryouts::Drill::Reality.new
27
+
27
28
  if runtime.nil?
28
29
  raise "We need a block to benchmark"
29
30
  else
@@ -34,11 +35,12 @@ class Tryouts; class Drill; module Sergeant
34
35
  @stats.sample run
35
36
  end
36
37
 
38
+ # We add the output after we run the block so that
39
+ # that it'll remain nil if an exception was raised
37
40
  response.output = @stats
38
41
 
39
42
  rescue => e
40
43
  puts e.message, e.backtrace if Tryouts.verbose > 2
41
- response.output = false
42
44
  response.etype = e.class
43
45
  response.error = e.message
44
46
  response.trace = e.backtrace
@@ -81,6 +81,14 @@ class Tryouts
81
81
  puts $/, ' ' << title.color(:red).att(:reverse)
82
82
  puts drill.report
83
83
  end
84
+ # Print errors for successful runs too
85
+ success = @drills.select { |d| !d.skip? && d.success? }
86
+ success.each do |drill,index|
87
+ next unless drill.has_error?
88
+ title = ' %-69s ' % ["\"#{drill.name}\""]
89
+ puts $/, ' ' << title.color(:red).att(:reverse)
90
+ puts drill.report
91
+ end
84
92
  end
85
93
 
86
94
  # Did every Tryout finish successfully?
data/tryouts.gemspec CHANGED
@@ -1,7 +1,7 @@
1
1
  @spec = Gem::Specification.new do |s|
2
2
  s.name = "tryouts"
3
3
  s.rubyforge_project = "tryouts"
4
- s.version = "0.7.2"
4
+ s.version = "0.7.3"
5
5
  s.summary = "Tryouts are high-level tests for your Ruby code. May all your dreams come true!"
6
6
  s.description = s.summary
7
7
  s.author = "Delano Mandelbaum"
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: delano-tryouts
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.7.2
4
+ version: 0.7.3
5
5
  platform: ruby
6
6
  authors:
7
7
  - Delano Mandelbaum