ZenTest 3.0.0 → 3.1.0

Sign up to get free protection for your applications and to get access to all the features.
@@ -1,3 +1,25 @@
1
+ *** 3.1.0 / 2006-03-29
2
+
3
+ + 2 major enhancements
4
+ + Added multiruby! YAY!
5
+ + Massive improvements to autotest: speed, reliability, reporting, etc.
6
+ + 10 minor enhancements
7
+ + multiruby builds in a centralized location. YAY!
8
+ + multiruby now allows reinstalls quickly and easily (can even skip config).
9
+ + multiruby exits with total sum of exit codes.
10
+ + autotest file search is muuuuch faster.
11
+ + autotest automatically detects rails mode.
12
+ + autotest deals with rails dependencies much better.
13
+ + autotest reruns a full suite after you go green to ensure full coverage.
14
+ + autotest always runs with unit_diff -u.
15
+ + autotest can now run cvs/svn/p4 up periodically to be a mini-tinderbox.
16
+ + autotest now has real help.
17
+ + 4 bug fixes
18
+ - ZenTest is now zentest. Yay for consistency! (do a rake uninstall to clean)
19
+ - ZenTest excludes pretty_print methods.
20
+ - Fixed unary operator issues (they were backwards... oops!) for ZenTest.
21
+ - unit_diff now runs diff.exe on Windoze. dunno if that will work.
22
+
1
23
  *** 3.0.0 / 2006-03-06
2
24
 
3
25
  + 2 major enhancements
@@ -3,23 +3,36 @@ LinuxJournalArticle.txt
3
3
  Manifest.txt
4
4
  README.txt
5
5
  Rakefile
6
- bin/ZenTest
7
6
  bin/autotest
7
+ bin/multiruby
8
8
  bin/unit_diff
9
+ bin/zentest
9
10
  example.txt
10
11
  example1.rb
11
12
  example2.rb
12
- lib/ZenTest.rb
13
13
  lib/autotest.rb
14
14
  lib/rails_autotest.rb
15
15
  lib/unit_diff.rb
16
+ lib/zentest.rb
17
+ test/data/normal/lib/.#photo.rb
18
+ test/data/normal/lib/blah.rb
16
19
  test/data/normal/lib/photo.rb
17
20
  test/data/normal/test/test_camelcase.rb
18
21
  test/data/normal/test/test_photo.rb
19
22
  test/data/normal/test/test_route.rb
20
23
  test/data/normal/test/test_user.rb
24
+ test/data/rails/app/controllers/admin/theme_controller.rb
25
+ test/data/rails/app/controllers/route_controller.rb
26
+ test/data/rails/app/models/flickr_photo.rb
27
+ test/data/rails/app/models/route.rb
28
+ test/data/rails/config/environment.rb
29
+ test/data/rails/config/routes.rb
21
30
  test/data/rails/test/fixtures/routes.yml
31
+ test/data/rails/test/functional/admin/themes_controller_test.rb
32
+ test/data/rails/test/functional/dummy_controller_test.rb
22
33
  test/data/rails/test/functional/route_controller_test.rb
34
+ test/data/rails/test/unit/flickr_photo_test.rb
35
+ test/data/rails/test/unit/photo_test.rb
23
36
  test/data/rails/test/unit/route_test.rb
24
37
  test/test_autotest.rb
25
38
  test/test_rails_autotest.rb
data/README.txt CHANGED
@@ -18,6 +18,9 @@ autotest is a continous testing facility meant to be used during
18
18
  development. As soon as you save a file, autotest will run the
19
19
  corresponding dependent tests.
20
20
 
21
+ multiruby runs anything you want on multiple versions of ruby. Great
22
+ for compatibility checking!
23
+
21
24
  There are two strategies intended for ZenTest: test conformance
22
25
  auditing and rapid XP.
23
26
 
@@ -35,14 +38,20 @@ implementation.
35
38
  * Scans your ruby code and tests and generates missing methods for you.
36
39
  * Includes a very helpful filter for Test::Unit output called unit_diff.
37
40
  * Continually and intelligently test only those files you change with autotest.
41
+ * Test against multiple versions with multiruby.
42
+ - Not the best doco in the world (my fault)
38
43
  * Includes a LinuxJournal article on testing with ZenTest written by Pat Eyler.
44
+ * See also: http://blog.zenspider.com/archives/zentest/
39
45
 
40
46
  == SYNOPSYS
41
47
 
42
48
  ZenTest MyProject.rb TestMyProject.rb > missing.rb
43
- # edit missing.rb and merge appropriate parts into the above files.
49
+
44
50
  ./TestMyProject.rb | unit_diff
45
- # Use unit_diff.rb to show you the actual differences in your failures.
51
+
52
+ autotest
53
+
54
+ multiruby ./TestMyProject.rb
46
55
 
47
56
  == RULES
48
57
 
data/Rakefile CHANGED
@@ -6,8 +6,7 @@ require 'rake/rdoctask'
6
6
  require 'rake/gempackagetask'
7
7
  require 'rbconfig'
8
8
 
9
- $: << 'lib'
10
- require 'ZenTest'
9
+ require './lib/zentest.rb'
11
10
 
12
11
  $VERBOSE = nil
13
12
 
@@ -19,7 +18,7 @@ spec = Gem::Specification.new do |s|
19
18
 
20
19
  s.files = File.read('Manifest.txt').split($/)
21
20
  s.require_path = 'lib'
22
- s.executables = %w[ZenTest unit_diff autotest]
21
+ s.executables = %w[ZenTest unit_diff autotest multiruby]
23
22
 
24
23
  paragraphs = File.read("README.txt").split(/\n\n+/)
25
24
  s.instance_variable_set "@description", paragraphs[3..9].join("\n\n")
@@ -55,10 +54,10 @@ end
55
54
  desc 'Generate RDoc'
56
55
  Rake::RDocTask.new :rdoc do |rd|
57
56
  rd.rdoc_dir = 'doc'
58
- rd.rdoc_files.add 'lib', 'README.txt', 'History.txt',
59
- 'LinuxJournalArticle.txt'
57
+ rd.rdoc_files.add 'lib', 'README.txt', 'History.txt', 'LinuxJournalArticle.txt'
60
58
  rd.main = 'README.txt'
61
- rd.options << '-d' if `which dot` =~ /\/dot/
59
+ rd.options << '-d' if `which dot` =~ /\/dot/ unless RUBY_PLATFORM =~ /win32/
60
+ rd.options << '-t "ZenTest RDoc"'
62
61
  end
63
62
 
64
63
  desc 'Build Gem'
@@ -69,8 +68,8 @@ end
69
68
  $prefix = ENV['PREFIX'] || Config::CONFIG['prefix']
70
69
  $bin = File.join($prefix, 'bin')
71
70
  $lib = Config::CONFIG['sitelibdir']
72
- $bins = %w(ZenTest autotest unit_diff)
73
- $libs = %w(ZenTest.rb autotest.rb rails_autotest.rb unit_diff.rb)
71
+ $bins = %w(zentest autotest unit_diff multiruby)
72
+ $libs = %w(zentest.rb autotest.rb rails_autotest.rb unit_diff.rb)
74
73
 
75
74
  task :install do
76
75
  $bins.each do |f|
@@ -83,6 +82,10 @@ task :install do
83
82
  end
84
83
 
85
84
  task :uninstall do
85
+ # add old versions
86
+ $bins << "ZenTest"
87
+ $libs << "ZenTest.rb"
88
+
86
89
  $bins.each do |f|
87
90
  rm_f File.join($bin, f)
88
91
  end
@@ -94,5 +97,5 @@ end
94
97
 
95
98
  desc 'Clean up'
96
99
  task :clean => [ :clobber_rdoc, :clobber_package ] do
97
- rm_rf %w(*~ doc)
100
+ rm_f Dir["**/*~"]
98
101
  end
@@ -1,6 +1,6 @@
1
1
  #!/usr/local/bin/ruby -swI .
2
2
 
3
- require 'ZenTest'
3
+ require 'zentest'
4
4
 
5
5
  $TESTING = true # for ZenWeb and any other testing infrastructure code
6
6
 
@@ -1,8 +1,43 @@
1
1
  #!/usr/local/bin/ruby -ws
2
2
 
3
- $rails ||= false
3
+ $h ||= false
4
+ $help ||= false
4
5
 
5
- if $rails then
6
+ if $h or $help then
7
+ help = []
8
+ help << "autotest [options]"
9
+ help << nil
10
+ help << "options:"
11
+ help << "\t-h"
12
+ help << "\t-help\t\tYou're looking at it."
13
+ help << nil
14
+ help << "\t-v\t\tBe verbose."
15
+ help << "\t\t\tPrints files that autotest doesn't know how to map to"
16
+ help << "\t\t\ttests."
17
+ help << nil
18
+ help << "\t-rails\t\tForce rails mode."
19
+ help << "\t\t\tRails will be automatically detected by the presence of"
20
+ help << "\t\t\tconfig/environment.rb"
21
+ help << nil
22
+ help << "\t-vcs=NAME\tVersion control system to update."
23
+ help << "\t\t\tAutotest will automatically update every vcstime"
24
+ help << "\t\t\tseconds."
25
+ help << "\t\t\tAutotest understands Perforce (p4), CVS (cvs) and"
26
+ help << "\t\t\tSubversion (svn)."
27
+ help << nil
28
+ help << "\t-vcstime=N\tUpdate source control every N seconds."
29
+ help << "\t\t\tDefaults to every five minutes (300 seconds)."
30
+ STDERR.puts help.join("\n")
31
+ exit 1
32
+ end
33
+
34
+ $v ||= false
35
+ $rails ||= false
36
+ $vcs ||= nil
37
+ $vcstime ||= 300
38
+ $vcstime = $vcstime.to_i
39
+
40
+ if $rails or File.exist? 'config/environment.rb' then
6
41
  require 'rails_autotest'
7
42
  RailsAutotest.run
8
43
  else
@@ -0,0 +1,82 @@
1
+ #!/usr/local/bin/ruby -w
2
+
3
+ require 'fileutils'
4
+
5
+ def run(cmd)
6
+ puts "Running command: #{cmd}"
7
+ raise "ERROR: Command failed with exit code #{$?}" unless system cmd
8
+ end
9
+
10
+ root_dir = File.expand_path(ENV['MULTIRUBY'] || File.join(ENV['HOME'], ".multiruby"))
11
+ unless test ?d, root_dir then
12
+ puts "creating #{root_dir}"
13
+ Dir.mkdir root_dir, 0700
14
+ end
15
+
16
+ versions = []
17
+ Dir.chdir root_dir do
18
+ %w(build install versions).each do |dir|
19
+ unless test ?d, dir then
20
+ puts "creating #{dir}"
21
+ Dir.mkdir dir
22
+ if dir == "versions" then
23
+ file = "ruby-#{RUBY_VERSION}.tar.gz"
24
+ puts " downloading #{file} via HTTP... this might take a while."
25
+ puts " Put other ruby tarballs in versions to use them."
26
+
27
+ Dir.chdir dir do
28
+ require 'open-uri'
29
+ open("http://ftp.ruby-lang.org/pub/ruby/#{file}") do |f|
30
+ File.open file, 'w' do |out|
31
+ out.write f.read
32
+ end
33
+ end
34
+ end
35
+ end
36
+ end
37
+ end
38
+
39
+ Dir.chdir "build" do
40
+ Dir["../versions/ruby*.tar.gz"].each do |tarball|
41
+ build_dir = File.basename tarball, ".tar.gz"
42
+ version = build_dir.sub(/^ruby-?/, '')
43
+ versions << version
44
+ inst_dir = "#{root_dir}/install/#{version}"
45
+ unless test ?d, inst_dir then
46
+ unless test ?d, build_dir then
47
+ puts "creating #{inst_dir}"
48
+ Dir.mkdir inst_dir
49
+ puts "unpacking #{tarball}"
50
+ system "tar zxf #{tarball}"
51
+ end
52
+ Dir.chdir build_dir do
53
+ run "autoconf" unless test ?f, "configure"
54
+ FileUtils.rm_r "ext/readline" if test ?d, "ext/readline"
55
+ run "./configure --prefix #{inst_dir} &> log.configure"
56
+ run "make -j4 &> log.build"
57
+ run "make install &> log.install"
58
+ end
59
+ end
60
+ end
61
+ end
62
+ end
63
+
64
+ results = versions.map do |version|
65
+ ruby = "#{root_dir}/install/#{version}/bin/ruby"
66
+ puts
67
+ puts "VERSION = #{version}"
68
+ puts
69
+ system ruby, *ARGV
70
+ puts
71
+ puts "RESULT = #{$?}"
72
+ $?
73
+ end
74
+
75
+ result = results.select { |n| n != 0 }.size
76
+
77
+ puts
78
+ puts "TOTAL RESULT = #{result} failures"
79
+
80
+ exit result
81
+
82
+
@@ -0,0 +1,28 @@
1
+ #!/usr/local/bin/ruby -swI .
2
+
3
+ require 'zentest'
4
+
5
+ $TESTING = true # for ZenWeb and any other testing infrastructure code
6
+
7
+ if defined? $v then
8
+ puts "#{File.basename $0} v#{ZenTest::VERSION}"
9
+ exit 0
10
+ end
11
+
12
+ if defined? $h then
13
+ puts "usage: #{File.basename $0} [-h -v] test-and-implementation-files..."
14
+ puts " -h display this information"
15
+ puts " -v display version information"
16
+ puts " -r Reverse mapping (ClassTest instead of TestClass)"
17
+ puts " -e (Rapid XP) eval the code generated instead of printing it"
18
+ exit 0
19
+ end
20
+
21
+ code = ZenTest.fix(*ARGV)
22
+ if defined? $e then
23
+ require 'test/unit'
24
+ eval code
25
+ else
26
+ print code
27
+ end
28
+
@@ -22,6 +22,17 @@ require 'find'
22
22
  # * Implementation files must match up with a test file named
23
23
  # test_.*implementation.rb
24
24
 
25
+ # New (proposed) strategy:
26
+ #
27
+ # 1) find all files and associate them from impl <-> test
28
+ # 2) run all tests
29
+ # 3) scan for failures
30
+ # 4) detect changes in ANY (ruby?) file, rerun all failures + changed files
31
+ # NOTE: this runs in a loop, loop handling should be improved slightly to
32
+ # have less crap (ruby command, failure count).
33
+ # 5) until 0 defects, goto 3
34
+ # 6) when 0 defects, goto 2
35
+
25
36
  class Autotest
26
37
 
27
38
  def self.run
@@ -39,20 +50,66 @@ class Autotest
39
50
  end
40
51
 
41
52
  ##
42
- # Maps failed class +klass+ to test files in +tests+ that have been updated.
43
-
44
- def failed_test_files(klass, tests)
45
- failed_klass = klass.sub('Test', '').gsub(/(.)([A-Z])/, '\1_?\2').downcase
46
- # tests that match this failure
47
- failed_files = tests.select { |test| test =~ /#{failed_klass}/ }
48
- # updated implementations that match this failure
49
- changed_impls = @files.keys.select do |file|
50
- file =~ %r%^lib.*#{failed_klass}.rb$% and updated? file
53
+ # Consolidates failed tests +failed+ for the same test class into a single
54
+ # test runner filter. Also maps failed class to its file regexp.
55
+
56
+ def consolidate_failures(failed)
57
+ filters = Hash.new { |h,k| h[k] = [] }
58
+
59
+ failed.each do |method, klass|
60
+ failed_file = klass.sub('Test', '').gsub(/(.)([A-Z])/, '\1_?\2')
61
+ filters[failed_file.downcase] << method
62
+ end
63
+
64
+ return filters.map do |klass, methods|
65
+ ["'/^(#{methods.join('|')})/'", /#{klass}/]
51
66
  end
52
- tests_to_run = map_file_names(changed_impls).flatten
53
- # add updated tests
54
- failed_files.each { |f| tests_to_run << f if updated? f }
55
- return tests_to_run.uniq
67
+ end
68
+
69
+
70
+ ##
71
+ # Selects test files to run that match failures in +failed_file+ based on
72
+ # +updated_files+ and +tests+.
73
+ #
74
+ # Only test files matching +failed_file+ will be returned so the test
75
+ # runner's -n flag will correctly match the failed tests.
76
+ #--
77
+ # failed_test_files must never check for updated files, retest_failed reuses
78
+ # +updated_files+.
79
+
80
+ def failed_test_files(failed_file, tests, updated_files)
81
+ return [] if updated_files.empty?
82
+
83
+ updated_tests = updated_files.select { |f| f =~ /^test/ }
84
+
85
+ tests_to_filter = if updated_files == updated_tests then
86
+ updated_tests
87
+ else
88
+ files = (updated_files + tests).uniq
89
+ tests_to_filter = map_file_names(files).flatten.uniq
90
+ end
91
+
92
+ return tests_to_filter.select { |test| test =~ failed_file }
93
+ end
94
+
95
+ ##
96
+ # Returns a report of remaining failures in +failures+.
97
+
98
+ def failure_report(failed)
99
+ out = []
100
+ out << "# failures remain in #{failed.length} files:"
101
+ failed.each do |filter, failed_test|
102
+ tests = @files.keys.select { |file| file =~ /^test.*#{failed_test}/ }
103
+ test = tests.sort_by { |f| f.length }.first
104
+
105
+ filter =~ /\((.*)\)/
106
+ filter = $1.split('|')
107
+
108
+ out << "# #{test}:"
109
+ out << "# #{filter.join "\n# "}"
110
+ end
111
+
112
+ return out.join("\n")
56
113
  end
57
114
 
58
115
  ##
@@ -75,34 +132,61 @@ class Autotest
75
132
  when %r%^(doc|pkg)/% then
76
133
  # ignore
77
134
  else
78
- STDERR.puts "Dunno! #{filename}" # What are you trying to pull?
135
+ STDERR.puts "Dunno! #{filename}" if $v or $TESTING
79
136
  end
80
137
  end
81
138
 
82
139
  return [tests]
83
140
  end
84
141
 
142
+ ##
143
+ # Resets all file timestamps in the list
144
+
145
+ def reset_times
146
+ ago = Time.at 0
147
+ @files.each_key { |file| @files[file] = ago }
148
+ end
149
+
85
150
  ##
86
151
  # Retests failed tests.
152
+ #--
153
+ # Runs through each failure and runs tests matching the failure. If an
154
+ # implementation file was updated all failed tests should be run.
155
+ #
156
+ # TODO collapse multiple failures in the same file (in test) and use | in
157
+ # the filter.
87
158
 
88
159
  def retest_failed(failed, tests)
160
+ puts "# Waiting for changes"
161
+
89
162
  # -t and -n includes all tests that match either filter, not tests that
90
163
  # match both filters, so figure out which TestCase to run from the filename,
91
164
  # and use -n on that.
92
165
  until failed.empty? do
93
166
  sleep 5 unless $TESTING
94
167
 
95
- failed.map! do |method, klass|
96
- failed_files = failed_test_files klass, tests
97
- break [method, klass] if failed_files.empty?
168
+ updated = updated_files
169
+
170
+ # REFACTOR
171
+ failed.map! do |filter, failed_test|
172
+ failed_files = failed_test_files failed_test, tests, updated
173
+ break [filter, failed_test] if failed_files.empty?
174
+
98
175
  puts "# Rerunning failures: #{failed_files.join ' '}"
99
- filter = "-n #{method} " unless method == 'default_test'
100
- cmd = "ruby -Ilib:test -S testrb #{filter}#{failed_files.join ' '}"
176
+
177
+ test_filter = " -n #{filter}" unless filter == "'/^(default_test)/'"
178
+ cmd = "ruby -Ilib:test #{failed_files.join ' '}#{test_filter} | unit_diff -u"
179
+
101
180
  puts "+ #{cmd}"
102
- system(cmd) ? nil : [method, klass] # clever
181
+ result = `#{cmd}`
182
+ puts result
183
+ status = result.split($/).last
184
+ rerun = status =~ / 0 failures, 0 errors/ ? nil : [filter, failed_test]
185
+ puts "# Waiting for changes" if rerun
186
+ rerun # needed for map!
103
187
  end
104
188
 
105
- failed.compact!
189
+ puts failure_report(failed) if failed.compact! and not failed.empty?
106
190
  end
107
191
  end
108
192
 
@@ -115,6 +199,7 @@ class Autotest
115
199
  puts "# Ok, you really want to quit, doing so"
116
200
  exit
117
201
  end
202
+ # STDERR.puts "\t#{caller.join "\n\t"}"
118
203
  puts "# hit ^C again to quit"
119
204
  sleep 1.5 # give them enough time to hit ^C again
120
205
  @interrupt = true # if they hit ^C again,
@@ -122,7 +207,23 @@ class Autotest
122
207
  end
123
208
 
124
209
  begin
210
+ last_update = Time.at 0
211
+
125
212
  loop do
213
+ if $vcs and Time.now > last_update + $vcstime then
214
+ last_update = Time.now
215
+ case $vcs
216
+ when 'cvs' then
217
+ system 'cvs up'
218
+ when 'p4' then
219
+ system 'p4 sync'
220
+ when 'svn' then
221
+ system 'svn up'
222
+ else
223
+ puts "# Sorry, I don't know what version control system \"#{$vcs}\" is"
224
+ end
225
+ end
226
+
126
227
  files = updated_files
127
228
  test files unless files.empty?
128
229
  sleep 5
@@ -138,12 +239,16 @@ class Autotest
138
239
  ##
139
240
  # Runs tests for files in +updated+. Implementation files are looked up
140
241
  # with map_file_names.
242
+ #
243
+ # Returns true if any of the tests ever failed.
141
244
 
142
245
  def test(updated)
246
+ ever_failed = false
247
+
143
248
  map_file_names(updated).each do |tests|
144
249
  next if tests.empty?
145
250
  puts '# Testing updated files'
146
- cmd = "ruby -Ilib:test -e '#{tests.inspect}.each { |f| load f }'"
251
+ cmd = "ruby -Ilib:test -e '#{tests.inspect}.each { |f| load f }' | unit_diff -u"
147
252
  puts "+ #{cmd}"
148
253
  results = `#{cmd}`
149
254
  puts results
@@ -153,18 +258,30 @@ class Autotest
153
258
  next
154
259
  end
155
260
 
261
+ ever_failed = true
262
+
156
263
  failed = results.scan(/^\s+\d+\) (?:Failure|Error):\n(.*?)\((.*?)\)/)
157
264
 
158
265
  if failed.empty? then
159
- puts '# Test::Unit died, you did a really bad thing, retrying in 10'
266
+ puts '# Test::Unit exited without a parseable failure or error message.'
267
+ puts '# You probably have a syntax error in your code.'
268
+ puts '# I\'ll retry in 10 seconds'
160
269
  sleep 10
161
270
  redo
162
271
  end
163
272
 
273
+ failed = consolidate_failures(failed)
274
+
275
+ # REFACTOR: I don't think the two routines merit real differences
164
276
  retest_failed failed, tests
165
277
  end
166
278
 
279
+ reset_times if ever_failed
280
+
167
281
  puts '# All passed'
282
+ puts "# Waiting for changes"
283
+
284
+ return ever_failed
168
285
  end
169
286
 
170
287
  ##
@@ -187,11 +304,22 @@ class Autotest
187
304
  updated = []
188
305
 
189
306
  Find.find '.' do |f|
307
+
308
+ if @exceptions then
309
+ if f =~ @exceptions then
310
+ Find.prune if Kernel.test ?d, f
311
+ next
312
+ end
313
+ end
314
+
315
+ Find.prune if f =~ /(?:\.svn|CVS)$/ # version control files
316
+
190
317
  next if File.directory? f
191
- next if f =~ /(?:swp|~|rej|orig)$/ # temporary/patch files
192
- next if f =~ %r%/(?:.svn|CVS)/% # version control files
193
- next if f =~ @exceptions unless @exceptions.nil? # custom exceptions
194
- f = f.sub(/^\.\//, '') # trim the ./ that Find gives us
318
+ next if f =~ /(?:swp|~|rej|orig)$/ # temporary/patch files
319
+ next if f =~ /\/\.#/ # Emacs autosave/cvs merge files
320
+
321
+ f = f.sub(/^\.\//, '')
322
+
195
323
  updated << f if updated? f
196
324
  end
197
325