phene-parallel_tests 0.6.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/Gemfile ADDED
@@ -0,0 +1,9 @@
1
+ source :rubygems
2
+
3
+ gem 'parallel'
4
+
5
+ group :dev do
6
+ gem 'rspec', '>=2.4'
7
+ gem 'rake'
8
+ gem 'jeweler'
9
+ end
@@ -0,0 +1,28 @@
1
+ GEM
2
+ remote: http://rubygems.org/
3
+ specs:
4
+ diff-lcs (1.1.2)
5
+ git (1.2.5)
6
+ jeweler (1.6.3)
7
+ bundler (~> 1.0)
8
+ git (>= 1.2.5)
9
+ rake
10
+ parallel (0.5.1)
11
+ rake (0.8.7)
12
+ rspec (2.6.0)
13
+ rspec-core (~> 2.6.0)
14
+ rspec-expectations (~> 2.6.0)
15
+ rspec-mocks (~> 2.6.0)
16
+ rspec-core (2.6.4)
17
+ rspec-expectations (2.6.0)
18
+ diff-lcs (~> 1.1.2)
19
+ rspec-mocks (2.6.0)
20
+
21
+ PLATFORMS
22
+ ruby
23
+
24
+ DEPENDENCIES
25
+ jeweler
26
+ parallel
27
+ rake
28
+ rspec (>= 2.4)
@@ -0,0 +1,20 @@
1
+ task :default => :spec
2
+ require "rspec/core/rake_task"
3
+ RSpec::Core::RakeTask.new(:spec) do |t|
4
+ t.rspec_opts = '--backtrace --color'
5
+ end
6
+
7
+ begin
8
+ require 'jeweler'
9
+ Jeweler::Tasks.new do |gem|
10
+ gem.name = "parallel_tests"
11
+ gem.summary = "Run tests / specs / features in parallel"
12
+ gem.email = "grosser.michael@gmail.com"
13
+ gem.homepage = "http://github.com/grosser/#{gem.name}"
14
+ gem.authors = "Michael Grosser"
15
+ end
16
+
17
+ Jeweler::GemcutterTasks.new
18
+ rescue LoadError
19
+ puts "Jeweler, or one of its dependencies, is not available. Install it with: sudo gem install jeweler"
20
+ end
@@ -0,0 +1,219 @@
1
+ Speedup Test::Unit + RSpec + Cucumber by running parallel on multiple CPUs (or cores).<br/>
2
+ ParallelTests splits tests into even groups and runs each group in a single process with its own database.
3
+
4
+ Setup for Rails
5
+ ===============
6
+ ## Install
7
+ ### Rails 3
8
+ If you use RSpec: ensure you got >= 2.4
9
+
10
+ As gem
11
+
12
+ # add to Gemfile
13
+ gem "parallel_tests", :group => :development
14
+
15
+ OR as plugin
16
+
17
+ rails plugin install git://github.com/grosser/parallel_tests.git
18
+
19
+ # add to Gemfile
20
+ gem "parallel", :group => :development
21
+
22
+
23
+ ### Rails 2
24
+
25
+ As gem
26
+
27
+ gem install parallel_tests
28
+
29
+ # add to config/environments/development.rb
30
+ config.gem "parallel_tests"
31
+
32
+ # add to Rakefile
33
+ begin; require 'parallel_tests/tasks'; rescue LoadError; end
34
+
35
+ OR as plugin
36
+
37
+ gem install parallel
38
+
39
+ # add to config/environments/development.rb
40
+ config.gem "parallel"
41
+
42
+ ./script/plugin install git://github.com/grosser/parallel_tests.git
43
+
44
+ ## Setup
45
+ ParallelTests uses 1 database per test-process, 2 processes will use `*_test` and `*_test2`.
46
+
47
+
48
+ ### 1: Add to `config/database.yml`
49
+ test:
50
+ database: xxx_test<%= ENV['TEST_ENV_NUMBER'] %>
51
+
52
+ ### 2: Create additional database(s)
53
+ rake parallel:create
54
+
55
+ ### 3: Copy development schema (repeat after migrations)
56
+ rake parallel:prepare
57
+
58
+ ### 4: Run!
59
+ rake parallel:test # Test::Unit
60
+ rake parallel:spec # RSpec
61
+ rake parallel:features # Cucumber
62
+
63
+ rake parallel:test[1] --> force 1 CPU --> 86 seconds
64
+ rake parallel:test --> got 2 CPUs? --> 47 seconds
65
+ rake parallel:test --> got 4 CPUs? --> 26 seconds
66
+ ...
67
+
68
+ Test by pattern (e.g. use one integration server per subfolder / see if you broke any 'user'-related tests)
69
+
70
+ rake parallel:test[^unit] # everything in test/unit folder (every test file matching /^unit/)
71
+ rake parallel:test[user] # run users_controller + user_helper + user tests
72
+ rake parallel:test['user|product'] # run user and product related tests
73
+
74
+
75
+ Example output
76
+ --------------
77
+ 2 processes for 210 specs, ~ 105 specs per process
78
+ ... test output ...
79
+
80
+ 843 examples, 0 failures, 1 pending
81
+
82
+ Took 29.925333 seconds
83
+
84
+ Spec Loggers
85
+ ===================
86
+
87
+ Even process runtimes
88
+ -----------------
89
+
90
+ Log test runtime to give each process the same test runtime.
91
+
92
+ Add to your `spec/parallel_spec.opts` (or `spec/spec.opts`) :
93
+
94
+ RSpec 1.x:
95
+ --format progress
96
+ --format ParallelSpecs::SpecRuntimeLogger:tmp/parallel_profile.log
97
+ RSpec >= 2.4:
98
+ Installed as plugin: -I vendor/plugins/parallel_tests/lib
99
+ --format progress
100
+ --format ParallelSpecs::SpecRuntimeLogger --out tmp/parallel_profile.log
101
+
102
+ SpecSummaryLogger
103
+ --------------------
104
+
105
+ This logger stops the different processes overwriting each other's output.
106
+
107
+ Add the following to your `spec/parallel_spec.opts` (or `spec/spec.opts`) :
108
+
109
+ RSpec 1.x:
110
+ --format progress
111
+ --format ParallelSpecs::SpecSummaryLogger:tmp/spec_summary.log
112
+ RSpec >= 2.2:
113
+ --format progress
114
+ --format ParallelSpecs::SpecSummaryLogger --out tmp/spec_summary.log
115
+
116
+ SpecFailuresLogger
117
+ -----------------------
118
+
119
+ This logger produces command lines for running any failing examples.
120
+
121
+ E.g.
122
+
123
+ spec /path/to/my_spec.rb -e "should do something"
124
+
125
+ Add the following to your `spec/parallel_spec.opts` (or `spec/spec.opts`) :
126
+
127
+ RSpec 1.x:
128
+ --format ParallelSpecs::SpecFailuresLogger:tmp/failing_specs.log
129
+ RSpec >= 2.4:
130
+ --format ParallelSpecs::SpecFailuresLogger --out tmp/failing_specs.log
131
+
132
+ Setup for non-rails
133
+ ===================
134
+ sudo gem install parallel_tests
135
+ # go to your project dir
136
+ parallel_test OR parallel_spec OR parallel_cucumber
137
+ # [Optional] use ENV['TEST_ENV_NUMBER'] inside your tests to select separate db/memcache/etc.
138
+
139
+ [optional] Only run selected files & folders:
140
+
141
+ parallel_test test/bar test/baz/xxx_text.rb
142
+
143
+ Options are:
144
+
145
+ -n [PROCESSES] How many processes to use, default: available CPUs
146
+ -p, --path [PATH] run tests inside this path only
147
+ --no-sort do not sort files before running them
148
+ -m, --multiply-processes [FLOAT] use given number as a multiplier of processes to run
149
+ -r, --root [PATH] execute test commands from this path
150
+ -e, --exec [COMMAND] execute this code parallel and with ENV['TEST_ENV_NUM']
151
+ -o, --test-options '[OPTIONS]' execute test commands with those options
152
+ -t, --type [TYPE] which type of tests to run? test, spec or features
153
+ --non-parallel execute same commands but do not in parallel, needs --exec
154
+ -v, --version Show Version
155
+ -h, --help Show this.
156
+
157
+ You can run any kind of code with -e / --execute
158
+
159
+ parallel_test -n 5 -e 'ruby -e "puts %[hello from process #{ENV[:TEST_ENV_NUMBER.to_s].inspect}]"'
160
+ hello from process "2"
161
+ hello from process ""
162
+ hello from process "3"
163
+ hello from process "5"
164
+ hello from process "4"
165
+
166
+ <table>
167
+ <tr><td></td><td>1 Process</td><td>2 Processes</td><td>4 Processes</td></tr>
168
+ <tr><td>RSpec spec-suite</td><td>18s</td><td>14s</td><td>10s</td></tr>
169
+ <tr><td>Rails-ActionPack</td><td>88s</td><td>53s</td><td>44s</td></tr>
170
+ </table>
171
+
172
+ TIPS
173
+ ====
174
+ - [Capybara + Selenium] add to env.rb: `Capybara.server_port = 8888 + ENV['TEST_ENV_NUMBER'].to_i`
175
+ - [RSpec] add a `spec/parallel_spec.opts` to use different options, e.g. no --drb (default: `spec/spec.opts`)
176
+ - [RSpec] if something looks fishy try to delete `script/spec`
177
+ - [RSpec] if `script/spec` is missing parallel:spec uses just `spec` (which solves some issues with double-loaded environment.rb)
178
+ - [RSpec] 'script/spec_server' or [spork](http://github.com/timcharper/spork/tree/master) do not work in parallel
179
+ - [RSpec] `./script/generate rspec` if you are running rspec from gems (this plugin uses script/spec which may fail if rspec files are outdated)
180
+ - [Bundler] if you have a `Gemfile` then `bundle exec` will be used to run tests
181
+ - [Capybara setup](https://github.com/grosser/parallel_tests/wiki)
182
+ - [Sphinx setup](https://github.com/grosser/parallel_tests/wiki)
183
+ - [SQL schema format] use :ruby schema format to get faster parallel:prepare`
184
+ - with zsh this would be `rake "parallel:prepare[3]"`
185
+
186
+ TODO
187
+ ====
188
+ - make jRuby compatible [basics](http://yehudakatz.com/2009/07/01/new-rails-isolation-testing/)
189
+ - make windows compatible
190
+
191
+ Authors
192
+ ====
193
+ inspired by [pivotal labs](http://pivotallabs.com/users/miked/blog/articles/849-parallelize-your-rspec-suite)
194
+
195
+ ### [Contributors](http://github.com/grosser/parallel_tests/contributors)
196
+ - [Charles Finkel](http://charlesfinkel.com/)
197
+ - [Indrek Juhkam](http://urgas.eu)
198
+ - [Jason Morrison](http://jayunit.net)
199
+ - [jinzhu](http://github.com/jinzhu)
200
+ - [Joakim Kolsjö](http://www.rubyblocks.se)
201
+ - [Kevin Scaldeferri](http://kevin.scaldeferri.com/blog/)
202
+ - [Kpumuk](http://kpumuk.info/)
203
+ - [Maksim Horbul](http://github.com/mhorbul)
204
+ - [Pivotal Labs](http://www.pivotallabs.com)
205
+ - [Rohan Deshpande](http://github.com/rdeshpande)
206
+ - [Tchandy](http://thiagopradi.net/)
207
+ - [Terence Lee](http://hone.heroku.com/)
208
+ - [Will Bryant](http://willbryant.net/)
209
+ - [Fred Wu](http://fredwu.me)
210
+ - [xxx](https://github.com/xxx)
211
+ - [Levent Ali](http://purebreeze.com/)
212
+ - [Michael Kintzer](https://github.com/rockrep)
213
+ - [nathansobo](https://github.com/nathansobo)
214
+ - [Joe Yates](http://titusd.co.uk)
215
+ - [asmega](http://www.ph-lee.com)
216
+
217
+ [Michael Grosser](http://grosser.it)<br/>
218
+ michael@grosser.it<br/>
219
+ Hereby placed under public domain, do what you want, just do not hold me accountable...
data/VERSION ADDED
@@ -0,0 +1 @@
1
+ 0.6.1
@@ -0,0 +1,2 @@
1
+ #!/usr/bin/env ruby
2
+ exec "#{File.join(File.dirname(__FILE__), 'parallel_test')} -t features #{ARGV * ' '}"
@@ -0,0 +1,2 @@
1
+ #!/usr/bin/env ruby
2
+ exec "#{File.join(File.dirname(__FILE__), 'parallel_test')} -t spec #{ARGV * ' '}"
@@ -0,0 +1,91 @@
1
+ #!/usr/bin/env ruby
2
+ require 'rubygems'
3
+ require 'optparse'
4
+ require 'parallel'
5
+ raise "please ' gem install parallel '" if Gem::Version.new(Parallel::VERSION) < Gem::Version.new('0.4.2')
6
+ $LOAD_PATH << File.join(File.dirname(__FILE__), '..', 'lib')
7
+ require "parallel_tests"
8
+
9
+ options = {}
10
+ OptionParser.new do |opts|
11
+ opts.banner = <<BANNER
12
+ Run all tests in parallel, giving each process ENV['TEST_ENV_NUMBER'] ('', '2', '3', ...)
13
+
14
+ [optional] Only run selected files & folders:
15
+ parallel_test test/bar test/baz/xxx_text.rb
16
+
17
+ Options are:
18
+ BANNER
19
+ opts.on("-n [PROCESSES]", Integer, "How many processes to use, default: available CPUs"){|n| options[:count] = n }
20
+ opts.on("-p", '--pattern [PATTERN]', "run tests matching this pattern"){|pattern| options[:pattern] = pattern }
21
+ opts.on("--no-sort", "do not sort files before running them"){ |no_sort| options[:no_sort] = no_sort }
22
+ opts.on("-m [FLOAT]", "--multiply-processes [FLOAT]", Float, "use given number as a multiplier of processes to run"){ |multiply| options[:multiply] = multiply }
23
+ opts.on("-r", '--root [PATH]', "execute test commands from this path"){|path| options[:root] = path }
24
+ opts.on("-e", '--exec [COMMAND]', "execute this code parallel and with ENV['TEST_ENV_NUM']"){|path| options[:execute] = path }
25
+ opts.on("-o", "--test-options '[OPTIONS]'", "execute test commands with those options"){|arg| options[:test_options] = arg }
26
+ opts.on("-t", "--type [TYPE]", "which type of tests to run? test, spec or features"){|type| options[:type] = type }
27
+ opts.on("--non-parallel", "execute same commands but do not in parallel, needs --exec"){ options[:non_parallel] = true }
28
+ opts.on("--chunk-timeout [TIMEOUT]", "timeout before re-printing the output of a child-process"){|timeout| options[:chunk_timeout] = timeout.to_f }
29
+ opts.on('-v', '--version', 'Show Version'){ puts ParallelTests::VERSION; exit}
30
+ opts.on("-h", "--help", "Show this.") { puts opts; exit }
31
+ end.parse!
32
+
33
+ # get files to run from arguments
34
+ options[:files] = ARGV if ARGV.size > 0
35
+
36
+ num_processes = options[:count] || Parallel.processor_count
37
+ num_processes = num_processes * (options[:multiply] || 1)
38
+
39
+ if options[:execute]
40
+ runs = (0...num_processes).to_a
41
+ results = if options[:non_parallel]
42
+ runs.map do |i|
43
+ ParallelTests.execute_command(options[:execute], i, options)
44
+ end
45
+ else
46
+ Parallel.map(runs, :in_processes => num_processes) do |i|
47
+ ParallelTests.execute_command(options[:execute], i, options)
48
+ end
49
+ end.flatten
50
+ abort if results.any?{|r| r[:exit_status] != 0 }
51
+ else
52
+ lib, name, task = {
53
+ 'test' => ["tests", "test", "test"],
54
+ 'spec' => ["specs", "spec", "spec"],
55
+ 'features' => ["cucumber", "feature", "features"]
56
+ }[options[:type]||'test']
57
+
58
+ require "parallel_#{lib}"
59
+ klass = eval("Parallel#{lib.capitalize}")
60
+
61
+ start = Time.now
62
+
63
+ tests_folder = task
64
+ tests_folder = File.join(options[:root], tests_folder) unless options[:root].to_s.empty?
65
+
66
+ groups = klass.tests_in_groups(options[:files] || tests_folder, num_processes, :no_sort => options[:no_sort], :pattern => options[:pattern])
67
+ num_processes = groups.size
68
+
69
+ #adjust processes to groups
70
+ abort "no #{name}s found!" if groups.size == 0
71
+
72
+ num_tests = groups.inject(0){|sum,item| sum + item.size }
73
+ puts "#{num_processes} processes for #{num_tests} #{name}s, ~ #{num_tests / groups.size} #{name}s per process"
74
+
75
+ test_results = Parallel.map(groups, :in_processes => num_processes) do |group|
76
+ klass.run_tests(group, groups.index(group), options)
77
+ end
78
+
79
+ #parse and print results
80
+ results = klass.find_results(test_results.map{|result| result[:stdout] }*"")
81
+ puts ""
82
+ puts klass.summarize_results(results)
83
+
84
+ #report total time taken
85
+ puts ""
86
+ puts "Took #{Time.now - start} seconds"
87
+
88
+ #exit with correct status code so rake parallel:test && echo 123 works
89
+ failed = test_results.any?{|result| result[:exit_status] != 0 }
90
+ abort "#{name.capitalize}s Failed" if failed
91
+ end
@@ -0,0 +1,36 @@
1
+ require File.join(File.dirname(__FILE__), 'parallel_tests')
2
+
3
+ class ParallelCucumber < ParallelTests
4
+ def self.run_tests(test_files, process_number, options)
5
+ color = ($stdout.tty? ? 'AUTOTEST=1 ; export AUTOTEST ;' : '')#display color when we are in a terminal
6
+ runtime_logging = " --format ParallelCucumber::RuntimeLogger --out #{runtime_log}"
7
+ cmd = "#{color} #{executable}"
8
+ cmd << runtime_logging if File.directory?(File.dirname(runtime_log))
9
+ cmd << " #{options[:test_options]} #{test_files*' '}"
10
+ execute_command(cmd, process_number, options)
11
+ end
12
+
13
+ def self.executable
14
+ if bundler_enabled?
15
+ "bundle exec cucumber"
16
+ elsif File.file?("script/cucumber")
17
+ "script/cucumber"
18
+ else
19
+ "cucumber"
20
+ end
21
+ end
22
+
23
+ def self.runtime_log
24
+ 'tmp/parallel_runtime_cucumber.log'
25
+ end
26
+
27
+ protected
28
+
29
+ def self.test_suffix
30
+ ".feature"
31
+ end
32
+
33
+ def self.line_is_result?(line)
34
+ line =~ /^\d+ (steps|scenarios)/
35
+ end
36
+ end
@@ -0,0 +1,57 @@
1
+ class ParallelCucumber
2
+ class RuntimeLogger
3
+
4
+ def initialize(step_mother, path_or_io, options=nil)
5
+ @io = prepare_io(path_or_io)
6
+ @example_times = Hash.new(0)
7
+ end
8
+
9
+ def before_feature(_)
10
+ @start_at = Time.now.to_f
11
+ end
12
+
13
+ def after_feature(feature)
14
+ @example_times[feature.file] += Time.now.to_f - @start_at
15
+ end
16
+
17
+ def after_features(*args)
18
+ lock_output do
19
+ @io.puts @example_times.map { |file, time| "#{file}:#{time}" }
20
+ end
21
+ end
22
+
23
+ private
24
+
25
+ def prepare_io(path_or_io)
26
+ if path_or_io.respond_to?(:write)
27
+ path_or_io
28
+ else # its a path
29
+ File.open(path_or_io, 'w').close # clean out the file
30
+ file = File.open(path_or_io, 'a')
31
+
32
+ at_exit do
33
+ unless file.closed?
34
+ file.flush
35
+ file.close
36
+ end
37
+ end
38
+
39
+ file
40
+ end
41
+ end
42
+
43
+ # do not let multiple processes get in each others way
44
+ def lock_output
45
+ if File === @io
46
+ begin
47
+ @io.flock File::LOCK_EX
48
+ yield
49
+ ensure
50
+ @io.flock File::LOCK_UN
51
+ end
52
+ else
53
+ yield
54
+ end
55
+ end
56
+ end
57
+ end