spec_distributed 0.0.2

data/History.txt

@@ -0,0 +1,2 @@
+== 1.2.2 / 2007-07-23
+* Initial version, moved from RSpec's subversion.

data/License.txt

@@ -0,0 +1,20 @@
+Copyright (c) 2007 Aslak Hellesoy and Bob Cotton
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/Manifest.txt

@@ -0,0 +1,27 @@
+History.txt
+License.txt
+Manifest.txt
+README.txt
+Rakefile
+examples/first_spec.rb
+examples/fourth_spec.rb
+examples/second_spec.rb
+examples/third_spec.rb
+lib/spec/distributed.rb
+lib/spec/distributed/master_runner.rb
+lib/spec/distributed/rinda_master_runner.rb
+lib/spec/distributed/rinda_slave_runner.rb
+lib/spec/distributed/slave_runner.rb
+lib/spec/distributed/tuple_args.rb
+lib/spec/distributed/version.rb
+scripts/txt2html
+setup.rb
+spec/spec/distributed/rinda_master_runner_spec.rb
+spec/spec/distributed/rinda_slave_runner_spec.rb
+spec/spec/distributed/tuple_args_spec.rb
+spec/spec_helper.rb
+website/index.html
+website/index.txt
+website/javascripts/rounded_corners_lite.inc.js
+website/stylesheets/screen.css
+website/template.rhtml

data/README.txt

@@ -0,0 +1,132 @@
+Spec::Distributed
+=================
+
+Spec::Distributed makes it possible to run specs in a distributed fashion, in parallel 
+on different slaves. It's something you can consider using when you have a *very* slow 
+RSpec suite (for example using Spec::Ui).
+
+== How it works ==
+When you use Spec::Distributed you will have one master process, and two or more slave processes.
+The master distributes behaviours (describe blocks) to slaves via DRb.
+
+== Example ==
+Note: If you want to run the examples from an svn checkout, replace 'spec' with
+'ruby -Ilib ../rspec/bin/spec'
+
+Start two slave runners:
+
+  spec examples/*_spec.rb --require spec/distributed --runner Spec::Distributed::SlaveRunner:druby://localhost:8991
+
+  spec examples/*_spec.rb --require spec/distributed --runner Spec::Distributed::SlaveRunner:druby://localhost:8992
+
+Start master runner:
+  
+  spec examples/*_spec.rb --require spec/distributed --runner  Spec::Distributed::MasterRunner:druby://localhost:8991,druby://localhost:8992
+  
+== Using Subversion ==
+It is very important that the slaves and the master have identical sources. If the master is run from
+a Subversion working copy, it will automatically detect the local revision and tell the slaves to 
+update accordingly prior to running the behaviours.
+
+== Spec::Ui and formatters ==
+Slaves should be using Spec::Ui::SlaveScreenshotFormatter and the master should be using
+Spec::Ui::MasterScreenshotFormatter. In order to get a report without dead links to screenshot
+PNGs and browser HTML snapshots, all formatters must write to the same location.
+Since slaves and masters typically will run on different machines, you might want to set
+up a shared location using Samba or NTFS shares.
+
+== Using Rinda for Autodiscovery ==
+The slave class Spec::Distributed::RindaSlaveRunner will be used in
+conjunction with Spec::Distributed::RindaMasterRunner so that masters
+and slaves may auto-discover each other.
+
+To use the Rinda Runners start one more slave runner:
+
+  spec examples/*_spec.rb --require spec/distributed --runner Spec::Distributed::RindaSlaveRunner
+
+  spec examples/*_spec.rb --require spec/distributed --runner Spec::Distributed::RindaSlaveRunner
+
+Then start the master runner:
+
+  spec examples/*_spec.rb --require spec/distributed --runner Spec::Distributed::MasterRunner
+
+Note the slaves and masters don't need prior knowledge of each other.
+
+The slave runner will attempt to contact any RingServer on the local
+network. If none exists it will start one. Subsequent slaves will use
+this RingServer to publish themselves.
+
+When the master starts, it will contact the RingServer and query for
+all available slave servers. Then the master will create a thread for
+each available slave.
+
+When the master uses a slave, it removes it from the pool of available
+slaves. The slave will re-publish itself back into the tuplespace
+after running the spec.
+
+== Partitioning the Tuplespace ==
+With no additional configuration options passed to either the Master
+or Slave runners the RindaMasterRunner will use all available slaves.
+
+Suppose you have more than one set of masters and slaves running at
+the same time. For example, Bob and Joe want to run a pool of slaves,
+but don't want to share them with each other. One solution would be to
+run seperate RingServers on different ports, but that defeats the
+purpose of auto-discovery.
+
+Both Spec::Distributed::RindaSlaveRunner and
+Spec::Distributed::RindaMasterRunner take an optional list of
+"tuplespace selectors", which are a comma seperated list of strings.
+
+For example, to Joe might start his slaves like this:
+
+  spec examples/*_spec.rb --require spec/distributed --runner Spec::Distributed::RindaSlaveRunner:Joe
+
+Joe would then start his RindaMasterRunner as so:
+
+  spec examples/*_spec.rb --require spec/distributed --runner Spec::Distributed::RindaMasterRunner:Joe
+  
+This master runner will only find slaves that have been configured
+with the same arguments.
+
+Joe may also have several builds he want to test, so he might setup
+two pools of slave servers to run:
+
+  spec examples/*_spec.rb --require spec/distributed --runner Spec::Distributed::RindaSlaveRunner:Joe,1
+  spec examples/*_spec.rb --require spec/distributed --runner Spec::Distributed::RindaSlaveRunner:Joe,2
+
+Then Joe could create two master runners, one for each build:
+
+  spec examples/*_spec.rb --require spec/distributed --runner Spec::Distributed::RindaMasterRunner:Joe,1
+  spec examples/*_spec.rb --require spec/distributed --runner Spec::Distributed::RindaMasterRunner:Joe,2
+
+Again, the master runners would only find slaves that have been
+configured with the same parameters.
+
+== Wildcarding the Tuplespace ==
+
+To continue with the example, lets suppose that Bob knows that Joe is
+out to lunch, and wants to use some of his slave runners while he
+gone. Bob has his own slave runners configured similarly to Joe's:
+
+  spec examples/*_spec.rb --require spec/distributed --runner Spec::Distributed::RindaSlaveRunner:Bob,1
+  spec examples/*_spec.rb --require spec/distributed --runner Spec::Distributed::RindaSlaveRunner:Bob,2
+
+So Bob wants to use all of Joe's (and everyone's) build #2 slave
+servers. So he starts his master runner and passes in a wild-card in
+the first position:
+
+  spec examples/*_spec.rb --require spec/distributed --runner Spec::Distributed::RindaMasterRunner:*,2
+
+This will select all the slave runners that were configured with two
+arguments, and the value of the second argument is 2.
+
+Which is to say, slaves will only be selected if the number of
+"tuplespace selectors" matches, and all of the values match or are a
+wildcard (*). Zero selectors will only match slaves started with zero
+selectors, a single wild-card will only match slaves started with one
+selector.
+
+This can be useful for partitioning seperate builds, platforms, dev
+groups etc.
+

data/Rakefile

@@ -0,0 +1,133 @@
+require 'rubygems'
+require 'rake'
+require 'rake/clean'
+require 'rake/testtask'
+require 'rake/packagetask'
+require 'rake/gempackagetask'
+require 'rake/rdoctask'
+require 'rake/contrib/rubyforgepublisher'
+require 'fileutils'
+require 'hoe'
+require 'spec/rake/spectask'
+
+include FileUtils
+require File.join(File.dirname(__FILE__), 'lib', 'spec', 'distributed', 'version')
+
+AUTHOR = 'Spec::Distributed'
+EMAIL = 'aslak.hellesoy@gmail.com', 'bob.cotton@rallydev.com'
+DESCRIPTION = "Run RSpec distributed with DRb or Rinda"
+GEM_NAME = 'spec_distributed' # what ppl will type to install your gem
+
+@config_file = "~/.rubyforge/user-config.yml"
+@config = nil
+def rubyforge_username
+  unless @config
+    begin
+      @config = YAML.load(File.read(File.expand_path(@config_file)))
+    rescue
+      puts <<-EOS
+ERROR: No rubyforge config file found: #{@config_file}"
+Run 'rubyforge setup' to prepare your env for access to Rubyforge
+ - See http://newgem.rubyforge.org/rubyforge.html for more details
+      EOS
+      exit
+    end
+  end
+  @rubyforge_username ||= @config["username"]
+end
+
+RUBYFORGE_PROJECT = 'rspec-ext' # The unix name for your project
+HOMEPATH = "http://#{RUBYFORGE_PROJECT}.rubyforge.org"
+
+
+#REV = YAML.load(`svn info`)['Revision'] rescue nil
+REV=nil
+VERS = Spec::Distributed::VERSION::STRING + (REV ? ".#{REV}" : "")
+CLEAN.include ['**/.*.sw?', '*.gem', '.config', '**/.DS_Store']
+RDOC_OPTS = ['--quiet', '--title', 'spec_distributed documentation',
+    "--opname", "index.html",
+    "--line-numbers", 
+    "--main", "README",
+    "--inline-source"]
+
+class Hoe
+  def extra_deps 
+    @extra_deps.reject { |x| Array(x).first == 'hoe' } 
+  end 
+end
+
+# Generate all the Rake tasks
+# Run 'rake -T' to see list of generated tasks (from gem root directory)
+hoe = Hoe.new(GEM_NAME, VERS) do |p|
+  p.author = AUTHOR 
+  p.description = DESCRIPTION
+  p.email = EMAIL
+  p.summary = DESCRIPTION
+  p.url = HOMEPATH
+  p.rubyforge_name = RUBYFORGE_PROJECT if RUBYFORGE_PROJECT
+  p.test_globs = ["test/**/test_*.rb"]
+  p.clean_globs |= CLEAN  #An array of file patterns to delete on clean.
+  
+  # == Optional
+  p.changes = p.paragraphs_of("History.txt", 0..1).join("\n\n")
+  p.extra_deps = [['rspec', '>= 1.0.8']]     # An array of rubygem dependencies [name, version], e.g. [ ['active_support', '>= 1.3.1'] ]
+  #p.spec_extras = {}    # A hash of extra values to set in the gemspec.
+end
+
+CHANGES = hoe.paragraphs_of('History.txt', 0..1).join("\n\n")
+PATH    = (RUBYFORGE_PROJECT == GEM_NAME) ? RUBYFORGE_PROJECT : "#{RUBYFORGE_PROJECT}/#{GEM_NAME}"
+hoe.remote_rdoc_dir = File.join(PATH.gsub(/^#{RUBYFORGE_PROJECT}\/?/,''), 'rdoc')
+
+desc 'Generate website files'
+task :website_generate do
+  Dir['website/**/*.txt'].each do |txt|
+    sh %{ ruby scripts/txt2html #{txt} > #{txt.gsub(/txt$/,'html')} }
+  end
+end
+
+desc 'Upload website files to rubyforge'
+task :website_upload do
+  host = "#{rubyforge_username}@rubyforge.org"
+  remote_dir = "/var/www/gforge-projects/#{PATH}/"
+  local_dir = 'website'
+  sh %{rsync -aCv #{local_dir}/ #{host}:#{remote_dir}}
+end
+
+desc 'Generate and upload website files'
+task :website => [:website_generate, :website_upload, :publish_docs]
+
+desc 'Release the website and new gem version'
+task :deploy => [:check_version, :website, :release] do
+  puts "Remember to create SVN tag:"
+  puts "svn copy svn+ssh://#{rubyforge_username}@rubyforge.org/var/svn/#{PATH}/trunk " +
+    "svn+ssh://#{rubyforge_username}@rubyforge.org/var/svn/#{PATH}/tags/REL-#{VERS} "
+  puts "Suggested comment:"
+  puts "Tagging release #{CHANGES}"
+end
+
+desc 'Runs tasks website_generate and install_gem as a local deployment of the gem'
+task :local_deploy => [:website_generate, :install_gem]
+
+task :check_version do
+  unless ENV['VERSION']
+    puts 'Must pass a VERSION=x.y.z release version'
+    exit
+  end
+  unless ENV['VERSION'] == VERS
+    puts "Please update your version.rb to match the release version, currently #{VERS}"
+    exit
+  end
+end
+
+desc "Run specs"
+Spec::Rake::SpecTask.new do |t|
+  t.spec_opts = ['--options', "spec/spec.opts"]
+  t.spec_files = FileList['spec/*_spec.rb']
+end
+
+# Hoe insists on setting task :default => :test
+# !@#$ no easy way to empty the default list of prerequisites
+Rake::Task['default'].send :instance_variable_set, "@prerequisites", FileList[]
+desc "Default task is to run specs"
+task :default => :spec
+

data/examples/first_spec.rb

@@ -0,0 +1,9 @@
+describe "first" do
+  it "example a" do
+    sleep 1
+  end
+
+  it "example b" do
+    sleep 1
+  end
+end

data/examples/fourth_spec.rb

@@ -0,0 +1,14 @@
+describe "fourth" do
+  it "example g" do
+    sleep 1
+  end
+
+  it "example h" do
+    sleep 0.5
+    1.should == 2
+  end
+
+  it "example i" do
+    sleep 0.5
+  end
+end

data/examples/second_spec.rb

@@ -0,0 +1,10 @@
+describe "second" do
+  it "example c" do
+    sleep 1
+    raise "error"
+  end
+
+  it "example d" do
+    sleep 1
+  end
+end

data/examples/third_spec.rb

@@ -0,0 +1,9 @@
+describe "third" do
+  it "example e" do
+    sleep 1
+  end
+
+  it "example f" do
+    sleep 1
+  end
+end

data/lib/spec/distributed.rb

@@ -0,0 +1,6 @@
+require 'spec/distributed/version'
+require 'spec/distributed/master_runner'
+require 'spec/distributed/slave_runner'
+require 'spec/distributed/tuple_args'
+require 'spec/distributed/rinda_master_runner'
+require 'spec/distributed/rinda_slave_runner'

data/lib/spec/distributed/master_runner.rb

@@ -0,0 +1,74 @@
+require 'thread'
+require 'spec/distributed/slave_runner'
+
+module Spec
+  module Distributed
+    class MasterRunner < ::Spec::Runner::BehaviourRunner
+      def initialize(options, args=nil)
+        super(options)
+        process_args(args)
+      end
+
+      def process_args(args)
+        @slave_urls = args.split(",")
+        raise "You must pass the DRb URLs: --runner #{self.class}:druby://host1:port1,drb://host2:port2" if @slave_urls.empty?
+      end
+      
+      def run(paths, exit_when_done)
+        @master_paths = paths
+        @svn_rev = `svn info`.match(/Revision: (\d+)/m)[1] rescue nil
+        STDERR.puts "WARNING - no local svn revision found. Your slaves may be out of sync." if @svn_rev.nil?
+        super(paths, exit_when_done)
+      end
+      
+      def run_behaviours
+        DRb.start_service
+        behaviour_reports = Queue.new
+        index_queue = Queue.new
+        @behaviours.length.times {|index| index_queue << index}
+
+        @threads = slave_runners.map do |slave_runner|
+          Thread.new do
+            slave_runner.prepare_run(@master_paths, @svn_rev)
+            drb_error = nil
+            while !index_queue.empty?
+              begin
+                i = index_queue.pop
+                behaviour = @behaviours[i]
+                behaviour_report = slave_runner.run_behaviour_at(i, @options.dry_run, @options.reverse, @options.timeout)
+                behaviour_reports << behaviour_report
+              rescue DRb::DRbConnError => e
+                # Maybe the slave is down. Put the index back and die
+                index_queue << i
+                drb_error = e
+                break
+              end
+            end
+            
+            unless drb_error
+              slave_runner.report_end
+              slave_runner.report_dump
+            end
+          end
+        end
+
+        return unless @threads.length > 0
+        
+        # Add a last thread for the reporter
+        @threads << Thread.new do
+          @behaviours.length.times do
+            behaviour_reports.pop.replay(@options.reporter)
+          end
+        end
+
+        @threads.each do |t| 
+          t.join
+        end
+      end
+
+      def slave_runners
+        @slave_urls.map { |slave_url| DRbObject.new_with_uri(slave_url) }
+      end
+    end
+  end
+end