testbot 0.2.6

data/CHANGELOG

@@ -0,0 +1,62 @@
+0.2.5
+
+Made testbot into a gem. Simplified the design. Added a CLI. Removed git support.
+
+0.2.3
+
+Added JRuby support.
+
+0.2.2
+
+Replaced runtime optimization code with a simpler file size based method.
+
+0.2.1
+
+Added support for Test:Unit and multiple projects.
+
+0.2.0
+
+Added "ssh_tunnel" so that you don't have to be able to access the http server directly.
+
+0.1.9
+
+The server can now handle if a runner is shutdown in the middle of a test run (it will give the
+job to another runner).
+
+0.1.8
+
+Added cucumber support and failure detection to the client. Added docs. Added code that uses
+historic test runtimes to reduce test runtime (in theory).
+
+0.1.7
+
+Added a basic client so that testbot can be installed as a rails plugin. 'rake testbot:spec'.
+
+0.1.6
+
+Now only fetching the code once, and only keeping one copy of it. Less overhead.
+
+0.1.5
+
+Added "/runners/total_instances" to be able to balance requesters by knowing how many instances
+there are in total.
+
+0.1.4
+
+Added support for fetching the latest version from git instead of rsync when doing a test run.
+
+0.1.3
+
+Added "/runners/available" to see how many runners are available.
+
+0.1.2
+
+Added cucumber support.
+
+0.1.1
+
+Added a CPU usage check to make sure that the runner does not start a job when the computer is busy.
+
+0.1
+
+Added hostname and STDERR to runner results.

data/Gemfile

@@ -0,0 +1,12 @@
+source "http://rubygems.org"
+gemspec
+
+group :development do
+  gem "shoulda"
+  gem "rack-test"
+  gem "flexmock"
+  gem "cucumber"
+  gem "rvm"
+  gem "rake"
+end
+

data/README.markdown

@@ -0,0 +1,128 @@
+Testbot is a test distribution tool that works with Rails, RSpec, Test::Unit and Cucumber. The basic idea is that you let testbot spread the load of running your tests across multiple machines to make the tests run faster.
+
+Using testbot on 11 machines (25 cores) we got our test suite down to **2 minutes from 30**. [More examples of how testbot is used](http://github.com/joakimk/testbot/wiki/How-testbot-is-being-used).
+
+Installing
+----
+
+    gem install testbot
+
+Try it out
+----
+
+    testbot --server
+    testbot --runner --connect localhost
+    mkdir -p testbotdemo/test; cd testbotdemo
+    echo 'require "test/unit"' > test/demo_test.rb
+    echo 'class DemoTest < Test::Unit::TestCase; def test_first; end; end' >> test/demo_test.rb
+    testbot --test --connect localhost
+        
+    # Cleanup
+    testbot --server stop
+    testbot --runner stop
+    cd ..; rm -rf testbotdemo
+    rm -rf /tmp/testbot
+
+The project files from the demo project are synced to /tmp/testbot/$USER (default). The runner syncs the files to /tmp/testbot/project (default). The tests are then run and the results returned through the server and displayed.
+
+How it works
+----
+
+Testbot is:
+
+* A **server** to distribute test jobs.
+* One or more **runners** to run test jobs and return the results (this is the "slave" process that runs tests).
+* One or more **requesters** that tells the server which tests to distribute and displays the results (the client you use to run tests, for example: **rake testbot:spec**).
+
+<pre>
+    Requester -- (files to run) --> Server -- (files to run) --> (many-)Runner(s)
+        ^                           |    ^                                  |
+        |---------------------------|    |----------------------------------|
+                 (results)                            (results)
+</pre>
+
+Example setup
+----
+
+Here I make the assumption that you have a user called **testbot** on a server at **192.168.0.100** that every computer [can log into without a password](http://github.com/joakimk/testbot/wiki/SSH-Public-Key-Authentication) and that you have **installed testbot** on each computer.
+
+    ssh testbot@192.168.0.100
+    testbot --server
+    
+On every computer that should share CPU resources run:
+
+    testbot --runner --connect 192.168.0.100
+
+Running tests:
+    
+    testbot --test --connect 192.168.0.100
+    # --test could also be --spec or --features
+
+Using testbot with Rails 3:
+
+    rails g testbot --connect 192.168.0.100
+    rake testbot:spec (or :test, :features)
+
+    # Gemfile:
+    gem 'testbot'
+
+Using testbot with Rails 2:
+
+    ruby script/plugin install git://github.com/joakimk/testbot.git -r 'refs/tags/v0.2.6'
+
+The rails 2 config generator isn't done yet so you can look at the following examples instead:
+  [lib/tasks/testbot.rake](https://gist.github.com/715836) and
+  [config/testbot.yml](https://gist.github.com/715839).
+
+    rake testbot:spec (or :test, :features)
+
+Updating testbot
+----
+
+To simplify updates there is a **--auto_update** option for the runner. The runner processes that use this option will be automatically updated and restarted when you change the server version.
+
+This requires testbot to be installed **without sudo** as the update simply runs "gem install testbot -v new_version". I recommend using [RVM](http://rvm.beginrescueend.com/) (it handles paths correctly).
+
+Example:
+    testbot --runner --connect 192.168.0.100 --auto_update
+
+More options
+----
+
+    testbot (or testbot --help)
+
+Could this readme be better somehow?
+----
+
+If there is anything missing or unclear you can create an [issue](http://github.com/joakimk/testbot/issues) (or send me a pull request).
+
+Features
+----
+* You can add and remove computers at any time. Testbot simply gives abandoned jobs to other computers.
+* Testbot will try to balance the testload so that every computer finishes running the tests at the same time to reduce the time it takes to run the entire test suite. It does a good job, but has potential for further improvement.
+* You can access your testbot network through SSH by using the built in SSH tunneling code.
+* Testbot is continuously tested for compability with Ruby 1.8.7 and 1.9.2.
+
+Contributing to testbot
+----
+
+First, get the tests to run:
+    bundle
+    rake
+
+For development I recommend: [grosser/autotest](http://github.com/grosser/autotest)
+    autotest -f -c
+
+Make your change (don't forget to write tests) and send me a pull request.
+
+You can also contribute by adding to the [wiki](http://github.com/joakimk/testbot/wiki).
+
+How to add support for more test frameworks and/or programming languages
+----
+
+Add a **lib/adapters/framework_name_adapter.rb** file, update **lib/adapters/adapter.rb** and this readme.
+
+More
+----
+
+* Check the [wiki](http://github.com/joakimk/testbot/wiki) for more info.

data/bin/testbot

@@ -0,0 +1,59 @@
+#!/usr/bin/env ruby
+
+require File.expand_path(File.join(File.dirname(__FILE__), '../lib/testbot.rb'))
+
+def show_help
+  puts "Testbot #{Testbot::VERSION}"
+  puts
+  puts "Testbot is a test distribution tool that works with Rails, RSpec, Test::Unit and Cucumber."
+  puts
+  puts "More info: http://github.com/joakimk/testbot"
+  puts "Wiki:      http://github.com/joakimk/testbot/wiki"
+  puts
+  puts "Examples:"
+  puts "  testbot --server"
+  puts "  testbot --runner --connect 192.168.0.100"
+  puts "  testbot --test --connect 192.168.0.100"
+  puts 
+  puts "Types:"
+  puts "  --server <start|stop|run>"
+  puts "  --runner <start|stop|run>"
+  Adapter.all.each do |adapter|
+    puts "  --#{adapter.type}\t\t\t# Run #{adapter.name} tests"
+  end
+  puts
+  puts "Runner options:"
+  puts "  --connect <host_or_ip>\t# Which server to use (required)"
+  puts "  --working_dir <path>\t\t# Where to store temporary files (default: #{Testbot::DEFAULT_WORKING_DIR})"
+  puts "  --cpus <number>\t\t# The number of CPU cores to use (default: use all)"
+  puts "  --ssh_tunnel\t\t\t# Use a ssh tunnel"
+  puts "  --user <username>\t\t# Use a custom ssh tunnel user (default: testbot)"
+  puts "  --auto_update\t\t\t# Keep testbot updated to the same version as the server."
+  puts "  --max_jruby_instances <num>\t# To use less instances when running JRuby (as it requires more memory)"
+  puts "  --jruby_opts <-J-X...>\t# Options to JRuby."
+  puts
+  puts "Test options:"
+  puts "  --connect <host_or_ip>\t# Which server to use (required)"
+  puts "  --rsync_ignores <ignores>\t# Files to ignore when syncing (default: include all)"
+  puts "  --rsync_path <path>\t\t# Sync path on the server (default: #{Testbot::DEFAULT_SERVER_PATH})"
+  puts "  --ssh_tunnel\t\t\t# Use a ssh tunnel"
+  puts "  --user <username>\t\t# Use a custom rsync / ssh tunnel user (default: #{Testbot::DEFAULT_USER})"
+  puts "  --project <project_name>\t# Use a custom project name (default: #{Testbot::DEFAULT_PROJECT})"
+  puts
+  puts "Other:"
+  puts "  --help\t\t\t# Show help (this page)"
+  puts "  --version\t\t\t# Show the testbot version"
+
+  # puts "                                           # (when the server gem version is changed,"
+  # puts "                                           #  this runs gem install with the same version"
+  # puts "                                           #  and restarts the test runner)"
+
+
+  # TODO:
+  # puts " --use_git_ignore                          # Don't rsync files that are ignored by git"
+  # puts "  --status                                 # Show running background processes"
+  # puts "  --port <number>                          # Use a custom port"
+
+end
+
+show_help unless Testbot::CLI.run(ARGV)

data/lib/adapters/adapter.rb

@@ -0,0 +1,22 @@
+require File.dirname(__FILE__) + '/rspec_adapter'
+require File.dirname(__FILE__) + '/cucumber_adapter'
+require File.dirname(__FILE__) + '/test_unit_adapter'
+
+class Adapter
+  def self.all
+    [ RSpecAdapter, CucumberAdapter, TestUnitAdapter ]
+  end
+
+  def self.find(type)
+    case type.to_sym
+    when :spec
+      RSpecAdapter
+    when :features
+      CucumberAdapter
+    when :test
+      TestUnitAdapter
+    else
+      raise "Unknown adapter: #{type}"
+    end
+  end
+end

data/lib/adapters/cucumber_adapter.rb

@@ -0,0 +1,31 @@
+class CucumberAdapter
+  
+  def self.command(ruby_interpreter, files)
+    "export AUTOTEST=1; #{ruby_interpreter} script/cucumber -f progress --backtrace -r features/support -r features/step_definitions #{files} -t ~@disabled_in_cruise"
+  end
+  
+  def self.file_pattern
+    '**/**/*.feature'
+  end
+  
+  def self.requester_port
+    2230
+  end
+  
+  def self.pluralized
+    'features'
+  end
+  
+  def self.base_path
+    pluralized
+  end
+  
+  def self.name
+    'Cucumber'
+  end
+  
+  def self.type
+    pluralized
+  end
+  
+end

data/lib/adapters/rspec_adapter.rb

@@ -0,0 +1,31 @@
+class RSpecAdapter
+  
+  def self.command(ruby_interpreter, files)
+    "export RSPEC_COLOR=true; #{ruby_interpreter} script/spec -O spec/spec.opts #{files}"
+  end
+  
+  def self.file_pattern
+    '**/**/*_spec.rb'
+  end
+  
+  def self.requester_port
+    2299
+  end
+  
+  def self.pluralized
+    'specs'
+  end
+  
+  def self.base_path
+    type
+  end
+  
+  def self.name
+    'RSpec'
+  end
+  
+  def self.type
+    'spec'
+  end
+  
+end

data/lib/adapters/test_unit_adapter.rb

@@ -0,0 +1,31 @@
+class TestUnitAdapter
+  
+  def self.command(ruby_interpreter, files)
+    "#{ruby_interpreter} -Itest -e '%w(#{files}).each { |file| require(file) }'"
+  end
+  
+  def self.file_pattern
+    '**/**/*_test.rb'
+  end
+  
+  def self.requester_port
+    2231
+  end
+  
+  def self.pluralized
+    'tests'
+  end
+  
+  def self.base_path
+    type
+  end
+  
+  def self.name
+    'Test::Unit'
+  end
+  
+  def self.type
+    'test'
+  end  
+  
+end

data/lib/generators/testbot/templates/testbot.rake.erb

@@ -0,0 +1,35 @@
+namespace :testbot do
+  task :before_request do
+    # This is run after you start a request (ex: rake testbot:spec)
+  end
+  
+  task :before_run do
+    # This is run by the runner after files are synced but before tests are run
+
+    # Example: Setting up a test database
+    database_yml = <<-DB_CONFIG
+test:
+  adapter: mysql
+  encoding: utf8
+  database: <%= options.project %>_testbot_test<%%= ENV['TEST_ENV_NUMBER'] %>
+  username: root
+  password:
+  host: localhost
+DB_CONFIG
+
+    # database_file_path = "config/database.yml"
+    # File.open(database_file_path, 'w') { |f| f.write(database_yml) }
+    # 
+    # # Setup databases for all instances
+    # 0.upto(ENV['TEST_INSTANCES'].to_i - 1) do |instance|
+    #   test_env_number = (instance == 0) ? '' : instance + 1
+    #   system "mysqladmin -u root -f drop <%= options.project %>_testbot_test#{test_env_number}  > /dev/null 2>&1"
+    #   system "mysqladmin -u root -f create <%= options.project %>_testbot_test#{test_env_number} > /dev/null 2>&1"
+    #   system "export RAILS_ENV=test; export TEST_ENV_NUMBER=#{test_env_number}; rake db:test:load"
+    # end
+
+    # Example: Building gems
+    # system "rm vendor/gems/*/ext/**/*.o > /dev/null 2>&1"
+    # system "rake gems:build:force > /dev/null 2>&1"
+  end
+end

data/lib/generators/testbot/templates/testbot.yml.erb

@@ -0,0 +1,40 @@
+# Which server to use.
+server_host: <%= options.connect %>
+<%- if options.user -%>
+server_user: <%= options.user %>
+<%- else -%>
+# server_user: <%= Testbot::DEFAULT_USER %>
+<%- end -%>
+
+# Project prefix. Used for prefixing project files
+# so that you can run multiple projects in the same testbot network.
+<%- if options.project %>
+project: <%= options.project %>
+<%- else -%>
+# project: <%= Testbot::DEFAULT_PROJECT %>
+<%- end -%>
+
+# RSync settings. The folder where your files are synced to
+# and then fetched from before running the tests.
+<%- if options.rsync_path -%>
+rsync_path: <%= options.rsync_path %>
+<%- else -%>
+# rsync_path: <%= Testbot::DEFAULT_SERVER_PATH %>
+<%- end -%>
+<%- if options.rsync_ignores -%>
+rsync_ignores: <%= options.rsync_ignores %>
+<%- else -%>
+# rsync_ignores:
+<%- end -%>
+
+# To tunnel traffic through SSH
+<%- if options.ssh_tunnel -%>
+ssh_tunnel: true
+<%- else -%>
+# ssh_tunnel: true
+<%- end -%>
+
+# Runner usage. Set to a lower percentage to not use
+# every available instance or higher to create more
+# jobs than there are instances.
+available_runner_usage: 100%