testbot_instructure 0.7.8

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: e91a209692e050fa35241bdfd5f9b07b244e2cf3
+  data.tar.gz: ddf819aa30078f4dc5615ad80e7038a8ab7eff46
+SHA512:
+  metadata.gz: 48be5c7591e44b08241add8f77ae2a1b770d47bcd13cfb25add5366ef80bbde7b7240b9d55838fdafc93f4b09003616479ae473a926958f5457b7cc559199236
+  data.tar.gz: 2cbf130feb93f11f9915dc81094cedb931303bbbae5bf2cf4d87747ca4924117359a45d016a9994e63c22e17ea7eaf50818979ef6df90ebd84a261ca2f5ac033

data/CHANGELOG

@@ -0,0 +1,264 @@
+0.7.8
+
+Fix test job bug that caused multiple jobs to use the same TEST_ENV_NUMBER.
+
+This caused multiple processes to use the same databases, etc.
+
+If you're seeing database deadlocks in tests, this will probably fix that.
+
+0.7.7
+
+Exit when file syncing fails so that you don't run tests for a version of the
+code you did not expect.
+
+0.7.5
+
+Fixed bug where a runner updating without a result would fail.
+
+Also dropping official support for 1.8 as it's such a hassle to maintain.
+
+0.7.3
+
+Increased time between updates to avoid timeouts. Allow timeout when posting
+incremental build results (but not when posting the final results).
+
+Should make testbot more stable for some people.
+
+0.7.2
+
+Increased timeout on http requests to the server to 10 seconds.
+
+0.7.1
+
+Fixed a bug introduced in 0.7.0. The TEST_INSTANCES env-variable are now
+present again when running before_run scripts.
+
+0.7.0
+
+Added per project rvm support. Using this you can run testbot using system ruby
+while being able to run test processes using the project rvm environment. This
+is also useful if you have several projects with different rvm rubies and gemsets.
+
+0.6.9
+
+Removed dependency on macaddr as that does not work on some platforms.
+
+0.6.8
+
+Fixed broken gem lookup in ruby 1.9.
+
+0.6.6
+
+The runner will now run config/testbot/before_run.rb if it exists.
+
+Replaced the old process code with the posix-spawn gem's methods. This fixed
+process shutdown in my current setup. Hopefully more stable overall.
+
+The requester will now request that the build is stopped when exiting early (like when killed),
+and not only on ctrl+c (SIGINT).
+
+Most of this was based on a fork by https://github.com/Skalar.
+
+0.6.5
+
+Fixed test unit require path so that it works with ruby 1.9.
+
+0.6.4
+
+Fixed bug with running test unit tests that caused errors like '/bin/ruby:1: Invalid char `\317' in expression`'.
+
+Now handling errors creating builds.
+
+Returning 503 and reporting that there are no runners when that is the case.
+
+0.6.3
+
+Results are now shown as they happen. It takes care not to break escape codes used for
+coloring output (only tested with rspec, add a script/spec file containing "rspec --tty $@").
+
+Also, you can add a "logging: true" option to testbot.yml to see when files are synced, etc.
+
+0.6.2
+
+Fixed "invalid byte sequence in UTF-8" errors cased by test output in some cases.
+
+0.6.1
+
+Fixed CPU core check for OSX Lion.
+
+Updated sinatra requirement so that it's a bit less strict
+(https://github.com/joakimk/testbot/pull/41).
+
+0.6.0
+
+Fixed bug that caused the runner to crash and block the server after checking for jobs (issue #34).
+
+Merged in https://github.com/joakimk/testbot/pull/33 (Bugfix: Run rake task testbot:before_run with bundler if present).
+
+0.5.9
+
+When you hit ctrl+c all related test jobs will be stopped (within about 5 seconds). You can
+now exit, change code and re-run much faster than before.
+
+It does not stop before_run or code fetch. But no tests will be run after
+those complete when the build is stopped.
+
+0.5.8
+
+Now only running tests from one build on a runner at a time.
+
+This is mostly usability. Testbot should behave like when you're running your tests locally.
+
+0.5.7
+
+Added @meeiw's patch to support ERB in config. Added test result summarization for RSpec and Cucumber.
+
+0.5.6
+
+Removed CPU usage check before running jobs (issue #25).
+
+0.5.5
+
+Added RSpec2 support. Thanks to Bryan Helmkamp, https://github.com/brynary.
+
+0.5.4
+
+Fixed some typos that caused the ruby interpreter to always be "ruby".
+
+0.5.3
+
+Added support for rubygems-test and http://gem-testers.org.
+
+0.5.1
+
+Fixed a bug that caused exit status to be 1 even if the test run is successful.
+
+0.5.0
+
+Made the status page load faster but check less often.
+
+0.4.9
+
+There is now a very basic status page at http://testbot_server:2288/status.
+
+0.4.8
+
+Removed all remaining native dependencies to make testbot simpler to install.
+
+0.4.7
+
+Refactored the code into modules with one directory for each. No longer dependent on mongel, now using webrick.
+
+0.4.6
+
+Fixed a bug that caused auto_update not to check for an update after a job had been run.
+
+0.4.5
+
+Made auto_update a bit more reliable.
+
+0.4.4
+
+Changed to using the deamons gem. Seems to fix some problems with old processes
+not closing down when using auto_update.
+
+0.4.2
+
+Added support for quick deploys when developing testbot.
+
+0.4.0
+
+Fixed RSpec 2 compability issue.
+
+0.3.9
+
+Added support for running tests with bundler exec.
+
+0.3.8
+
+Testbot now uses exit code to determine if the test run has failed.
+
+0.3.7
+
+The runner now checks right away for another job if it got one last time around.
+
+0.3.6
+
+Added Rails 2 generator support.
+
+0.3.5
+
+Fixed ssh_tunnel bug when using a custom user.
+
+0.3.4
+
+Bundler support. Testbot now runs "bundle" before "testbot:before_run" if you
+have "bundle" in path and have a "Gemfile" in the project so that the
+rails environment can load.
+
+0.3.3
+
+Fixed loading issue in rails 2, testbot depended on having rails installed as a gem.
+
+0.3.0
+
+Made testbot into a gem. Simplified the design. Added a CLI. Removed git support.
+
+0.2.3
+
+Added JRuby support.
+
+0.2.2
+
+Replaced runtime optimization code with a simpler file size based method.
+
+0.2.1
+
+Added support for Test:Unit and multiple projects.
+
+0.2.0
+
+Added "ssh_tunnel" so that you don't have to be able to access the http server directly.
+
+0.1.9
+
+The server can now handle if a runner is shutdown in the middle of a test run (it will give the
+job to another runner).
+
+0.1.8
+
+Added cucumber support and failure detection to the client. Added docs. Added code that uses
+historic test runtimes to reduce test runtime (in theory).
+
+0.1.7
+
+Added a basic client so that testbot can be installed as a rails plugin. 'rake testbot:spec'.
+
+0.1.6
+
+Now only fetching the code once, and only keeping one copy of it. Less overhead.
+
+0.1.5
+
+Added "/runners/total_instances" to be able to balance requesters by knowing how many instances
+there are in total.
+
+0.1.4
+
+Added support for fetching the latest version from git instead of rsync when doing a test run.
+
+0.1.3
+
+Added "/runners/available" to see how many runners are available.
+
+0.1.2
+
+Added cucumber support.
+
+0.1.1
+
+Added a CPU usage check to make sure that the runner does not start a job when the computer is busy.
+
+0.1
+
+Added hostname and STDERR to runner results.

data/Gemfile

@@ -0,0 +1,3 @@
+source "http://rubygems.org"
+gemspec
+

data/README.markdown

@@ -0,0 +1,141 @@
+Testbot is a test distribution tool that works with Rails, RSpec, RSpec2, Test::Unit and Cucumber. The basic idea is that you let testbot spread the load of running your tests across multiple machines to make the tests run faster.
+
+Using testbot on 11 machines (25 cores) we got our test suite down to **2 minutes from 30**. [More examples of how testbot is used](http://github.com/joakimk/testbot/wiki/How-testbot-is-used).
+
+If you intend to use testbot with cloud computing (like EC2), take a look at [TestbotCloud](https://github.com/joakimk/testbot_cloud).
+
+Installing
+----
+
+    gem install testbot
+
+Try it out
+----
+
+    testbot --server
+    testbot --runner --connect localhost
+    sleep 5 # wait for the runner to register with the server
+
+    mkdir -p testbotdemo/test; cd testbotdemo
+    echo 'require "test/unit"' > test/demo_test.rb
+    echo 'class DemoTest < Test::Unit::TestCase; def test_first; end; end' >> test/demo_test.rb
+
+    testbot --test --connect localhost
+        
+    # Cleanup
+    testbot --server stop
+    testbot --runner stop
+    cd ..; rm -rf testbotdemo
+    rm -rf /tmp/testbot
+
+The project files from the demo project are synced to /tmp/testbot/$USER (default). The runner syncs the files to /tmp/testbot/project (default). The tests are then run and the results returned through the server and displayed.
+
+How it works
+----
+
+Testbot is:
+
+* A **server** to distribute test jobs.
+* One or more **runners** to run test jobs and return the results (this is the "worker" process).
+* One or more **requesters** that tells the server which tests to distribute and displays the results (the client you use to run tests, for example: **rake testbot:spec**).
+
+<pre>
+    Requester -- (files to run) --> Server -- (files to run) --> (many-)Runner(s)
+        ^                           |    ^                                  |
+        |---------------------------|    |----------------------------------|
+                 (results)                            (results)
+</pre>
+
+Example setup
+----
+
+Here I make the assumption that you have a user called **testbot** on a server at **192.168.0.100** that every computer [can log into without a password](http://github.com/joakimk/testbot/wiki/SSH-Public-Key-Authentication) and that you have **installed testbot** on each computer.
+
+    ssh testbot@192.168.0.100
+    testbot --server
+    
+On every computer that should share CPU resources run:
+
+    testbot --runner --connect 192.168.0.100
+
+Running tests:
+    
+    testbot --test --connect 192.168.0.100
+    # --test could also be --spec (RSpec), --rspec (RSpec 2) or --features
+
+Using testbot with Rails 2:
+
+    # Add testbot to your Gemfile if you use bundler. You also need the plugin because
+    # Rails 2 does not load raketasks from gems.
+    ruby script/plugin install git://github.com/joakimk/testbot.git -r 'refs/tags/v0.7.8'
+    script/generate testbot --connect 192.168.0.100
+
+    rake testbot:spec (or :rspec, :test, :features)
+
+Using testbot with Rails 3:
+
+    rails g testbot --connect 192.168.0.100
+    rake testbot:spec (or :rspec, :test, :features)
+
+    # Gemfile:
+    gem 'testbot'
+
+You can keep track of the testbots on:
+
+    http://192.168.0.100:2288/status
+
+Updating testbot
+----
+
+To simplify updates there is a **--auto_update** option for the runner. The runner processes that use this option will be automatically updated and restarted when you change the server version.
+
+This requires testbot to be installed **without sudo** as the update simply runs "gem install testbot -v new_version". I recommend using [RVM](http://rvm.beginrescueend.com/) (it handles paths correctly).
+
+Example:
+    testbot --runner --connect 192.168.0.100 --auto_update
+
+More options
+----
+
+    testbot (or testbot --help)
+
+Could this readme be better somehow?
+----
+
+If there is anything missing or unclear you can create an [issue](http://github.com/joakimk/testbot/issues) (or send me a pull request).
+
+Features
+----
+* You can add and remove computers at any time. Testbot simply gives abandoned jobs to other computers.
+* Testbot will try to balance the testload so that every computer finishes running the tests at the same time to reduce the time it takes to run the entire test suite. It does a good job, but has potential for further improvement.
+* You can access your testbot network through SSH by using the built in SSH tunneling code.
+* You can use the same testbot network with multiple projects.
+* You can abort a test run with ctrl+c and all remote processes will be stopped.
+* It shows you the output as it happens.
+
+Contributing to testbot
+----
+
+First, get the tests to run:
+
+    bundle
+    rake
+
+For development I recommend using guard.
+
+    # OSX needs: gem install rb-fsevent
+    guard
+
+Make your change (don't forget to write tests) and send me a pull request.
+
+You can also contribute by adding to the [wiki](http://github.com/joakimk/testbot/wiki).
+
+How to add support for more test frameworks and/or programming languages
+----
+
+Add a **lib/shared/adapters/framework_name_adapter.rb** file and update this readme.
+
+More
+----
+
+* Check the [wiki](http://github.com/joakimk/testbot/wiki) for more info.

data/Rakefile

@@ -0,0 +1,35 @@
+require 'bundler'
+
+Bundler::GemHelper.install_tasks
+
+task :default => [ :test ] do
+end
+
+desc "Run Test::Unit tests"
+task :test do
+  Dir["test/**/*_test.rb"].each { |test| require(File.expand_path(test)) }
+end
+
+
+desc "Used for quickly deploying and testing updates without pusing to rubygems.org"
+task :deploy do
+  File.open("DEV_VERSION", "w") { |f| f.write(".DEV.#{Time.now.to_i}") }
+  
+  gem_file = "testbot-#{Testbot.version}.gem"
+  config = YAML.load_file(".deploy_config.yml")
+  Rake::Task["build"].invoke
+  
+  begin
+    system(config["upload_gem"].gsub(/GEM_FILE/, gem_file)) || fail
+    system(config["update_server"].gsub(/GEM_FILE/, gem_file)) || fail
+    system(config["restart_server"]) || fail
+  ensure
+    system("rm DEV_VERSION")
+  end
+end
+
+desc "Used to restart the server when developing testbot"
+task :restart do
+  config = YAML.load_file(".deploy_config.yml")
+  system(config["restart_server"]) || fail
+end

data/bin/testbot

@@ -0,0 +1,59 @@
+#!/usr/bin/env ruby
+
+require File.expand_path(File.join(File.dirname(__FILE__), '../lib/shared/testbot.rb'))
+
+def show_help
+  puts "Testbot #{Testbot.version}"
+  puts
+  puts "Testbot is a test distribution tool that works with Rails, RSpec, Test::Unit and Cucumber."
+  puts
+  puts "More info: http://github.com/joakimk/testbot"
+  puts "Wiki:      http://github.com/joakimk/testbot/wiki"
+  puts
+  puts "Examples:"
+  puts "  testbot --server"
+  puts "  testbot --runner --connect 192.168.0.100"
+  puts "  testbot --test --connect 192.168.0.100"
+  puts 
+  puts "Types:"
+  puts "  --server <start|stop|run>"
+  puts "  --runner <start|stop|run>"
+  Adapter.all.each do |adapter|
+    puts "  --#{adapter.type}\t\t\t# Run #{adapter.name} tests"
+  end
+  puts
+  puts "Runner options:"
+  puts "  --connect <host_or_ip>\t# Which server to use (required)"
+  puts "  --working_dir <path>\t\t# Where to store temporary files (default: #{Testbot::DEFAULT_WORKING_DIR})"
+  puts "  --cpus <number>\t\t# The number of CPU cores to use (default: use all)"
+  puts "  --ssh_tunnel\t\t\t# Use a ssh tunnel"
+  puts "  --user <username>\t\t# Use a custom ssh tunnel user (default: testbot)"
+  puts "  --auto_update\t\t\t# Keep testbot updated to the same version as the server."
+  puts "  --max_jruby_instances <num>\t# To use less instances when running JRuby (as it requires more memory)"
+  puts "  --jruby_opts <-J-X...>\t# Options to JRuby."
+  puts
+  puts "Test options:"
+  puts "  --connect <host_or_ip>\t# Which server to use (required)"
+  puts "  --rsync_ignores <ignores>\t# Files to ignore when syncing (default: include all)"
+  puts "  --rsync_path <path>\t\t# Sync path on the server (default: #{Testbot::DEFAULT_SERVER_PATH})"
+  puts "  --ssh_tunnel\t\t\t# Use a ssh tunnel"
+  puts "  --user <username>\t\t# Use a custom rsync / ssh tunnel user (default: #{Testbot::DEFAULT_USER})"
+  puts "  --project <project_name>\t# Use a custom project name (default: #{Testbot::DEFAULT_PROJECT})"
+  puts
+  puts "Other:"
+  puts "  --help\t\t\t# Show help (this page)"
+  puts "  --version\t\t\t# Show the testbot version"
+
+  # puts "                                           # (when the server gem version is changed,"
+  # puts "                                           #  this runs gem install with the same version"
+  # puts "                                           #  and restarts the test runner)"
+
+
+  # TODO:
+  # puts " --use_git_ignore                          # Don't rsync files that are ignored by git"
+  # puts "  --status                                 # Show running background processes"
+  # puts "  --port <number>                          # Use a custom port"
+
+end
+
+show_help unless Testbot::CLI.run(ARGV)

data/lib/generators/testbot/templates/testbot.rake.erb

@@ -0,0 +1,35 @@
+namespace :testbot do
+  task :before_request do
+    # This is run after you start a request (ex: rake testbot:spec)
+  end
+  
+  task :before_run do
+    # This is run by the runner after files are synced but before tests are run
+
+    # Example: Setting up a test database
+    database_yml = <<-DB_CONFIG
+test:
+  adapter: mysql
+  encoding: utf8
+  database: <%= options[:project] %>_testbot_test<%%= ENV['TEST_ENV_NUMBER'] %>
+  username: root
+  password:
+  host: localhost
+DB_CONFIG
+
+    # database_file_path = "config/database.yml"
+    # File.open(database_file_path, 'w') { |f| f.write(database_yml) }
+    # 
+    # # Setup databases for all instances
+    # 0.upto(ENV['TEST_INSTANCES'].to_i - 1) do |instance|
+    #   test_env_number = (instance == 0) ? '' : instance + 1
+    #   system "mysqladmin -u root -f drop <%= options[:project] %>_testbot_test#{test_env_number}  > /dev/null 2>&1"
+    #   system "mysqladmin -u root -f create <%= options[:project] %>_testbot_test#{test_env_number} > /dev/null 2>&1"
+    #   system "export RAILS_ENV=test; export TEST_ENV_NUMBER=#{test_env_number}; rake db:test:load"
+    # end
+
+    # Example: Building gems
+    # system "rm vendor/gems/*/ext/**/*.o > /dev/null 2>&1"
+    # system "rake gems:build:force > /dev/null 2>&1"
+  end
+end

data/lib/generators/testbot/templates/testbot.yml.erb

@@ -0,0 +1,45 @@
+# You can use ERB here.
+
+# Which server to use.
+server_host: <%= options[:connect] %>
+<%- if options[:user] -%>
+server_user: <%= options[:user] %>
+<%- else -%>
+# server_user: <%= Testbot::DEFAULT_USER %>
+<%- end -%>
+
+# Project prefix. Used for prefixing project files
+# so that you can run multiple projects in the same testbot network.
+<%- if options[:project] %>
+project: <%= options[:project] %>
+<%- else -%>
+# project: <%= Testbot::DEFAULT_PROJECT %>
+<%- end -%>
+
+# RSync settings. The folder where your files are synced to
+# and then fetched from before running the tests.
+<%- if options[:rsync_path] -%>
+rsync_path: <%= options[:rsync_path] %>
+<%- else -%>
+# rsync_path: <%= Testbot::DEFAULT_SERVER_PATH %>
+<%- end -%>
+<%- if options[:rsync_ignores] -%>
+rsync_ignores: <%= options[:rsync_ignores] %>
+<%- else -%>
+# rsync_ignores:
+<%- end -%>
+
+# To tunnel traffic through SSH
+<%- if options[:ssh_tunnel] -%>
+ssh_tunnel: true
+<%- else -%>
+# ssh_tunnel: true
+<%- end -%>
+
+# Runner usage. Set to a lower percentage to not use
+# every available instance or higher to create more
+# jobs than there are instances.
+available_runner_usage: 100%
+
+# Enable more logging from the requester
+# logging: true

data/lib/generators/testbot/testbot_generator.rb

@@ -0,0 +1,19 @@
+require File.expand_path(File.dirname(__FILE__) + "/../../shared/testbot") 
+require "acts_as_rails3_generator"
+
+class TestbotGenerator < Rails::Generators::Base
+  source_root File.expand_path('../templates', __FILE__)
+
+  class_option :connect, :type => :string, :required => true, :desc => "Which server to use (required)" 
+  class_option :project, :type => :string, :default => nil, :desc => "The name of your project (default: #{Testbot::DEFAULT_PROJECT})" 
+  class_option :rsync_path, :type => :string, :default => nil, :desc => "Sync path on the server (default: #{Testbot::DEFAULT_SERVER_PATH})"
+  class_option :rsync_ignores, :type => :string, :default => nil, :desc => "Files to rsync_ignores when syncing (default: include all)"
+  class_option :ssh_tunnel, :type => :boolean, :default => nil, :desc => "Use a ssh tunnel"
+  class_option :user, :type => :string, :default => nil, :desc => "Use a custom rsync / ssh tunnel user (default: #{Testbot::DEFAULT_USER})"
+
+  def generate_config
+    template "testbot.yml.erb", "config/testbot.yml"
+    template "testbot.rake.erb", "lib/tasks/testbot.rake"
+  end
+end
+

data/lib/railtie.rb

@@ -0,0 +1,16 @@
+begin
+  require 'rails'
+  @rails_loaded = true
+rescue LoadError => ex
+  @rails_loaded = false
+end
+
+if @rails_loaded
+  module Testbot
+    class Railtie < Rails::Railtie
+      rake_tasks do
+        load File.expand_path(File.join(File.dirname(__FILE__), "tasks/testbot.rake"))
+      end
+    end
+  end
+end