lobot 2.0.0pre → 2.1.0

data/.gitignore

@@ -1,11 +1,8 @@
-*.gem
-.bundle
-Gemfile.lock
-pkg/*
-features/config/secrets.yml
-.idea
-spec/tmp
-spec/reports/
+/tmp
 .vagrant
-.kitchen
-.env
+/spec/reports
+Gemfile.lock
+/config
+/pkg
+.env*
+*.log

data/.gitmodules

@@ -1,3 +1,3 @@
 [submodule "chef/travis-cookbooks"]
 	path = chef/travis-cookbooks
-	url = https://github.com/pivotalexperimental/travis-cookbooks
+	url = https://github.com/travis-ci/travis-cookbooks.git

data/.pairs

@@ -0,0 +1,15 @@
+# This file is managed by Chef!
+# Override node.default["git_pairs_authors"] and node.default["git_pairs_domain"] in attributes/git_pairs.rb to modify.
+
+pairs:
+  dr: Doc Ritezel; doc
+  mk: Matthew Kocher; mkocher
+  km: Ken Mayer; ken
+  dv: David Varvel; dvarvel
+  am: Alexander Murmann; Alexander
+
+email:
+  prefix: pair
+  domain: pivotallabs.com
+
+global: true

data/.rvmrc

@@ -1 +1 @@
-rvm --create use ruby-1.9.3-p194@lobot
+rvm --create ruby-1.9.3-p327@lobot

data/.travis.yml

@@ -0,0 +1,5 @@
+language: ruby
+rvm:
+  - 1.9.3
+  - 1.9.2
+script: bundle exec rspec spec --tag ~slow --tag ~vagrant --tag ~osx

data/Gemfile

@@ -1,6 +1,6 @@
-source "http://rubygems.org"
+source "https://rubygems.org"
 
-gem "builder", "~> 3.0.0" # Bundler goes into an infinite loop looking for this dep.  Feel free to remove.
+#gem "builder", "~> 3.0.0" # Bundler occasionally goes into an infinite loop looking for this dep if version isn't specified.
 
 # Specify your gem's dependencies in lobot.gemspec
 gemspec

data/LICENSE.txt

@@ -1,6 +1,6 @@
 The MIT License
 
-Copyright (c) 2011 Pivotal Labs
+Copyright (c) 2011, 2012 Pivotal Labs
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

data/README.md

@@ -1,9 +1,12 @@
-Lobot: Your Chief Administrative Aide on Cloud City
-============================
+# Lobot: Your Chief Administrative Aide on Cloud City
 
 ![Lobot](http://cheffiles.pivotallabs.com/lobot/logo.png)
 
-### Easily create your CI server on EC2
+[![Code Climate](https://codeclimate.com/github/pivotal/lobot.png)](https://codeclimate.com/github/pivotal/lobot)
+[![Build Status](https://travis-ci.org/pivotal/lobot.png?branch=master)](https://travis-ci.org/pivotal/lobot)
+
+
+## Easily create your CI server on EC2
 
 Lando Calrissian relies on Lobot to keep Cloud City afloat, and now you can rely on Lobot to get your continuous integration server running in the cloud. Lobot is a gem that will help you spin-up, bootstrap, and install Jenkins CI for your Rails app on Amazon EC2.
 
@@ -13,41 +16,60 @@ Lando Calrissian relies on Lobot to keep Cloud City afloat, and now you can rely
 * The full [Travis CI](http://travis-ci.org) environment on Ubuntu 12.04
 * A Jenkins frontend for monitoring your builds
 
-After you add `gem "lobot"` to your Gemfile, all you'll need to do is run the following commands:
-
-    [lobot configure] - COMING SOON - See Setup for now.
-    lobot create
-    lobot bootstrap
-    lobot add_build BobTheBuild git@github.com:you/some_repo.git master script/ci_build.sh
-    lobot chef
+```
+Tasks:
+  lobot add_build <name> <repository> <branch> <command>  # Adds a build to Lobot
+  lobot bootstrap          # Configures Lobot's master node
+  lobot certificate        # Dump the certificate
+  lobot chef               # Uploads chef recipes and runs them
+  lobot config             # Dumps all configuration data for Lobot
+  lobot create             # Create a new Lobot server using EC2
+  lobot create_vagrant     # Creates a vagrant instance
+  lobot destroy_ec2        # Destroys all the lobot resources on EC2
+  lobot help [TASK]        # Describe available tasks or one specific task
+  lobot open               # Open a browser to Lobot
+  lobot setup              # Sets up lobot through a series of questions
+  lobot ssh                # SSH into Lobot
+  lobot trust_certificate  # Adds the current master's certificate to your OSX keychain
+```
 
 Read on for an explanation of what each one of these steps does.
 
 ## Install
 
-Add lobot to your Gemfile, in the development group:
+    gem install lobot
 
-    gem "lobot", :group => :development
+Lobot runs independently of your project and is not a dependency.
 
 ## Setup
 
-Create a config/lobot.yml file in your project:
+If this is your first time running `lobot` and you do not have configuration file, yet, run:
 
-    ---
-    server_ssh_key: ~/.ssh/id_rsa
-    github_ssh_key: ~/.ssh/id_rsa
-    aws_key: <your AWS Key>
-    aws_secret: <your AWS Secret>
-    node_attributes:
-      nginx:
-        basic_auth_user: ci
-        basic_auth_password: password
+    lobot setup
 
-See [https://aws-portal.amazon.com/gp/aws/developer/account/index.html?ie=UTF8&action=access-key](https://aws-portal.amazon.com/gp/aws/developer/account/index.html?ie=UTF8&action=access-key) to generate AWS key/secret.
+It will ask you a series of questions that will get you up and running.
 
 ## Adjust Defaults (Optional)
 
-In your config/lobot.yml, there are defaults set for values that have the recommened value. For example, the instance size used for EC2 is set to "c1.medium".
+If you don't like the default, Rails-centric, build script you can create your own:
+
+```sh
+#!/bin/bash -le
+
+source .rvmrc
+
+# install bundler if necessary
+set -e
+
+gem install bundler --no-ri --no-rdoc && bundle install
+
+# debugging info
+echo USER=$USER && ruby --version && which ruby && which bundle
+
+bundle exec rake spec
+```
+
+In your config/lobot.yml, there are defaults set for recommended values. For example, the EC2 instance size is set to "c1.medium".
 
 You can save on EC2 costs by using a tool like [projectmonitor](https://github.com/pivotal/projectmonitor) or ylastic to schedule when your instances are online.
 
@@ -65,7 +87,7 @@ You can modify the chef run list by setting the `recipes` key in config/lobot.ym
 
 Because we're using the cookbooks from Travis CI, you can look through [all the recipes Travis has available](https://github.com/travis-ci/travis-cookbooks/), and add any that you need.
 
-## Starting your lobot instance
+## Manually starting your lobot instance
 
 1. Launch an instance, allocate and associates an elastic IP and updates config/lobot.yml:
 
@@ -79,8 +101,15 @@ Because we're using the cookbooks from Travis CI, you can look through [all the
 
         lobot chef
 
-Your lobot instance should now be up and running. You will be able to access your CI server at: http://&lt;your instance address&gt;/ with the username and password you chose during configuration.
-For more information about Jenkins CI, see [http://jenkins-ci.org](http://jenkins-ci.org).
+Your lobot instance should now be up and running. You will be able to access your CI server at: http://&lt;your instance address&gt;/ with the username and password you chose during configuration. Or, if you are on a Mac, run `lobot open`. For more information about Jenkins CI, see [http://jenkins-ci.org](http://jenkins-ci.org).
+
+## Custom Chef Recipes
+
+If you need to write your own chef recipes to install your project's dependencies, you can add a cookbooks directory to
+the root of your project.  Make sure to delete the cookbook_paths section from your lobot.yml (to use the default values),
+or add ./chef/project-cookbooks to the cookbook_paths section.
+
+So, to have a bacon recipe, you should have cookbooks/pork/recipes/bacon.rb file in your repository.
 
 ## Troubleshooting
 
@@ -104,11 +133,20 @@ end
 
 ## Dependencies
 
-* fog
 * ci_reporter
-* thor
+* fog
+* godot
+* haddock
 * hashie
+* httpclient
 * net-ssh
+* thor
+
+## Forking
+
+Please be aware that Lobot uses git submodules.  In order to git source Lobot in your `Gemfile`, you will need the following line:
+
+    gem "lobot", :github => "pivotal/lobot", :submodules => true
 
 ## Testing
 

data/Rakefile

@@ -1,11 +1,16 @@
-require 'ci/reporter/rake/rspec'
+#!/usr/bin/env rake
+require 'bundler/gem_tasks'
 require 'rspec/core/rake_task'
-require 'bundler'
 
-Bundler::GemHelper.install_tasks
+file "chef/travis-cookbooks/README.md" do
+  sh "git submodule init"
+  sh "git submodule update"
+end
 
-task :default => ['ci:setup:rspec', :spec]
-
-desc "Run all specs in spec directory (excluding plugin specs)"
 RSpec::Core::RakeTask.new(:spec)
 
+desc ""
+task :spec => ["chef/travis-cookbooks/README.md"]
+
+desc "Run specs"
+task :default => :spec

data/Vagrantfile

@@ -3,13 +3,16 @@ Vagrant::Config.run do |config|
   config.vm.box_url = "http://files.vagrantup.com/#{config.vm.box}.box"
   config.vm.network :hostonly, "192.168.33.10"
 
-  config.vm.provision :shell, :inline => "sudo mkdir /etc/skel/.ssh"
+  config.vm.provision :shell, :inline => "grep -q ubuntu /etc/passwd || sudo -n useradd -m ubuntu -G sudo,admin -s /bin/bash"
+
+  config.vm.provision :shell, :inline => "test -d /home/ubuntu/.ssh || sudo -n mkdir /home/ubuntu/.ssh"
+  config.vm.provision :shell, :inline => "sudo -n chmod 0700 /home/ubuntu/.ssh"
 
   config.vm.provision :shell do |shell|
-    shell.inline = "echo $@ | sudo tee /etc/skel/.ssh/authorized_keys"
-    ssh_key = ENV['LOBOT_SSH_KEY'] || File.expand_path("~/.ssh/id_rsa.pub")
+    shell.inline = "sudo -n echo $@ > /home/ubuntu/.ssh/authorized_keys"
+    ssh_key = File.join(File.dirname(__FILE__), 'spec', 'fixtures', 'ssh_keys', 'vagrant_test_key.pub')
     shell.args = File.read(ssh_key)
   end
 
-  config.vm.provision :shell, :inline => "sudo useradd -m ubuntu -G sudo,admin -s /bin/bash"
-end
+  config.vm.provision :shell, :inline => "sudo -n chown -R ubuntu /home/ubuntu/.ssh"
+end

data/bin/lobot

@@ -1,7 +1,5 @@
 #!/usr/bin/env ruby
 $: << File.expand_path('../../lib', __FILE__)
 
-require "rubygems"
-require "lobot/cli"
-
-Lobot::CLI.start
+require "lobot"
+Lobot::CLI.start

data/chef/cookbooks/pivotal_ci/attributes/jenkins.rb

@@ -1,2 +1,3 @@
 default["jenkins"] = {}
 default["jenkins"]["home"] = "/var/lib/jenkins"
+default["jenkins"]["builds"] = []

data/chef/cookbooks/pivotal_ci/files/default/tests/minitest/default_test.rb

@@ -1,4 +1,4 @@
-describe_recipe "pivotal_ci::jenkins" do
+describe "pivotal_ci::jenkins" do
   include MiniTest::Chef::Assertions
   include MiniTest::Chef::Context
   include MiniTest::Chef::Resources
@@ -16,7 +16,45 @@ describe_recipe "pivotal_ci::jenkins" do
   end
 end
 
-describe_recipe "pivotal_ci::default" do
+describe "pivotal_ci::id_rsa" do
+  include MiniTest::Chef::Assertions
+  include MiniTest::Chef::Context
+  include MiniTest::Chef::Resources
+
+  describe ".ssh/id_rsa" do
+    it "exists" do
+      file("/var/lib/jenkins/.ssh/id_rsa").must_exist
+    end
+
+    it "is user-writable" do
+      file("/var/lib/jenkins/.ssh/id_rsa").must_have(:mode, "600")
+    end
+
+    it "is owned by jenkins" do
+      file("/var/lib/jenkins/.ssh/id_rsa").must_have(:user, "jenkins")
+    end
+  end
+
+  describe ".ssh/known_hosts" do
+    it "exists" do
+      file("/var/lib/jenkins/.ssh/known_hosts").must_exist
+    end
+
+    it "is user-writable" do
+      file("/var/lib/jenkins/.ssh/known_hosts").must_have(:mode, "600")
+    end
+
+    it "is owned by jenkins" do
+      file("/var/lib/jenkins/.ssh/known_hosts").must_have(:user, "jenkins")
+    end
+  end
+end
+
+describe "pivotal_ci::default" do
+  include MiniTest::Chef::Assertions
+  include MiniTest::Chef::Context
+  include MiniTest::Chef::Resources
+
   def wait_for(match, options="")
     Timeout.timeout(10) do
       sleep 1 until `curl #{options} -ks https://localhost/` =~ match

data/chef/cookbooks/pivotal_ci/recipes/id_rsa.rb

@@ -20,6 +20,11 @@ file "#{node["jenkins"]["home"]}/.ssh/id_rsa" do
   owner "jenkins"
 end
 
+file "#{node["jenkins"]["home"]}/.ssh/known_hosts" do
+  mode 0600
+  owner "jenkins"
+end
+
 execute "add github to known hosts if necessary" do
   github_string = 'github.com,207.97.227.239 ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ=='
   command "echo '#{github_string}' >> #{node["jenkins"]["home"]}/.ssh/known_hosts"

data/chef/cookbooks/pivotal_ci/recipes/jenkins_config.rb

@@ -15,7 +15,7 @@ execute "download lobot plugin" do
   jenkins_plugin.call(self, "lobot", "http://cheffiles.pivotallabs.com/lobot/lobot.hpi")
 end
 
-['git', 'ansicolor', 'xvfb'].each do |plugin|
+['git-client', 'git', 'ansicolor', 'xvfb', 'greenballs'].each do |plugin|
   execute "download #{plugin} plugin" do
     jenkins_plugin.call(self, plugin, "http://mirrors.jenkins-ci.org/plugins/#{plugin}/latest/#{plugin}.hpi")
   end
@@ -32,14 +32,17 @@ node["jenkins"]["builds"].each do |build|
     owner "jenkins"
   end
 
+  require "cgi"
   template "#{node["jenkins"]["home"]}/jobs/#{build['name']}/config.xml" do
     source "jenkins-job-config.xml.erb"
     owner "jenkins"
     notifies :restart, "service[jenkins]"
+    junit_publisher = build['junit_publisher'].nil? ? true : build['junit_publisher']
     variables(
       :branch => build['branch'],
       :command => build['command'],
-      :repository => build['repository']
+      :repository => build['repository'],
+      :junit_publisher => junit_publisher
     )
   end
 end

data/chef/cookbooks/pivotal_ci/recipes/limited_travis_ci_environment.rb

@@ -33,3 +33,7 @@ include_recipe "xserver"
 include_recipe "firefox"
 include_recipe "chromium"
 include_recipe "phantomjs::tarball"
+
+# Packages for xserver
+#
+package "libicu48"

data/chef/cookbooks/pivotal_ci/templates/default/jenkins-job-config.xml.erb

@@ -51,15 +51,17 @@
   <concurrentBuild>false</concurrentBuild>
   <builders>
     <hudson.tasks.Shell>
-      <command><%= @command %></command>
+      <command><%= CGI.escapeHTML @command %></command>
     </hudson.tasks.Shell>
   </builders>
   <publishers>
-    <hudson.tasks.junit.JUnitResultArchiver>
-      <testResults>spec/reports/*.xml</testResults>
-      <keepLongStdio>false</keepLongStdio>
-      <testDataPublishers/>
-    </hudson.tasks.junit.JUnitResultArchiver>
+    <% if @junit_publisher %>
+        <hudson.tasks.junit.JUnitResultArchiver>
+          <testResults>spec/reports/*.xml</testResults>
+          <keepLongStdio>false</keepLongStdio>
+          <testDataPublishers/>
+        </hudson.tasks.junit.JUnitResultArchiver>
+    <% end %>
   </publishers>
   <buildWrappers>
     <hudson.plugins.ansicolor.AnsiColorBuildWrapper>

data/chef/cookbooks/pivotal_ci/test/kitchen/cookbooks/pivotal_ci_test/attributes/default.rb

@@ -1,3 +1,2 @@
 node['nginx']['basic_auth_user'] = "wheelbarrow"
 node['nginx']['basic_auth_password'] = "full-of-garbage"
-node['jenkins']['builds'] = []

data/chef/cookbooks/pivotal_ci/test/kitchen/cookbooks/pivotal_ci_test/recipes/bootstrap.rb

@@ -0,0 +1,41 @@
+bash "bootstrap" do
+  cwd "/home/vagrant"
+  code <<-BOOTSTRAP
+#!/bin/bash
+
+set -e
+
+sudo apt-get update > /dev/null
+
+packages="git build-essential openssl libreadline6 libreadline6-dev libreadline5 curl git-core zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt1-dev autoconf libc6-dev libncurses5-dev automake libtool bison subversion pkg-config"
+selections=`dpkg --get-selections`
+
+for package in $packages
+do
+  if ! echo "$selections" | grep "^$package\s" > /dev/null
+  then
+    to_install="$to_install $package"
+  fi
+done
+
+if [ ! -z "$to_install" ]
+then
+  sudo apt-get install -y $to_install
+fi
+
+test -d /usr/local/rvm || curl -L https://get.rvm.io | sudo bash -s stable
+
+sudo tee /etc/profile.d/rvm.sh > /dev/null <<RVMSH_CONTENT
+[[ -s "/usr/local/rvm/scripts/rvm" ]] && source "/usr/local/rvm/scripts/rvm"
+RVMSH_CONTENT
+
+sudo tee /etc/rvmrc > /dev/null <<RVMRC_CONTENTS
+rvm_install_on_use_flag=1
+rvm_trust_rvmrcs_flag=1
+rvm_gemset_create_on_use_flag=1
+RVMRC_CONTENTS
+
+sudo usermod vagrant -a -G rvm
+
+BOOTSTRAP
+end