engineyard-hudson 0.1.0 → 0.2.0

data/Gemfile.lock

@@ -1,7 +1,8 @@
 PATH
   remote: .
   specs:
-    engineyard-hudson (0.0.1)
+    engineyard-hudson (0.2.0)
+      engineyard (~> 1.3.3)
       thor (~> 0.14.3)
 
 GEM
@@ -16,11 +17,27 @@ GEM
       json (~> 1.4.6)
       term-ansicolor (~> 1.0.5)
     diff-lcs (1.1.2)
+    engineyard (1.3.3)
+      engineyard-serverside-adapter (= 1.3.5)
+      escape (~> 0.0.4)
+      highline (~> 1.5.2)
+      json_pure
+      rest-client (~> 1.4)
+      thor (~> 0.14.3)
+    engineyard-serverside-adapter (1.3.5)
+      escape (~> 0.0.4)
+      json_pure
+    escape (0.0.4)
     gherkin (2.2.9)
       json (~> 1.4.6)
       term-ansicolor (~> 1.0.5)
+    highline (1.5.2)
     json (1.4.6)
+    json_pure (1.4.6)
+    mime-types (1.16)
     rake (0.8.7)
+    rest-client (1.6.1)
+      mime-types (>= 1.16)
     rspec (2.0.1)
       rspec-core (~> 2.0.1)
       rspec-expectations (~> 2.0.1)
@@ -40,6 +57,7 @@ PLATFORMS
 DEPENDENCIES
   awesome_print
   cucumber (~> 0.9.3)
+  engineyard (~> 1.3.3)
   engineyard-hudson!
   json (~> 1.4.0)
   rake (~> 0.8.7)

data/History.md

@@ -1,5 +1,9 @@
 # History
 
+## 0.2.0 - 2010-10-30
+
+* Initial 'server' task implementation
+
 ## 0.1.0 - 2010-10-30
 
 * Initial 'ey-hudson install .' command

data/README.md

@@ -10,29 +10,57 @@ It's a nightmare. It was for me. [Hudson CI](http://hudson-ci.org/), `engineyard
 
     gem install engineyard-hudson
 
+You might also like the `hudson` CLI to play with your Hudson CI from the command line:
+
+    gem install hudson
+
 ## Assumptions
 
-It is assumed you are familiar with the `engineyard` CLI gem.
+It is assumed you are familiar with the [engineyard](http://github.com/engineyard/engineyard) CLI gem.
 
 You **do not** need to be familiar with custom chef recipes. Just follow the simple commands. Easy peasy.
 
-## Warning
+## Warning (aka TODO list)
+
+In the very first release of `engineyard-hudson`:
+
+* there is no support for authentication/authorization of Hudson CI.
+* git URLs are converted to public `git://` urls on Hudson; until deploy key support is added
+* no mail server configured for Hudson CI build failure notifications
+
+That is, its really only useful - at this very "alpha" instant in time - to Open Source Rails projects. But that's just me being brutally honest.
+
+## Hosting Hudson CI
 
-In the very first release of `engineyard-hudson`, there is no support for authentication/authorization of Hudson CI.
+Hosting Hudson CI on Engine Yard AppCloud is optional; yet delightfully simple. Hudson CI can be hosted anywhere.
 
-## Hosting Hudson CI on Engine Yard AppCloud
+### Hosting on Engine Yard AppCloud
 
-Instructions *coming soon*.
+Just a few steps and you will have your own Hudson CI:
+
+    $ mkdir hudson_server
+    $ cd hudson_server
+    $ ey-hudson server . --plugins 'googleanalytics,chucknorris'
+    
+    Finally:
+    * edit cookbooks/hudson_master/attributes/default.rb as necessary.
+    * run: ey recipes upload # use --environment(-e) & --account(-c)
+    * run: ey recipes apply  #   to select environment
+    * Boot your environment if not already booted.
 
-Hosting Hudson CI on Engine Yard AppCloud is optional. It can be hosted anywhere. You need the following information about your Hudson CI:
+Do those steps and you're done! Now, you either visit your Hudson CI site or use `hudson list` to see the status of your projects being tested.
 
-* Hudson CI host (& port)
+### Hosting elsewhere
+
+You need the following information about your Hudson CI:
+
+* Hudson CI public host (& port)
 * Hudson CI's user's public key (probably at /home/hudson/.ssh/id_rsa.pub)
 * Hudson CI's user's private key path (probably /home/hudson/.ssh/id_rsa)
 
 ## Running your tests in Hudson against Engine Yard AppCloud
 
-After a couple manual steps, you will have your applications tests running.
+Just a few steps and you will have your applications' tests running.
 
     $ cd /my/project
     $ ey-hudson install .
@@ -43,7 +71,7 @@ After a couple manual steps, you will have your applications tests running.
     * run: ey recipes apply  #   to select environment
     * Boot your environment if not already booted.
 
-Done! Either visit your Hudson CI site or use `hudson list` to see the status of your projects being tested.
+Do those steps and you're done! Now, you either visit your Hudson CI site or use `hudson list` to see the status of your projects being tested.
 
 ## Automatically triggering job builds
 
@@ -59,6 +87,10 @@ It is often desirable to have your SCM trigger Hudson CI to run your job build w
 * Enter the URL `http://HUDSON-CI-URL/job/APP-NAME/build`
 * Click "Update Settings"
 
+And here's a picture.
+
+<img src="http://img.skitch.com/20101031-d5wrc7hysrahihqr9k53xgxi1t.png" style="width: 100%;">
+
 You can also use the "Test Hook" link to test this is wired up correctly.
 
 ### CLI
@@ -72,3 +104,31 @@ Using the `hudson` CLI:
 You are triggering the build via a GET call to an URL endpoint. So you can also use `curl`:
 
     curl http://HUDSON-CI-URL/job/APP-NAME/build
+
+## Contributions
+
+* Dr Nic Williams
+* Bo Jeanes - initial chef recipes for Hudson server + slave
+
+## License
+
+Copyright (c) 2010 Dr Nic Williams, Engine Yard
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/engineyard-hudson.gemspec

@@ -2,7 +2,7 @@
 
 Gem::Specification.new do |s|
   s.name        = "engineyard-hudson"
-  s.version     = '0.1.0'
+  s.version     = '0.2.0'
   s.platform    = Gem::Platform::RUBY
   s.authors     = ["Dr Nic Williams"]
   s.email       = ["drnicwilliams@gmail.com"]

data/features/server.feature

@@ -1,12 +1,40 @@
-@wip
 Feature: Managing ey hudson server
   I want to setup and manage a Hudson CI server hosted on Engine Yard AppCloud
   
   Scenario: Setup new Hudson CI server on AppCloud
     Given I have an environment "hudson" on account "drnic" on AppCloud
-    And I am in the "rails" project folder
-    When I run local executable "ey-hudson" with arguments "server -e hudson -c drnic"
-    Then I should see "Hudson CI server recipe installed."
+    When I run local executable "ey-hudson" with arguments "server ."
+    Then file "cookbooks/main/recipes/default.rb" is created
+    And file "cookbooks/hudson_master/recipes/default.rb" is created
+    And file "cookbooks/hudson_master/attributes/default.rb" contains ":plugins => %w[git github rake ruby greenballs]"
+    And I should see exactly
+      """
+            create  cookbooks
+            create  cookbooks/hudson_master/recipes/default.rb
+            create  cookbooks/hudson_master/templates/default/init.sh.erb
+            create  cookbooks/hudson_master/templates/default/proxy.conf.erb
+            create  cookbooks/main/attributes/recipe.rb
+            create  cookbooks/main/definitions/ey_cloud_report.rb
+            create  cookbooks/main/libraries/ruby_block.rb
+            create  cookbooks/main/libraries/run_for_app.rb
+            create  cookbooks/main/recipes/default.rb
+            create  cookbooks/hudson_master/attributes/default.rb
+      
+      Finally:
+      * edit cookbooks/hudson_master/attributes/default.rb as necessary.
+      * run: ey recipes upload # use --environment(-e) & --account(-c)
+      * run: ey recipes apply  #   to select environment
+      * Boot your environment if not already booted.
+      When the recipe completes, your solo instance will host a Hudson CI!
+      """
+  
+  Scenario: Setup Hudson CI server with additional Hudson plugins
+    Given I have an environment "hudson" on account "drnic" on AppCloud
+    When I run local executable "ey-hudson" with arguments "server . -p ' chucknorris , googleanalytics '"
+    Then file "cookbooks/main/recipes/default.rb" is created
+    And file "cookbooks/hudson_master/recipes/default.rb" is created
+    And file "cookbooks/hudson_master/attributes/default.rb" contains ":plugins => %w[git github rake ruby greenballs chucknorris googleanalytics]"
+  
   
   
   

data/features/step_definitions/common_steps.rb

@@ -85,6 +85,14 @@ Then /^file "(.*)" contents (does|does not) match \/(.*)\// do |file, does, rege
   end
 end
 
+Then /^file "([^"]*)" contains "([^"]*)"$/ do |file, text|
+  in_project_folder do
+    actual_output = File.read(file)
+    actual_output.should contain(text)
+  end
+end
+
+
 Then /gem file "(.*)" and generated file "(.*)" should be the same/ do |gem_file, project_file|
   File.exists?(gem_file).should be_true
   File.exists?(project_file).should be_true

data/lib/engineyard-hudson/cli.rb

@@ -1,4 +1,5 @@
 require 'thor'
+require 'engineyard-hudson/thor-ext/actions/directory'
 
 module Engineyard
   module Hudson
@@ -17,9 +18,10 @@ module Engineyard
         Engineyard::Hudson::Install.start(ARGV[1..-1])
       end
       
-      desc "server", "Setup a Hudson CI server on AppCloud."
-      def server
-        shell.say "Coming soon!", :green
+      desc "server PROJECT_PATH", "Setup a Hudson CI server on AppCloud."
+      def server(project_path)
+        require 'engineyard-hudson/cli/server'
+        Engineyard::Hudson::Server.start(ARGV[1..-1])
       end
       
       desc "version", "show version information"

data/lib/engineyard-hudson/cli/install.rb

@@ -19,7 +19,7 @@ module Engineyard
       end
       
       def attributes
-        template "attributes.rb.erb", "cookbooks/hudson_slave/attributes/default.rb"
+        template "attributes.rb.tt", "cookbooks/hudson_slave/attributes/default.rb"
       end
       
       def recipe
@@ -36,13 +36,6 @@ module Engineyard
         end
       end
       
-      # README:
-      #   Finally:
-      #   * edit cookbooks/hudson_slave/attributes/default.rb as necessary
-      #   * run: ey recipes upload
-      #   * run: ey recipes apply
-      #   * Boot your environment if not already booted.
-      #   When the recipe completes, your project will commence its first build on Hudson CI.
       def readme
         say ""
         say "Finally:"

data/lib/engineyard-hudson/cli/server.rb

@@ -0,0 +1,41 @@
+require 'thor/group'
+
+module Engineyard
+  module Hudson
+    class Server < Thor::Group
+      include Thor::Actions
+      
+      argument :project_path
+      
+      class_option :plugins, :aliases => '-p', :desc => 'additional Hudson CI plugins (comma separated)'
+      
+      def self.source_root
+        File.join(File.dirname(__FILE__), "server", "templates")
+      end
+      
+      def cookbooks
+        directory "cookbooks"
+      end
+      
+      def attributes
+        @plugins = %w[git github rake ruby greenballs] + (options[:plugins] || '').strip.split(/\s*,\s*/)
+        template "attributes.rb.tt", "cookbooks/hudson_master/attributes/default.rb"
+      end
+      
+      def readme
+        say ""
+        say "Finally:"
+        say "* edit "; say "cookbooks/hudson_master/attributes/default.rb ", :yellow; say "as necessary."
+        say "* run: "; say "ey recipes upload ", :green; say "# use --environment(-e) & --account(-c)"
+        say "* run: "; say "ey recipes apply  ", :green; say "#   to select environment"
+        say "* "; say "Boot your environment ", :yellow; say "if not already booted."
+        say "When the recipe completes, your solo instance will host a Hudson CI!"
+      end
+
+      private
+      def say(msg, color = nil)
+        color ? shell.say(msg, color) : shell.say(msg)
+      end
+    end
+  end
+end

data/lib/engineyard-hudson/cli/server/templates/attributes.rb.tt

@@ -0,0 +1,3 @@
+hudson_master({
+  :plugins => %w[<%= @plugins.join(" ") %>]
+})

data/lib/engineyard-hudson/cli/server/templates/cookbooks/hudson_master/recipes/default.rb

@@ -0,0 +1,97 @@
+#
+# Cookbook Name:: hudson
+# Recipe:: default
+#
+
+# Using manual hudson for now not hudson gem. No ebuild seems to exist.
+# Based on http://bit.ly/9Y852l
+
+# You can use this in combination with http://github.com/bjeanes/ey_hudson_proxy
+# to serve hudson publicly on a Hudson-only EY instance. This is so you don't have to
+# find a simple app to run on the instance in lieu of an actual staging/production site.
+# Alternatively, set up nginx asa reverse proxy manually.
+
+# We'll assume running hudson under the default username
+hudson_user = node[:users].first[:username]
+hudson_port = 8082 # change this in your proxy if modified
+hudson_home = "/data/hudson-ci"
+hudson_pid  = "#{hudson_home}/tmp/pid"
+plugins     = node[:hudson_master][:plugins]
+
+if ['solo'].include?(node[:instance_role])
+  gem_package "bundler" do
+    source "http://gemcutter.org"
+    action :install
+  end
+
+  execute "setup-git-config-for-tagging" do
+    command %Q{ sudo su #{hudson_user} -c "git config --global user.email 'you@example.com' && git config --global user.name 'You are Special'" }
+    not_if  %Q{ sudo su #{hudson_user} -c "git config user.email" }
+  end
+
+  %w[logs tmp war plugins .].each do |dir|
+    directory "#{hudson_home}/#{dir}" do
+      owner hudson_user
+      group hudson_user
+      mode  0755 unless dir == "war"
+      action :create
+      recursive true
+    end
+  end
+
+  remote_file "#{hudson_home}/hudson.war" do
+    source "http://hudson-ci.org/latest/hudson.war"
+    owner hudson_user
+    group hudson_user
+    not_if { FileTest.exists?("#{hudson_home}/hudson.war") }
+  end
+
+  template "/etc/init.d/hudson" do
+    source "init.sh.erb"
+    owner "root"
+    group "root"
+    mode 0755
+    variables(
+      :user => hudson_user,
+      :port => hudson_port,
+      :home => hudson_home,
+      :pid  => hudson_pid
+    )
+    not_if { FileTest.exists?("/etc/init.d/hudson") }
+  end
+
+  plugins.each do |plugin|
+    remote_file "#{hudson_home}/plugins/#{plugin}.hpi" do
+      source "http://hudson-ci.org/latest/#{plugin}.hpi"
+      owner hudson_user
+      group hudson_user
+      not_if { FileTest.exists?("#{hudson_home}/plugins/#{plugin}.hpi") }
+    end
+
+  end
+
+  template "/data/nginx/servers/hudson_reverse_proxy.conf" do
+    source "proxy.conf.erb"
+    owner hudson_user
+    group hudson_user
+    mode 0644
+    variables(
+      :port => hudson_port
+    )
+    not_if { FileTest.exists?("/data/nginx/servers/hudson_reverse_proxy.conf") }
+  end
+
+  execute "ensure-hudson-is-running" do
+    command "/etc/init.d/hudson restart"
+  end
+
+  execute "Restart nginx" do
+    command "/etc/init.d/nginx restart"
+  end
+
+  execute "Generate key pair for slaves" do
+    key_path = "/home/#{node[:users].first[:username]}/.ssh/id_rsa"
+    command "ssh-keygen -f #{key_path} -N ''"
+    not_if { FileTest.exists?(key_path) }
+  end
+end

data/lib/engineyard-hudson/cli/server/templates/cookbooks/hudson_master/templates/default/init.sh.erb

@@ -0,0 +1,24 @@
+#!/sbin/runscript
+
+# Start/stop script for hudson
+
+PIDFILE=<%= @pid %>
+HUDSON_HOME=<%= @home %>
+COMMAND=/usr/bin/java
+ARGUMENTS="-Djava.io.tmpdir=${HUDSON_HOME}/tmp -jar ${HUDSON_HOME}/hudson.war --webroot=${HUDSON_HOME}/war --httpPort=<%= @port %> ajp13ListenAddress=127.0.0.1"
+RUN_AS=<%= @user %>
+NAME=Hudson-CI
+
+start() {
+  ebegin "Starting $NAME"
+  export HUDSON_HOME=${HUDSON_HOME}
+  start-stop-daemon -b --start --make-pidfile --pidfile $PIDFILE --chuid $RUN_AS --exec $COMMAND -- $ARGUMENTS &>$HUDSON_HOME/logs/hudson_log
+  eend $?
+}
+
+stop() {
+  ebegin "Stopping $NAME"
+  start-stop-daemon --stop --quiet --pidfile $PIDFILE
+  rm -f $PIDFILE
+  eend $?
+}

data/lib/engineyard-hudson/cli/server/templates/cookbooks/hudson_master/templates/default/proxy.conf.erb

@@ -0,0 +1,20 @@
+# Nginx config for Hudson CI behind a virtual host
+# TODO: Add SSL support as per http://gist.github.com/460906
+
+upstream hudson {
+  server localhost:<%= @port %>;
+}
+
+server {
+  listen 80 default;
+  server_name hudson.example.com;
+
+  location / {
+    proxy_set_header X-Real-IP $remote_addr;
+    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+    proxy_set_header X-Forwarded-Proto https;
+    proxy_set_header Host $http_host;
+    proxy_next_upstream error;
+    proxy_pass http://hudson;
+  }
+}

data/lib/engineyard-hudson/cli/server/templates/cookbooks/main/attributes/recipe.rb

@@ -0,0 +1,3 @@
+recipes('main')
+owner_name(@attribute[:users].first[:username])
+owner_pass(@attribute[:users].first[:password])

data/lib/engineyard-hudson/cli/server/templates/cookbooks/main/definitions/ey_cloud_report.rb

@@ -0,0 +1,6 @@
+define :ey_cloud_report do
+  execute "reporting for #{params[:name]}" do
+    command "ey-enzyme --report '#{params[:message]}'"
+    epic_fail true
+  end
+end

data/lib/engineyard-hudson/cli/server/templates/cookbooks/main/libraries/ruby_block.rb

@@ -0,0 +1,40 @@
+
+class Chef
+  class Resource
+    class RubyBlock < Chef::Resource
+      def initialize(name, collection=nil, node=nil)
+        super(name, collection, node)
+        @resource_name = :ruby_block
+        @action = :create
+        @allowed_actions.push(:create)
+      end
+ 
+      def block(&block)
+        if block
+          @block = block
+        else
+          @block
+        end
+      end
+    end
+  end
+end
+
+
+class Chef
+  class Provider
+    class RubyBlock < Chef::Provider
+      def load_current_resource
+        Chef::Log.debug(@new_resource.inspect)
+        true
+      end
+ 
+      def action_create
+        @new_resource.block.call
+      end
+    end
+  end
+end
+
+Chef::Platform.platforms[:default].merge! :ruby_block => Chef::Provider::RubyBlock
+

data/lib/engineyard-hudson/cli/server/templates/cookbooks/main/libraries/run_for_app.rb

@@ -0,0 +1,12 @@
+class Chef
+  class Recipe
+    def run_for_app(*apps, &block)
+      apps.map! {|a| a.to_s }
+      node[:applications].map{|k,v| [k,v] }.sort_by {|a,b| a }.each do |name, app_data|
+        if apps.include?(name)
+          block.call(name, app_data)
+        end
+      end
+    end
+  end
+end

data/lib/engineyard-hudson/cli/server/templates/cookbooks/main/recipes/default.rb

@@ -0,0 +1 @@
+require_recipe 'hudson_master'

data/lib/engineyard-hudson/thor-ext/actions/directory.rb

@@ -0,0 +1,33 @@
+# Extension: Sorts the Dir[lookup] to ensure deterministic ordering of actions
+# to allow test assertions
+
+class Thor
+  module Actions
+    class Directory < EmptyDirectory #:nodoc:
+      protected
+
+        def execute!
+          lookup = config[:recursive] ? File.join(source, '**') : source
+          lookup = File.join(lookup, '{*,.[a-z]*}')
+
+          Dir[lookup].sort.each do |file_source|
+            next if File.directory?(file_source)
+            file_destination = File.join(given_destination, file_source.gsub(source, '.'))
+            file_destination.gsub!('/./', '/')
+
+            case file_source
+              when /\.empty_directory$/
+                dirname = File.dirname(file_destination).gsub(/\/\.$/, '')
+                next if dirname == given_destination
+                base.empty_directory(dirname, config)
+              when /\.tt$/
+                destination = base.template(file_source, file_destination[0..-4], config, &@block)
+              else
+                destination = base.copy_file(file_source, file_destination, config, &@block)
+            end
+          end
+        end
+
+    end
+  end
+end

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification 
 name: engineyard-hudson
 version: !ruby/object:Gem::Version 
-  hash: 27
+  hash: 23
   prerelease: false
   segments: 
   - 0
-  - 1
+  - 2
   - 0
-  version: 0.1.0
+  version: 0.2.0
 platform: ruby
 authors: 
 - Dr Nic Williams
@@ -162,12 +162,23 @@ files:
 - lib/engineyard-hudson.rb
 - lib/engineyard-hudson/cli.rb
 - lib/engineyard-hudson/cli/install.rb
-- lib/engineyard-hudson/cli/install/templates/attributes.rb.erb
+- lib/engineyard-hudson/cli/install/templates/attributes.rb.tt
 - lib/engineyard-hudson/cli/install/templates/cookbooks/main/attributes/recipe.rb
 - lib/engineyard-hudson/cli/install/templates/cookbooks/main/definitions/ey_cloud_report.rb
 - lib/engineyard-hudson/cli/install/templates/cookbooks/main/libraries/ruby_block.rb
 - lib/engineyard-hudson/cli/install/templates/cookbooks/main/libraries/run_for_app.rb
 - lib/engineyard-hudson/cli/install/templates/recipes.rb
+- lib/engineyard-hudson/cli/server.rb
+- lib/engineyard-hudson/cli/server/templates/attributes.rb.tt
+- lib/engineyard-hudson/cli/server/templates/cookbooks/hudson_master/recipes/default.rb
+- lib/engineyard-hudson/cli/server/templates/cookbooks/hudson_master/templates/default/init.sh.erb
+- lib/engineyard-hudson/cli/server/templates/cookbooks/hudson_master/templates/default/proxy.conf.erb
+- lib/engineyard-hudson/cli/server/templates/cookbooks/main/attributes/recipe.rb
+- lib/engineyard-hudson/cli/server/templates/cookbooks/main/definitions/ey_cloud_report.rb
+- lib/engineyard-hudson/cli/server/templates/cookbooks/main/libraries/ruby_block.rb
+- lib/engineyard-hudson/cli/server/templates/cookbooks/main/libraries/run_for_app.rb
+- lib/engineyard-hudson/cli/server/templates/cookbooks/main/recipes/default.rb
+- lib/engineyard-hudson/thor-ext/actions/directory.rb
 has_rdoc: true
 homepage: http://github.com/engineyard/engineyard-hudson
 licenses: []