boxgrinder-build 0.8.1 → 0.9.0

data/CHANGELOG

@@ -1,3 +1,17 @@
+v0.9.0
+
+* [BGBUILD-174] Move plugins to boxgrinder-build gem
+* [BGBUILD-169] S3 plugin temporary work-around for EL5
+* [BGBUILD-103] README to indicate supported operating systems / requirements
+* [BGBUILD-175] Rewrite boxgrinder CLI to remove thor dependency
+* [BGBUILD-81] post command execution w/ setarch breaks commands which are scripts
+* [BGBUILD-173] Include setarch package in default package list for RPM-based OSes
+* [BGBUILD-177] Fedora 13 builds have enabled firewall although they shouldn't have it
+* [BGBUILD-178] Remove sensitive data from logs
+* [BGBUILD-179] Boolean and numeric parameters in hash-like values are not recognized
+* [BGBUILD-176] Fail the build with appropriate message if any of post section commands fails to execute
+* [BGBUILD-183] Add support for Fedora 15
+
 v0.8.1
 
 * [BGBUILD-141] Long delay after "Preparing guestfs" message when creating new image

data/README.md

@@ -0,0 +1,136 @@
+# BoxGrinder
+
+BoxGrinder is a set of projects that help you grind out appliances for multiple virtualization and Cloud providers. 
+
+Visit us at [boxgrinder.org](http://www.boxgrinder.org)
+
+## BoxGrinder Build
+
+BoxGrinder Build is a command line tool to help you build appliances. With an appliance definition and just a single command, BoxGrinder can create your appliance, target it to a platform and deliver it.   
+
+### Boxgrinder Build Meta Appliance: Boxgrinder the Easy Way
+
+The Boxgrinder Meta appliance is pre-configured with an optimal environment prepared to use Boxgrinder right from launch. Just [download the latest appliance](http://boxgrinder.org/download/boxgrinder-build-meta-appliance/) from boxgrinder.org in your desired format, launch it, and you're ready to grind out images!
+
+It is a great way to use or test BoxGrinder in a virtual environment without affecting your local system.
+
+Visit the [Boxgrinder Meta appliance usage article](http://boxgrinder.org/tutorials/boxgrinder-build-meta-appliance/) to learn more.
+
+### Supported OSes
+
+At present the project officially supports the following OSes in x86_64 and i386 variants:
+
+* Fedora (13, 14)
+* RHEL (5.x, 6.x) and CentOS (5.x)
+
+### Requirements
+
+* Acquiring and installing BoxGrinder is very simple, with RPMs ensuring the correct dependencies are pulled and installed.  However, depending upon your OS of choice, the prerequisites for installing BoxGrinder vary slightly. 
+* Administrative level permissions (root or equivalent)
+
+#### Fedora
+
+BoxGrinder and all of its dependencies reside within the official Fedora repositories, therefore there are no special requirements. Simply install via your package manager.
+
+#### RHEL/CentOS
+
+EPEL and BoxGrinder repositories locations must be added to your package manager in order to install BoxGrinder and resolve its dependencies successfully.
+
+For detailed instructions, see: [Preparing your environment](http://boxgrinder.org/tutorials/boxgrinder-build-quick-start/preparing-environment/)
+
+### Installing
+
+Once the prerequisites are satisfied, install BoxGrinder via a package manager, for instance in YUM;
+
+* `sudo yum install "rubygem-boxgrinder*"` to install BoxGrinder Core, BoxGrinder Build and all standard plugins.
+
+You may only wish to install a subset of the available plugins, see the [BoxGrinder plugins list](http://boxgrinder.org/tutorials/#Plugin_list) to find out which are available.  You can, of course, [write your own](http://boxgrinder.org/tutorials/how-to-write-a-plugin-for-boxgrinder-build/) plugins!
+
+### Removing
+
+You should remove BoxGrinder through your system package manager, for instance with YUM:
+
+* `sudo yum remove "rubygem-boxgrinder*"`
+
+### Usage
+
+BoxGrinder.org's [quick-start](http://boxgrinder.org/tutorials/boxgrinder-build-quick-start/) tutorial is the best place to learn the fundamentals of BoxGrinder Build, enabling you to rapidly leverage the feature-set on offer.  
+
+The following sections provide a basic overview of functionality. 
+
+### Plugins
+
+Most of the features of BoxGrinder Build are provided through plugins, with three primary variants (Operating System, Platform and Delivery), each catering for a phase of the build process.  Furthermore, the flexible and extensible structure of BoxGrinder Build enables users to seamlessly add new features and functionality.
+
+* Operating System - provide support to run and build for a given OS
+* Platform - ability to produce appliances for a specific platform, such as Amazon's EC2 or VirtualBox VM
+* Delivery - send the completed appliance, for instance by SFTP or bundled as an AMI and uploaded to S3.
+
+Learn more: [BoxGrinder Build](http://boxgrinder.org/build/)
+
+#### Plugin Configuration
+
+Many plugins allow, or mandate, some degree of configuration before they are executed.  These properties are aggregated into a single per-user configuration file, at `~/.boxgrinder/config`.  The user should consult the documentation of a given plugin to determine what fields it should be configured with, and which fields (if any), are requisite. 
+
+    plugins:
+      sftp:
+        host: 192.168.0.1
+        username: boxgrinder
+        path: /home/boxgrinder/appliances
+
+* In this example, the SFTP plugin's fields are [all mandatory](http://boxgrinder.org/tutorials/boxgrinder-build-plugins/#SFTP_Delivery_Plugin).
+
+Learn more: [BoxGrinder Plugins](http://boxgrinder.org/tutorials/boxgrinder-build-plugins/#Plugin_configuration) 
+
+#### Defining Appliances
+
+Appliances are defined in YAML, the following "JEOS" definition can be used by BoxGrinder to produce a basic, but fully functional appliance;
+
+    name: f14-basic
+    summary: Just Enough Operating System based on Fedora 14
+    os:
+      name: fedora
+      version: 14
+    hardware:
+      partitions:
+        "/":
+          size: 2
+    packages:
+      - @core
+  
+  * With this simple definition, BoxGrinder will produce a Fedora 14 appliance with the Core [group of packages](http://yum.baseurl.org/wiki/YumGroups). You can easily add packages, repositories, and harness a plethora of powerful features in a concise, declarative manner. 
+  * For an introduction to Appliance Definitions, see: [Quick-start, building your first appliance](http://boxgrinder.org/tutorials/boxgrinder-build-quick-start/build-your-first-appliance) 
+  * For fuller explanations of all parameters BoxGrinder offers, see: [Appliance definition parameters](http://boxgrinder.org/tutorials/appliance-definition/)
+  
+#### Building Appliances
+The BoxGrinder Build CLI is a simple interface through which you can instruct BoxGrinder to build your appliances. You can view a brief manual on each task by utilising the help function: 
+
+    boxgrinder help [TASK]
+
+BoxGrinder's primary task is `build`, and is run with a mandatory appliance definition, along with optional platform and delivery plugin specifiers.  BoxGrinder resolves the packages and associated dependencies in the appliance definition, and installs them into the new image it generates based upon the operating system and versions specified.
+
+    boxgrinder build [appliance definition file] [options]
+    
+ * The two most common options are `[-p|--platform=]` and `[-d|--delivery=]`. Neither is mandatory, if platform is omitted then only the raw KVM image is created. You can can later return and target a build to different platforms, and BoxGrinder will always reuse the intermediary data where it is available. If you wish to force a rebuild, you can use the `[-f|--force]` flag.
+ * It is possible to manually provide key-value pairs for platform and delivery plugin configuration on the command line. These will override any pre-existing plugin parameters set in the BoxGrinder config file.
+ * Shell commands can be executed in `post` sections of the appliance definition files, which is useful for basic configuration. However, it is advisable that more complex configuration and installation of custom software is performed properly through RPM files. This is often best achieved through local repositories, which can be configured as [_ephemeral_](http://boxgrinder.org/tutorials/appliance-definition/) in order to avoid the repository being installed into the resultant image's package manager. 
+ 
+See: [BoxGrinder Build Usage Instructions](http://boxgrinder.org/tutorials/boxgrinder-build-usage-instructions/), [How to use local repositories](http://boxgrinder.org/tutorials/how-to-use-local-repository), and `boxgrinder help build` 
+
+##### Examples
+
+    boxgrinder build fedora-14.appl -p vmware -d sftp
+
+Build an image based upon _fedora-14.appl_, and produce an image targeted at the _vmware_ platform.  Once complete, deliver by _sftp_. Note that each of these plugins must be configured in `~/.boxgrinder/conf` or by providing the key-value pairs as command-line arguments.
+
+-----------------------
+
+    boxgrinder build fedora-14.appl -p virtualbox -d local
+
+Assuming that BoxGrinder had succeeded in building the prior image, the RAW file is again used as an intermediary without needing to be rebuild. An image targeted at VirtualBox is then produced, and delivered to a local file, as determined in _conf_.
+
+-----------------------
+
+    setarch i386 boxgrinder build fedora-14.appl
+    
+Build an i386 appliance (on an x86_64 machine).    

data/Rakefile

@@ -21,12 +21,17 @@ require 'spec/rake/spectask'
 require 'echoe'
 
 Echoe.new("boxgrinder-build") do |p|
-  p.project     = "BoxGrinder Build"
-  p.author      = "Marek Goldmann"
-  p.summary     = "A tool for creating appliances from simple plain text files for various virtual environments."
-  p.url         = "http://www.jboss.org/boxgrinder"
-  p.email       = "info@boxgrinder.org"
-  p.runtime_dependencies = ["thor >=0.13", "boxgrinder-core ~>0.2.1"]
+  p.project = "BoxGrinder Build"
+  p.author = "Marek Goldmann"
+  p.summary = "A tool for creating appliances from simple plain text files for various virtual environments."
+  p.url = "http://boxgrinder.org/"
+  p.email = "info@boxgrinder.org"
+  p.runtime_dependencies = [
+      "boxgrinder-core ~>0.3.0",
+      'aws', # S3
+      'amazon-ec2', # EBS and S3
+      'net-sftp', 'net-ssh', 'progressbar', # SFTP
+  ]
 end
 
 desc "Run all tests"

data/bin/boxgrinder-build

@@ -0,0 +1,199 @@
+#!/usr/bin/env ruby
+
+# Copyright 2010 Red Hat, Inc.
+#
+# This is free software; you can redistribute it and/or modify it
+# under the terms of the GNU Lesser General Public License as
+# published by the Free Software Foundation; either version 3 of
+# the License, or (at your option) any later version.
+#
+# This software is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this software; if not, write to the Free
+# Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
+# 02110-1301 USA, or see the FSF site: http://www.fsf.org.
+
+require 'optparse'
+require 'rubygems'
+require 'hashery/opencascade'
+require 'boxgrinder-core/models/config'
+require 'boxgrinder-core/helpers/log-helper'
+require 'boxgrinder-build/appliance'
+
+if Process.uid != 0
+  puts "This program must be executed with root privileges. Try 'sudo #{File.basename($0)}'."
+  abort
+end
+
+options = OpenCascade.new(
+    :platform => :none,
+    :delivery => :none,
+    :force => false,
+    :os_config => {},
+    :platform_config => {},
+    :delivery_config => {},
+    :additional_plugins => [],
+    :debug => false,
+    :trace => false,
+    :backtrace => false,
+    :log_level => :info
+)
+
+def validate_hash_option(options, name, value)
+  value.each do |entry|
+    if entry =~ /^(\S+):(\S+)$/
+
+      k = $1.strip
+      v = $2.strip
+
+      if v =~ /^([-+]?(0|[1-9][0-9_]*))$/
+        v = v.to_i
+      elsif v =~ /^(y|Y|yes|Yes|YES|true|True|TRUE|on|On|ON)$/
+        v = true
+      elsif v =~ /^(n|N|no|No|NO|false|False|FALSE|off|Off|OFF)$/
+        v = false
+      elsif v =~ /^([-+]?([0-9][0-9_]*)?\.[0-9.]*([eE][-+][0-9]+)?)$/
+        v = v.to_f
+      end
+
+      options[name][k] = v
+    else
+      yield entry if block_given?
+      abort
+    end
+  end
+end
+
+def process_options(options)
+  OptionParser.new do |opts|
+    program = File.basename($0)
+
+    opts.banner = <<EOB
+Usage: #{program} [appliance definition file] [options]
+
+A tool for building VM images from simple definition files.
+        
+Homepage:
+    http://boxgrinder.org/
+
+Documentation:
+    http://boxgrinder.org/tutorials/
+
+Examples:
+    $ #{program} jeos.appl                                                           # Build KVM image for jeos.appl
+    $ #{program} jeos.appl -f                                                        # Build KVM image for jeos.appl with removing previous build for this image
+    $ #{program} jeos.appl --os-config format:qcow2                                  # Build KVM image for jeos.appl with a qcow2 disk
+    $ #{program} jeos.appl -p vmware --platform-config type:personal,thin_disk:true  # Build VMware image for VMware Server, Player, Fusion using thin (growing) disk
+    $ #{program} jeos.appl -p ec2 -d ami                                             # Build and register AMI for jeos.appl
+    $ #{program} jeos.appl -p vmware -d local                                        # Build VMware image for jeos.appl and deliver it to local directory
+EOB
+
+
+    opts.separator ""
+    opts.separator "Options:"
+
+    opts.on("-p", "--platform [TYPE]", "The name of platform you want to convert to.") { |v| options.platform = v.to_sym }
+    opts.on("-d", "--delivery [METHOD]", "The delivery method for selected appliance.") { |v| options.delivery = v.to_sym }
+    opts.on("-f", "--force", "Force image creation - removes all previous builds for selected appliance. Default: false.") { |v| options.force = v }
+
+    opts.separator ""
+    opts.separator "Plugin configuration options:"
+
+    opts.on("-l", "--plugins [PLUGINS]", Array, "Comma separated list of additional plugins. Default: empty.") do |v|
+      options[:additional_plugins] = v
+    end
+
+    opts.separator ""
+
+    opts.on("--os-config [CONFIG]", Array, "Operating system plugin configuration in format: key1:value1,key2:value2.") do |v|
+      validate_hash_option(options, :os_config, v) do |entry|
+        puts "Operating system plugin configuration is not valid at '#{entry}'. Valid format is: 'key:value'."
+      end
+    end
+
+    opts.on("--platform-config [CONFIG]", Array, "Platform plugin configuration in format: key1:value1,key2:value2.") do |v|
+      validate_hash_option(options, :platform_config, v) do |entry|
+        puts "Platform plugin configuration is not valid at '#{entry}'. Valid format is: 'key:value'."
+      end
+    end
+
+    opts.on("--delivery-config [CONFIG]", Array, "Delivery plugin configuration in format: key1:value1,key2:value2.") do |v|
+      validate_hash_option(options, :delivery_config, v) do |entry|
+        puts "Delivery plugin configuration is not valid at '#{entry}'. Valid format is: 'key:value'."
+      end
+    end
+
+    opts.separator ""
+    opts.separator "Logging options:"
+
+    opts.on("--debug", "Prints debug information while building. Default: false.") { |v| options[:debug] = v }
+    opts.on("--trace", "Prints trace information while building. Default: false.") { |v| options[:trace] = v }
+    opts.on("-b", "--backtrace", "Prints full backtrace if errors occur whilst building. Default: true if console log is set to debug or trace, otherwise false.") { |v| options[:backtrace] = v }
+
+    opts.separator ""
+    opts.separator 'Common options:'
+
+    opts.on_tail('--help', 'Show this message.') do
+      puts opts
+      exit
+    end
+    opts.on_tail('--version', 'Print the version.') do
+      puts "BoxGrinder Build #{File.read("#{File.dirname(__FILE__)}/../CHANGELOG").match(/^v(.*)/)[1]}"
+
+      [:os, :platform, :delivery].each do |type|
+        puts
+        puts "Available #{type} plugins:"
+        BoxGrinder::PluginManager.instance.plugins[type].each do |name, plugin_info|
+          puts " - #{name} plugin for #{plugin_info[:full_name]}"
+        end
+      end
+
+      exit
+    end
+  end
+end
+
+opts = process_options(options)
+
+begin
+  opts.parse!
+  if ARGV.empty? or ARGV.size > 1
+    puts opts
+    puts
+    puts 'No or more than one appliance definition file specified.'
+    abort
+  end
+
+  appliance_definition_file = ARGV.first
+
+  unless File.exists?(appliance_definition_file)
+    puts "Appliance definition file '#{appliance_definition_file}' could not be found."
+    abort
+  end
+
+  options.log_level = :debug if options.debug
+  options.log_level = :trace if options.trace
+  options.backtrace = :true if [:debug, :trace].include?(options.log_level)
+
+  log = BoxGrinder::LogHelper.new(:level => options.log_level)
+
+  begin
+    BoxGrinder::Appliance.new(appliance_definition_file, BoxGrinder::Config.new(options), :log => log).create
+  rescue Exception => e
+    if options.backtrace
+      log.fatal e
+    else # demote backtrace to debug so that it is in file log only
+      log.fatal "#{e.message}. See the log file for detailed information."
+      log.debug e.backtrace.join($/)
+    end
+  end
+rescue => e
+  puts opts
+  puts
+  puts e.message
+  abort
+end

data/boxgrinder-build.gemspec

@@ -2,19 +2,19 @@
 
 Gem::Specification.new do |s|
   s.name = %q{boxgrinder-build}
-  s.version = "0.8.1"
+  s.version = "0.9.0"
 
   s.required_rubygems_version = Gem::Requirement.new(">= 1.2") if s.respond_to? :required_rubygems_version=
   s.authors = ["Marek Goldmann"]
-  s.date = %q{2011-02-22}
-  s.default_executable = %q{boxgrinder}
+  s.date = %q{2011-03-09}
+  s.default_executable = %q{boxgrinder-build}
   s.description = %q{A tool for creating appliances from simple plain text files for various virtual environments.}
   s.email = %q{info@boxgrinder.org}
-  s.executables = ["boxgrinder"]
-  s.extra_rdoc_files = ["CHANGELOG", "LICENSE", "README", "bin/boxgrinder", "lib/boxgrinder-build.rb", "lib/boxgrinder-build/appliance.rb", "lib/boxgrinder-build/helpers/appliance-customize-helper.rb", "lib/boxgrinder-build/helpers/augeas-helper.rb", "lib/boxgrinder-build/helpers/guestfs-helper.rb", "lib/boxgrinder-build/helpers/image-helper.rb", "lib/boxgrinder-build/helpers/linux-helper.rb", "lib/boxgrinder-build/helpers/package-helper.rb", "lib/boxgrinder-build/helpers/plugin-helper.rb", "lib/boxgrinder-build/helpers/thor-helper.rb", "lib/boxgrinder-build/managers/plugin-manager.rb", "lib/boxgrinder-build/plugins/base-plugin.rb"]
-  s.files = ["CHANGELOG", "LICENSE", "Manifest", "README", "Rakefile", "bin/boxgrinder", "boxgrinder-build.gemspec", "lib/boxgrinder-build.rb", "lib/boxgrinder-build/appliance.rb", "lib/boxgrinder-build/helpers/appliance-customize-helper.rb", "lib/boxgrinder-build/helpers/augeas-helper.rb", "lib/boxgrinder-build/helpers/guestfs-helper.rb", "lib/boxgrinder-build/helpers/image-helper.rb", "lib/boxgrinder-build/helpers/linux-helper.rb", "lib/boxgrinder-build/helpers/package-helper.rb", "lib/boxgrinder-build/helpers/plugin-helper.rb", "lib/boxgrinder-build/helpers/thor-helper.rb", "lib/boxgrinder-build/managers/plugin-manager.rb", "lib/boxgrinder-build/plugins/base-plugin.rb", "rubygem-boxgrinder-build.spec", "spec/Rakefile", "spec/appliance-spec.rb", "spec/helpers/appliance-customize-helper-spec.rb", "spec/helpers/augeas-helper-spec.rb", "spec/helpers/guestfs-helper-spec.rb", "spec/helpers/image-helper-spec.rb", "spec/helpers/linux-helper-spec.rb", "spec/helpers/package-helper-spec.rb", "spec/helpers/plugin-helper-spec.rb", "spec/managers/plugin-manager-spec.rb", "spec/plugins/base-plugin-spec.rb"]
-  s.homepage = %q{http://www.jboss.org/boxgrinder}
-  s.rdoc_options = ["--line-numbers", "--inline-source", "--title", "Boxgrinder-build", "--main", "README"]
+  s.executables = ["boxgrinder-build"]
+  s.extra_rdoc_files = ["CHANGELOG", "LICENSE", "README.md", "bin/boxgrinder-build", "lib/boxgrinder-build.rb", "lib/boxgrinder-build/appliance.rb", "lib/boxgrinder-build/helpers/appliance-customize-helper.rb", "lib/boxgrinder-build/helpers/augeas-helper.rb", "lib/boxgrinder-build/helpers/guestfs-helper.rb", "lib/boxgrinder-build/helpers/image-helper.rb", "lib/boxgrinder-build/helpers/linux-helper.rb", "lib/boxgrinder-build/helpers/package-helper.rb", "lib/boxgrinder-build/helpers/plugin-helper.rb", "lib/boxgrinder-build/managers/plugin-manager.rb", "lib/boxgrinder-build/plugins/base-plugin.rb", "lib/boxgrinder-build/plugins/delivery/ebs/ebs-plugin.rb", "lib/boxgrinder-build/plugins/delivery/local/local-plugin.rb", "lib/boxgrinder-build/plugins/delivery/s3/s3-plugin.rb", "lib/boxgrinder-build/plugins/delivery/s3/src/cert-ec2.pem", "lib/boxgrinder-build/plugins/delivery/sftp/sftp-plugin.rb", "lib/boxgrinder-build/plugins/delivery/usb/usb-plugin.rb", "lib/boxgrinder-build/plugins/os/centos/centos-plugin.rb", "lib/boxgrinder-build/plugins/os/fedora/fedora-plugin.rb", "lib/boxgrinder-build/plugins/os/rhel/rhel-plugin.rb", "lib/boxgrinder-build/plugins/os/rpm-based/kickstart.rb", "lib/boxgrinder-build/plugins/os/rpm-based/rpm-based-os-plugin.rb", "lib/boxgrinder-build/plugins/os/rpm-based/rpm-dependency-validator.rb", "lib/boxgrinder-build/plugins/os/rpm-based/src/appliance.ks.erb", "lib/boxgrinder-build/plugins/os/rpm-based/src/base.repo", "lib/boxgrinder-build/plugins/os/rpm-based/src/motd.init", "lib/boxgrinder-build/plugins/platform/ec2/ec2-plugin.rb", "lib/boxgrinder-build/plugins/platform/ec2/src/fstab_32bit", "lib/boxgrinder-build/plugins/platform/ec2/src/fstab_64bit", "lib/boxgrinder-build/plugins/platform/ec2/src/ifcfg-eth0", "lib/boxgrinder-build/plugins/platform/ec2/src/menu.lst", "lib/boxgrinder-build/plugins/platform/ec2/src/rc_local", "lib/boxgrinder-build/plugins/platform/virtualbox/virtualbox-plugin.rb", "lib/boxgrinder-build/plugins/platform/vmware/src/README-enterprise", "lib/boxgrinder-build/plugins/platform/vmware/src/README-personal", "lib/boxgrinder-build/plugins/platform/vmware/src/base.vmdk", "lib/boxgrinder-build/plugins/platform/vmware/src/base.vmx", "lib/boxgrinder-build/plugins/platform/vmware/vmware-plugin.rb"]
+  s.files = ["CHANGELOG", "LICENSE", "Manifest", "README.md", "Rakefile", "bin/boxgrinder-build", "boxgrinder-build.gemspec", "lib/boxgrinder-build.rb", "lib/boxgrinder-build/appliance.rb", "lib/boxgrinder-build/helpers/appliance-customize-helper.rb", "lib/boxgrinder-build/helpers/augeas-helper.rb", "lib/boxgrinder-build/helpers/guestfs-helper.rb", "lib/boxgrinder-build/helpers/image-helper.rb", "lib/boxgrinder-build/helpers/linux-helper.rb", "lib/boxgrinder-build/helpers/package-helper.rb", "lib/boxgrinder-build/helpers/plugin-helper.rb", "lib/boxgrinder-build/managers/plugin-manager.rb", "lib/boxgrinder-build/plugins/base-plugin.rb", "lib/boxgrinder-build/plugins/delivery/ebs/ebs-plugin.rb", "lib/boxgrinder-build/plugins/delivery/local/local-plugin.rb", "lib/boxgrinder-build/plugins/delivery/s3/s3-plugin.rb", "lib/boxgrinder-build/plugins/delivery/s3/src/cert-ec2.pem", "lib/boxgrinder-build/plugins/delivery/sftp/sftp-plugin.rb", "lib/boxgrinder-build/plugins/delivery/usb/usb-plugin.rb", "lib/boxgrinder-build/plugins/os/centos/centos-plugin.rb", "lib/boxgrinder-build/plugins/os/fedora/fedora-plugin.rb", "lib/boxgrinder-build/plugins/os/rhel/rhel-plugin.rb", "lib/boxgrinder-build/plugins/os/rpm-based/kickstart.rb", "lib/boxgrinder-build/plugins/os/rpm-based/rpm-based-os-plugin.rb", "lib/boxgrinder-build/plugins/os/rpm-based/rpm-dependency-validator.rb", "lib/boxgrinder-build/plugins/os/rpm-based/src/appliance.ks.erb", "lib/boxgrinder-build/plugins/os/rpm-based/src/base.repo", "lib/boxgrinder-build/plugins/os/rpm-based/src/motd.init", "lib/boxgrinder-build/plugins/platform/ec2/ec2-plugin.rb", "lib/boxgrinder-build/plugins/platform/ec2/src/fstab_32bit", "lib/boxgrinder-build/plugins/platform/ec2/src/fstab_64bit", "lib/boxgrinder-build/plugins/platform/ec2/src/ifcfg-eth0", "lib/boxgrinder-build/plugins/platform/ec2/src/menu.lst", "lib/boxgrinder-build/plugins/platform/ec2/src/rc_local", "lib/boxgrinder-build/plugins/platform/virtualbox/virtualbox-plugin.rb", "lib/boxgrinder-build/plugins/platform/vmware/src/README-enterprise", "lib/boxgrinder-build/plugins/platform/vmware/src/README-personal", "lib/boxgrinder-build/plugins/platform/vmware/src/base.vmdk", "lib/boxgrinder-build/plugins/platform/vmware/src/base.vmx", "lib/boxgrinder-build/plugins/platform/vmware/vmware-plugin.rb", "rubygem-boxgrinder-build.spec", "spec/Rakefile", "spec/appliance-spec.rb", "spec/helpers/appliance-customize-helper-spec.rb", "spec/helpers/augeas-helper-spec.rb", "spec/helpers/guestfs-helper-spec.rb", "spec/helpers/image-helper-spec.rb", "spec/helpers/linux-helper-spec.rb", "spec/helpers/package-helper-spec.rb", "spec/helpers/plugin-helper-spec.rb", "spec/managers/plugin-manager-spec.rb", "spec/plugins/base-plugin-spec.rb", "spec/plugins/delivery/ebs/ebs-plugin-spec.rb", "spec/plugins/delivery/ebs/ebs.yaml", "spec/plugins/delivery/local/local-plugin-spec.rb", "spec/plugins/delivery/s3/s3-plugin-spec.rb", "spec/plugins/delivery/sftp/sftp-plugin-spec.rb", "spec/plugins/os/centos/centos-plugin-spec.rb", "spec/plugins/os/fedora/fedora-plugin-spec.rb", "spec/plugins/os/rhel/rhel-plugin-spec.rb", "spec/plugins/os/rpm-based/kickstart-spec.rb", "spec/plugins/os/rpm-based/rpm-based-os-plugin-spec.rb", "spec/plugins/os/rpm-based/rpm-dependency-validator-spec.rb", "spec/plugins/os/rpm-based/src/jeos-f13-plain.ks", "spec/plugins/os/rpm-based/src/jeos-f13-without-version.ks", "spec/plugins/os/rpm-based/src/jeos-f13.ks", "spec/plugins/platform/ec2/ec2-plugin-spec.rb", "spec/plugins/platform/virtualbox/virtualbox-plugin-spec.rb", "spec/plugins/platform/vmware/vmware-plugin-spec.rb"]
+  s.homepage = %q{http://boxgrinder.org/}
+  s.rdoc_options = ["--line-numbers", "--inline-source", "--title", "Boxgrinder-build", "--main", "README.md"]
   s.require_paths = ["lib"]
   s.rubyforge_project = %q{BoxGrinder Build}
   s.rubygems_version = %q{1.3.7}
@@ -25,14 +25,26 @@ Gem::Specification.new do |s|
     s.specification_version = 3
 
     if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then
-      s.add_runtime_dependency(%q<thor>, [">= 0.13"])
-      s.add_runtime_dependency(%q<boxgrinder-core>, ["~> 0.2.1"])
+      s.add_runtime_dependency(%q<boxgrinder-core>, ["~> 0.3.0"])
+      s.add_runtime_dependency(%q<aws>, [">= 0"])
+      s.add_runtime_dependency(%q<amazon-ec2>, [">= 0"])
+      s.add_runtime_dependency(%q<net-sftp>, [">= 0"])
+      s.add_runtime_dependency(%q<net-ssh>, [">= 0"])
+      s.add_runtime_dependency(%q<progressbar>, [">= 0"])
     else
-      s.add_dependency(%q<thor>, [">= 0.13"])
-      s.add_dependency(%q<boxgrinder-core>, ["~> 0.2.1"])
+      s.add_dependency(%q<boxgrinder-core>, ["~> 0.3.0"])
+      s.add_dependency(%q<aws>, [">= 0"])
+      s.add_dependency(%q<amazon-ec2>, [">= 0"])
+      s.add_dependency(%q<net-sftp>, [">= 0"])
+      s.add_dependency(%q<net-ssh>, [">= 0"])
+      s.add_dependency(%q<progressbar>, [">= 0"])
     end
   else
-    s.add_dependency(%q<thor>, [">= 0.13"])
-    s.add_dependency(%q<boxgrinder-core>, ["~> 0.2.1"])
+    s.add_dependency(%q<boxgrinder-core>, ["~> 0.3.0"])
+    s.add_dependency(%q<aws>, [">= 0"])
+    s.add_dependency(%q<amazon-ec2>, [">= 0"])
+    s.add_dependency(%q<net-sftp>, [">= 0"])
+    s.add_dependency(%q<net-ssh>, [">= 0"])
+    s.add_dependency(%q<progressbar>, [">= 0"])
   end
 end

data/lib/boxgrinder-build/appliance.rb

@@ -62,11 +62,11 @@ module BoxGrinder
     def validate_definition
       ApplianceConfigValidator.new(@appliance_config).validate
 
-      raise "No operating system plugins installed. Install one or more operating system plugin. See http://community.jboss.org/docs/DOC-15081 and http://community.jboss.org/docs/DOC-15214 for more info." if PluginManager.instance.plugins[:os].empty?
+      raise "No operating system plugins installed. Install one or more operating system plugin. See http://boxgrinder.org/tutorials/boxgrinder-build-plugins/#Operating_system_plugins for more info." if PluginManager.instance.plugins[:os].empty?
 
       os_plugin = PluginManager.instance.plugins[:os][@appliance_config.os.name.to_sym]
 
-      raise "Not supported operating system selected: #{@appliance_config.os.name}. Make sure you have installed right operating system plugin, see http://community.jboss.org/docs/DOC-15214. Supported OSes are: #{PluginManager.instance.plugins[:os].keys.join(", ")}" if os_plugin.nil?
+      raise "Not supported operating system selected: #{@appliance_config.os.name}. Make sure you have installed right operating system plugin, see http://boxgrinder.org/tutorials/boxgrinder-build-plugins/#Operating_system_plugins. Supported OSes are: #{PluginManager.instance.plugins[:os].keys.join(", ")}" if os_plugin.nil?
       raise "Not supported operating system version selected: #{@appliance_config.os.version}. Supported versions are: #{os_plugin[:versions].join(", ")}" unless @appliance_config.os.version.nil? or os_plugin[:versions].include?(@appliance_config.os.version)
     end
 
@@ -84,7 +84,7 @@ module BoxGrinder
 
     def create
       @log.debug "Launching new BoxGrinder build..."
-      @log.trace "Used configuration: #{@config.to_yaml}"
+      @log.trace "Used configuration: #{@config.to_yaml.gsub(/(\S*(key|account|cert|username|host)\S*).*:(.*)/, '\1' + ": <REDACTED>")}"
 
       PluginHelper.new(@config, :log => @log).load_plugins
       read_definition
@@ -94,7 +94,7 @@ module BoxGrinder
     end
 
     def execute_os_plugin
-      raise "No operating system plugins installed. Install one or more operating system plugin. See http://community.jboss.org/docs/DOC-15081 and http://community.jboss.org/docs/DOC-15214 for more info." if PluginManager.instance.plugins[:os].empty?
+      raise "No operating system plugins installed. Install one or more operating system plugin. See http://boxgrinder.org/tutorials/boxgrinder-build-plugins/#Operating_system_plugins for more info." if PluginManager.instance.plugins[:os].empty?
 
       os_plugin, os_plugin_info = PluginManager.instance.initialize_plugin(:os, @appliance_config.os.name.to_sym)
       os_plugin.init(@config, @appliance_config, :log => @log, :plugin_info => os_plugin_info)
@@ -117,7 +117,7 @@ module BoxGrinder
         return previous_plugin_output
       end
 
-      raise "No platform plugins installed. Install one or more platform plugin. See http://community.jboss.org/docs/DOC-15081 and http://community.jboss.org/docs/DOC-15214 for more info." if PluginManager.instance.plugins[:platform].empty?
+      raise "No platform plugins installed. Install one or more platform plugin. See http://boxgrinder.org/tutorials/boxgrinder-build-plugins/#Platform_plugins for more info." if PluginManager.instance.plugins[:platform].empty?
 
       platform_plugin, platform_plugin_info = PluginManager.instance.initialize_plugin(:platform, @config.platform)
       platform_plugin.init(@config, @appliance_config, :log => @log, :plugin_info => platform_plugin_info, :previous_plugin_info => previous_plugin_output[:plugin_info], :previous_deliverables => previous_plugin_output[:deliverables])
@@ -140,7 +140,7 @@ module BoxGrinder
         return
       end
 
-      raise "No delivery plugins installed. Install one or more delivery plugin. See http://community.jboss.org/docs/DOC-15081 and http://community.jboss.org/docs/DOC-15214 for more info" if PluginManager.instance.plugins[:delivery].empty?
+      raise "No delivery plugins installed. Install one or more delivery plugin. See http://boxgrinder.org/tutorials/boxgrinder-build-plugins/#Delivery_plugins for more info" if PluginManager.instance.plugins[:delivery].empty?
 
       delivery_plugin, delivery_plugin_info = PluginManager.instance.initialize_plugin(:delivery, @config.delivery)
       delivery_plugin.init(@config, @appliance_config, :log => @log, :plugin_info => delivery_plugin_info, :previous_plugin_info => previous_plugin_output[:plugin_info], :previous_deliverables => previous_plugin_output[:deliverables])

data/lib/boxgrinder-build/helpers/guestfs-helper.rb

@@ -66,7 +66,8 @@ module Guestfs
         output = sh_original(command)
         puts output
       rescue => e
-        puts "Error occurred while executing above command. Appliance may not work properly."
+        puts "Error occurred while executing above command, aborting."
+        raise e
       end
 
       output
@@ -250,8 +251,9 @@ module BoxGrinder
     def sh(cmd, options = {})
       arch = options[:arch] || `uname -m`.chomp.strip
 
-      @log.debug "Executing #{cmd}"
-      @guestfs.sh("setarch #{arch} << SETARCH_EOF\n#{cmd.gsub('$', '\$')}\nSETARCH_EOF\n")
+      @log.debug "Executing '#{cmd}' command..."
+      @guestfs.sh("setarch #{arch} << 'SETARCH_EOF'\n#{cmd}\nSETARCH_EOF")
+      @log.debug "Command '#{cmd}' executed."
     end
 
     def augeas(&block)

data/lib/boxgrinder-build/helpers/image-helper.rb

@@ -52,6 +52,9 @@ module BoxGrinder
         @exec_helper.execute("mount #{loop_device} '#{mount_dir}#{mount_point}'")
       end
 
+      # Give some time to mount the images
+      sleep 2
+
       @log.trace "Mounts:\n#{mounts}"
 
       mounts
@@ -64,6 +67,9 @@ module BoxGrinder
 
       @exec_helper.execute("umount -d #{mounts['/']}")
 
+      # Give some time to umount the image
+      sleep 2
+
       FileUtils.rm_rf(mount_dir)
     end
 

data/lib/boxgrinder-build/helpers/plugin-helper.rb

@@ -20,6 +20,19 @@ require 'rubygems'
 require 'boxgrinder-build/managers/plugin-manager'
 require 'boxgrinder-core/helpers/log-helper'
 
+require 'boxgrinder-build/plugins/delivery/s3/s3-plugin'
+require 'boxgrinder-build/plugins/delivery/sftp/sftp-plugin'
+require 'boxgrinder-build/plugins/delivery/ebs/ebs-plugin'
+require 'boxgrinder-build/plugins/delivery/local/local-plugin'
+
+require 'boxgrinder-build/plugins/platform/vmware/vmware-plugin'
+require 'boxgrinder-build/plugins/platform/ec2/ec2-plugin'
+require 'boxgrinder-build/plugins/platform/virtualbox/virtualbox-plugin'
+
+require 'boxgrinder-build/plugins/os/centos/centos-plugin'
+require 'boxgrinder-build/plugins/os/rhel/rhel-plugin'
+require 'boxgrinder-build/plugins/os/fedora/fedora-plugin'
+
 module BoxGrinder
   class PluginHelper
     def initialize( config, options = {} )
@@ -42,9 +55,7 @@ module BoxGrinder
     end
 
     def read_and_require
-      plugins = %w(boxgrinder-build-fedora-os-plugin boxgrinder-build-rhel-os-plugin boxgrinder-build-centos-os-plugin boxgrinder-build-ec2-platform-plugin boxgrinder-build-vmware-platform-plugin boxgrinder-build-virtualbox-platform-plugin boxgrinder-build-s3-delivery-plugin boxgrinder-build-sftp-delivery-plugin boxgrinder-build-local-delivery-plugin boxgrinder-build-ebs-delivery-plugin) + @options.additional_plugins
-
-      plugins.flatten.each do |plugin|
+      @options.additional_plugins.each do |plugin|
         @log.trace "Loading plugin '#{plugin}'..."
 
         begin
@@ -52,7 +63,7 @@ module BoxGrinder
           @log.trace "- OK"
         rescue LoadError => e
           @log.trace "- Not found: #{e.message.strip.chomp}"
-          @log.warn "Specified plugin: '#{plugin}' wasn't found. Make sure its name is correct, skipping..." unless plugin.match(/^boxgrinder-build-(.*)-plugin/)
+          @log.warn "Specified plugin: '#{plugin}' wasn't found. Make sure its name is correct, skipping..."
         end
       end
     end

data/lib/boxgrinder-build/plugins/base-plugin.rb

@@ -20,6 +20,7 @@ require 'rubygems'
 require 'boxgrinder-core/helpers/exec-helper'
 require 'boxgrinder-core/helpers/log-helper'
 require 'boxgrinder-build/helpers/image-helper'
+require 'boxgrinder-build/managers/plugin-manager'
 require 'ostruct'
 require 'hashery/opencascade'
 require 'fileutils'