bosh-bootstrap 0.9.0 → 0.10.0

data/.travis.yml

@@ -15,8 +15,8 @@ env:
   matrix:
     - SUITE=unit
     # - SUITE=integration:aws:basic
-    - SUITE=integration:aws:edge
-    # - SUITE=integration:aws:edge_prebuilt
+    # - SUITE=integration:aws:edge
+    - SUITE=integration:aws:edge_prebuilt
     # - SUITE=integration:aws:edge_prebuilt_ami
   global:
     - secure: "OS3pnXOAVRP0QPDl/Nn/0iBPZbrnn9irFYwEbMJ2fGPsV00u4IYDwoIN5Gzb\nX+cXyiPZB0jdQlbiCMaQEYjYig3fgnDhiLj/MnxkVYtILm+0uPSrX2Zf7jvR\nmxCT5jfTSZoPXQeNi9h8rEqvaqKv4z930xp/Z0QNoZLPWdwY5wk="

data/ChangeLog.md

@@ -2,6 +2,17 @@
 
 `bosh-bootstrap` is a command line tool that you can run on your laptop and automatically get a microbosh (and an inception VM) deployed on either AWS or OpenStack.
 
+    gem install bosh-bootstrap
+    bosh-bootstrap deploy
+
+## v0.10
+
+* Only using latest pre-release bosh gems & stemcells/amis - do not upgrade if you want the old 2012 gems
+* AWS us-east-1 uses a pre-built AMI for extra speed
+* AWS all regions are working (using pre-created stemcell)
+* OpenStack is now working (using pre-created stemcell)
+* `deploy --create-inception` chooses to create an inception VM
+
 ## v0.9
 
 * v0.8 wasn't working for many people; and neither will v0.9; but its a move in the right direction.

data/CleanupCi.md

@@ -0,0 +1,8 @@
+# Clean up Ci
+
+Currently, the travis runs are not self-destroying the servers. If CI fails, ask @drnic to run:
+
+```
+fog = Fog::Compute::AWS.new(Fog.credentials.merge(region: 'us-east-1'))
+fog.security_groups.select {|sg| p sg.name =~ /inception-vm/}.each {|sg| fog.servers.select {|s| s.security_group_ids.include? sg.group_id}.each {|s| puts s; s.destroy}}; nil
+```

data/README.md

@@ -48,7 +48,7 @@ Stage 1: Choose infrastructure
 Found infrastructure API credentials at ~/.fog (override with --fog)
 1. AWS (default)
 2. AWS (bosh)
-3. Rackspace (default)
+3. Openstack (default)
 Choose infrastructure:  1
 
 Confirming: using AWS infrastructure.

data/docs/README.md

@@ -0,0 +1,3 @@
+#Tutorials
+
+[bosh-bootstrap on OpenStack](devstack-openstack-tutorial.md)

data/docs/devstack-openstack-tutorial.md

@@ -0,0 +1,215 @@
+#Deploy bosh-bootstrap on OpenStack (Using DevStack)
+
+
+This tutorial is intended for developers/testers without prior experience with OpenStack. However, it is also useful for OpenStack users as it provides interesting insights into this highly popular project.
+It is mainly designed to run bosh-bootstrap with minimum hardware requirements.
+
+######Note: 
+DevStack is only used for development/testing purposes. It is not for suitable for production mode.
+However, this tutorial still holds good for production scale OpenStack setup as there is hardly anything you have to do on your own,whatever the underlying IaaS is :)
+
+##Prerequisities
+
+###Hardware (Minimum)
+
+```
+
+4GB RAM (8 GB Preferred), 2 CPU Cores (4 Cores Preferred), 160GB Hard Disk
+
+```
+
+######Note: 
+The hardware requirements will scale up when you move towards deploying bosh-cloudfoundry (It's not covered in this tutorial)
+
+###Software
+
+```
+
+1. OS - Ubuntu 12.04 Server
+2. VM Image - Ubuntu 10.04 Server
+3. IaaS - OpenStack (DevStack in this case)
+
+```
+
+##Preparation
+
+#####Download devstack
+
+```
+
+git clone https://github.com/openstack-dev/devstack.git
+
+```
+#####Modify end-points in devstack/files/keystone_data.sh
+
+Change
+
+```
+
+keystone endpoint-create \
+            --region RegionOne \
+            --service_id $GLANCE_SERVICE \
+            --publicurl "http://$SERVICE_HOST:9292" \
+            --adminurl "http://$SERVICE_HOST:9292" \
+            --internalurl "http://$SERVICE_HOST:9292"
+            
+```
+
+to
+
+```
+
+keystone endpoint-create \
+            --region RegionOne \
+            --service_id $GLANCE_SERVICE \
+            --publicurl "http://$SERVICE_HOST:9292/v1.0" \
+            --adminurl "http://$SERVICE_HOST:9292/v1.0" \
+            --internalurl "http://$SERVICE_HOST:9292/v1.0"
+            
+```
+
+#####DevStack has a 5GB volume limit. However, you can increase it by modifying the following line in devstack/stackrc
+
+```
+
+VOLUME_BACKING_FILE_SIZE=${VOLUME_BACKING_FILE_SIZE:-5130M}
+
+```
+
+######Note : 
+This tutorial is based on 5GB volume limit only. But in general, having atleast 10GB volume for
+each instance is preferred. Feel free to play with volume limit.
+
+#####Install devstack
+
+```
+
+cd devstack & ./stack.sh
+
+```
+
+Define OpenStack env - OS_USERNAME,OS_PASSWORD,OS_TENANT_NAME,OS_AUTH_URL
+
+#####Download Ubuntu 10.04 server cloud image from http://cloud-images.ubuntu.com
+
+```
+
+wget http://cloud-images.ubuntu.com/lucid/current/lucid-server-cloudimg-amd64-disk1.img
+
+```
+
+#####Add image in OpenStack
+
+```
+
+$ name=Ubuntu_10.04
+$ image=lucid-server-cloudimg-amd64-disk1.img
+$ glance image-create --name=$name --is-public=true --container-format=bare --disk-format=qcow2 < $image
+
+```
+
+#####Add flavor [Make sure to add ephemeral disk as shown]
+
+```
+
+$ nova flavor-create m1.bosh 6 2048 20 2 --ephemeral 20 --rxtx-factor 1 --is-public true
+
+```
+
+#####Also, install ruby 1.9.3. Use rvm or any other method.
+
+#####Generate keypair (OPTIONAL)
+######Note : 
+As of 0.8+ bosh-bootstrap deploy uses openstack/aws to create a dedicated keypair and its stored in ~/.bosh_bootstrap/ssh/inception
+
+```
+
+ssh-keygen
+
+```
+#####Set up git
+
+```
+
+git config --global user.name "Your Name Here"
+git config --global user.email "your_email@example.com" 
+
+```
+
+We are done preparing the IaaS(OpenStack) part. Now lets move up the stack.
+
+##Play with bosh-bootstrap
+
+#####Download Gem
+
+```
+
+gem install bosh-bootstrap
+
+```
+
+#####bosh-bootstrap is designed to boot instance with 32GB inception VM and 16GB for BOSH server.
+For testing/development purpose, we will scale down the requirements(for devstack) to 3GB and 2GB respectively. Or change as per your requirements
+
+```
+
+export BOSH_VOLUME_SIZE=2 OR export MICROBOSH_VOLUME_SIZE=2 OR export MICRO_BOSH_VOLUME_SIZE=2
+
+```
+
+Run
+
+```
+
+INCEPTION_VOLUME_SIZE=3 bosh-bootstrap deploy
+
+```
+
+#####Start bootstrapping
+Deploy
+
+```
+
+bosh-bootstrap deploy
+
+```
+
+Answer few questions asked by bosh-bootstrap in the initial stages and then, sit back and relax (hopefully!)
+
+```
+
+Stage 1: Choose infrastructure
+Stage 2: BOSH configuration
+Stage 3: Create/Allocate the Inception VM
+Stage 4: Preparing the Inception VM
+Stage 5: Deploying micro BOSH
+Stage 6: Setup bosh
+
+```
+
+######Notes:
+
+a) If the process fails in between due to some reason, you can restart bootstrapping after correcting the error, bootstrapping will continue from the point where it failed instead of from the beginning.
+
+b) If you want to start the process from the beginning, delete "~/.bosh-boostrap/manifest.yml" file.
+
+c) In case, the Inception VM fails to connect to internet or bosh-bootstrap is unable to mount volume to the instance, then the most probable reason is due to floating ip.Then
+
+```
+
+Disassociate floating IP from the OpenStack dashboard.
+  
+Edit "~/.bosh-bootstrap/manifest.yml" file and change the public IP to fixed IP of the instance.
+  
+Redeploy bosh-bootstrap
+
+```
+
+#####Finishing bootstrap
+
+i) If everything goes fine, you can see the list of VMs created by bosh-bootstrap. Also you can ssh into the inception VM by "bosh-bootstrap ssh". Check BOSH status and so on.
+ 
+ii) You can see the list of commands by executing "bosh-bootstrap help"
+
+iii) In case, you face any issue please raise a [ticket](https://github.com/StarkAndWayne/bosh-bootstrap/issues). 
+

data/lib/bosh-bootstrap/cli.rb

@@ -28,6 +28,7 @@ module Bosh::Bootstrap
     method_option :"edge", :type => :boolean, :desc => "Use pre-built gems; create microbosh from source [temporary default]"
     method_option :fog, :type => :string, :desc => "fog config file (default: ~/.fog)"
     method_option :"upgrade-deps", :type => :boolean, :desc => "Force upgrade dependencies, packages & gems"
+    method_option :"create-inception", :type => :boolean, :desc => "Choose to create an inception VM"
     def deploy
       migrate_old_settings
       load_deploy_options # from method_options above
@@ -39,8 +40,9 @@ module Bosh::Bootstrap
       deploy_stage_3_create_allocate_inception_vm
       deploy_stage_4_prepare_inception_vm
       deploy_stage_5_salted_password
-      deploy_stage_6_deploy_micro_bosh
-      deploy_stage_7_setup_new_bosh
+      deploy_stage_6_download_micro_bosh
+      deploy_stage_7_deploy_micro_bosh
+      deploy_stage_8_setup_new_bosh
     end
 
     desc "upgrade-inception", "Upgrade inception VM with latest packages, gems, security group ports"
@@ -102,6 +104,13 @@ module Bosh::Bootstrap
       DEFAULT_MICROBOSH_VOLUME_SIZE = 16 # Gb
 
       def deploy_stage_1_choose_infrastructure_provider
+        settings["git"] ||= {}
+        settings["git"]["name"] ||= `git config user.name`.strip
+        settings["git"]["email"] ||= `git config user.email`.strip
+        if settings["git"]["name"].empty? || settings["git"]["email"].empty?
+          error "Checking for git identity....Cannot find your git identity. Please set git user.name and user.email before deploying"
+        end
+
         header "Stage 1: Choose infrastructure"
         unless settings[:fog_credentials]
           choose_fog_provider
@@ -298,30 +307,40 @@ module Bosh::Bootstrap
         end
       end
 
-      def deploy_stage_6_deploy_micro_bosh
-        header "Stage 6: Deploying micro BOSH"
+      def deploy_stage_6_download_micro_bosh
+        header "Stage 6: Download micro BOSH"
         recreate_local_ssh_keys_for_inception_vm
         switch_to_prebuilt_microbosh_ami_if_available
 
-        unless run_server(Bosh::Bootstrap::Stages::MicroBoshDeploy.new(settings).commands)
-          error "Failed to complete Stage 6: Deploying micro BOSH"
+        unless run_server(Bosh::Bootstrap::Stages::MicroBoshDownload.new(settings).commands)
+          error "Failed to complete Stage 6: Downloading micro BOSH"
         end
         # Settings are updated by this stage
-        # it generates a salted password from settings.bosh.password
-        # and stores it in settings.bosh.salted_password
+        # It may update the micro_bosh_stemcell_name
         save_settings!
 
         confirm "Successfully built micro BOSH"
       end
 
-      def deploy_stage_7_setup_new_bosh
+      def deploy_stage_7_deploy_micro_bosh
+        header "Stage 7: Deploying micro BOSH"
+        recreate_local_ssh_keys_for_inception_vm
+
+        unless run_server(Bosh::Bootstrap::Stages::MicroBoshDeploy.new(settings).commands)
+          error "Failed to complete Stage 7: Deploying micro BOSH"
+        end
+
+        confirm "Successfully built micro BOSH"
+      end
+
+      def deploy_stage_8_setup_new_bosh
         # TODO change to a polling test of director being available
         say "Pausing to wait for BOSH Director..."
         sleep 5
 
-        header "Stage 7: Setup bosh"
+        header "Stage 8: Setup bosh"
         unless run_server(Bosh::Bootstrap::Stages::SetupNewBosh.new(settings).commands)
-          error "Failed to complete Stage 6: Setup bosh"
+          error "Failed to complete Stage 7: Setup bosh"
         end
 
         say "Locally targeting and login to new BOSH..."
@@ -495,30 +514,25 @@ module Bosh::Bootstrap
         else
           settings.delete("upgrade_deps")
         end
+
+        if options["create-inception"]
+          settings["inception"] = {"create_new" => true}
+        end
         save_settings!
       end
 
       def load_provider_specific_options
         # before deploy stage - need to change type => ami if AWS us-east-1?
         if options[:"edge-prebuilt"] || settings.delete("edge-prebuilt")
-          if aws?
-            settings["micro_bosh_stemcell_type"] = "edge-prebuilt"
-            settings["micro_bosh_stemcell_name"] = "edge-prebuilt"
-          else
-            error "Currently --edge-prebuilt is not available for #{bosh_provider} :("
-          end
+          settings["micro_bosh_stemcell_type"] = "edge-prebuilt"
+          settings["micro_bosh_stemcell_name"] = "edge-prebuilt"
         elsif options[:"edge"] || settings.delete("edge")
           settings["micro_bosh_stemcell_type"] = "custom"
           settings["micro_bosh_stemcell_name"] = "custom"
         else
-          if settings["fog_credentials"] && aws?
-            # currently defaulting to latest prebuilt stemcells/amis until 1.5.0 is released
-            settings["micro_bosh_stemcell_type"] = "edge-prebuilt"
-            settings["micro_bosh_stemcell_name"] = "edge-prebuilt"
-          else
-            settings["micro_bosh_stemcell_type"] = "custom"
-            settings["micro_bosh_stemcell_name"] = "custom"
-          end
+          # currently defaulting to latest prebuilt stemcells/amis until 1.5.0 is released
+          settings["micro_bosh_stemcell_type"] = "edge-prebuilt"
+          settings["micro_bosh_stemcell_name"] = "edge-prebuilt"
         end
       end
 
@@ -657,9 +671,9 @@ module Bosh::Bootstrap
           menu.choice("OpenStack") do
             creds[:provider] = "OpenStack"
             creds[:openstack_username] = hl.ask("Username: ")
-            creds[:openstack_api_key] = hl.ask("API key: ")
+            creds[:openstack_api_key] = hl.ask("Password: ")
             creds[:openstack_tenant] = hl.ask("Tenant: ")
-            creds[:openstack_auth_url] = hl.ask("Authorization URL: ")
+            creds[:openstack_auth_url] = hl.ask("Authorization Token URL: ")
           end
         end
         @fog_credentials = creds

data/lib/bosh-bootstrap/commander/remote_server.rb

@@ -54,6 +54,9 @@ class Bosh::Bootstrap::Commander::RemoteServer
     last_line = $1
     # store output into a settings field, if requested
     # TODO replace this with SettingsSetting#setting(settings_key, last_line.strip)
+
+    puts "Last line of script: #{last_line}"
+    puts "Storing to settings_key #{settings_key}" if settings_key
     if settings_key
       settings_key_portions = settings_key.split(".")
       parent_key_portions, final_key = settings_key_portions[0..-2], settings_key_portions[-1]
@@ -64,6 +67,7 @@ class Bosh::Bootstrap::Commander::RemoteServer
       end
       target_settings_field[final_key] = last_line.strip
     end
+
     status
   rescue StandardError => e
     logfile.puts e.message

data/lib/bosh-bootstrap/helpers/settings.rb

@@ -5,6 +5,7 @@ module Bosh; module Bootstrap; module Helpers; end; end; end
 
 # Helper methods for loading/saving settings
 module Bosh::Bootstrap::Helpers::Settings
+
   # Previously selected settings are stored in a YAML manifest
   # Protects the manifest file with user-only priveleges
   def settings

data/lib/bosh-bootstrap/stages/stage_micro_bosh_deploy.rb

@@ -13,12 +13,6 @@ module Bosh::Bootstrap::Stages
       settings[:bosh_name] ||= "unnamed_bosh"
 
       @commands ||= Bosh::Bootstrap::Commander::Commands.new do |server|
-        server.download "micro-bosh stemcell", script("download_micro_bosh_stemcell",
-                      "MICRO_BOSH_STEMCELL_NAME" => settings.micro_bosh_stemcell_name,
-                      "MICRO_BOSH_STEMCELL_TYPE" => settings.micro_bosh_stemcell_type,
-                      "PROVIDER" => settings.bosh_provider),
-                      :settings => settings,
-                      :save_output_to_settings_key => "micro_bosh_stemcell_name"
         server.upload_file \
                       "/var/vcap/store/microboshes/deployments/#{settings.bosh_name}/micro_bosh.yml",
                       micro_bosh_manifest

data/lib/bosh-bootstrap/stages/{stage_micro_bosh_deploy → stage_micro_bosh_download}/download_micro_bosh_stemcell

@@ -6,9 +6,9 @@
 # * $MICRO_BOSH_STEMCELL_NAME - public stemcell name to be downloaded
 #                             - if 'custom' the create stemcell from BOSH source
 # * $MICRO_BOSH_STEMCELL_TYPE - what type of stemcell (ami vs tgz)
+# * $PROVIDER - e.g. aws, openstack
 #
 # Optional:
-# * $PROVIDER - required for 'custom' $MICRO_BOSH_STEMCELL_NAME; e.g. aws, openstack
 # * $ISO_NAME - defaults to ubuntu-10.04.4-server-amd64.iso for creating stemcell
 
 set -e # exit immediately if a simple command exits with a non-zero status
@@ -20,7 +20,7 @@ if [[ "${MICRO_BOSH_STEMCELL_NAME}X" == "X" ]]; then
 fi
 
 STEMCELLS_DIR=/var/vcap/store/stemcells
-LAST_SUCCESSFUL_MICROBOSH_STEMCELL_URL=http://bosh-jenkins-artifacts.s3.amazonaws.com/last_successful_micro-bosh-stemcell.tgz
+LAST_SUCCESSFUL_MICROBOSH_STEMCELL_URL=http://bosh-jenkins-artifacts.s3.amazonaws.com/last_successful_micro-bosh-stemcell-$PROVIDER.tgz
 
 if [[ "${MICRO_BOSH_STEMCELL_NAME}" == "custom" ]]; then
 
@@ -66,8 +66,15 @@ if [[ "${MICRO_BOSH_STEMCELL_NAME}" == "custom" ]]; then
 
 elif [[ "${MICRO_BOSH_STEMCELL_TYPE}" == "edge-prebuilt" ]]; then
   cd $STEMCELLS_DIR
-  curl -O $LAST_SUCCESSFUL_MICROBOSH_STEMCELL_URL
-  MICRO_BOSH_STEMCELL_NAME=$(basename $LAST_SUCCESSFUL_MICROBOSH_STEMCELL_URL)
+  echo Looking for latest microbosh stemcell at $LAST_SUCCESSFUL_MICROBOSH_STEMCELL_URL
+  etag=$(curl -I $LAST_SUCCESSFUL_MICROBOSH_STEMCELL_URL 2>&1 | awk '/^ETag/ {print $2}' | sed -e 's/"//g')
+  MICRO_BOSH_STEMCELL_NAME="micro_bosh-${etag}-latest.tgz"
+  if [[ ! -f $MICRO_BOSH_STEMCELL_NAME ]]; then
+    echo "Downloading latest microbosh stemcell, etag $etag..."
+    curl -o $MICRO_BOSH_STEMCELL_NAME $LAST_SUCCESSFUL_MICROBOSH_STEMCELL_URL
+  else
+    echo "Latest microbosh stemcell with etag $etag already downloaded, skipping."
+  fi
 
 elif [[ "${MICRO_BOSH_STEMCELL_TYPE}" == "ami" ]]; then
   echo "Using AMI ${MICRO_BOSH_STEMCELL_NAME}, so there is nothing to download."

data/lib/bosh-bootstrap/stages/stage_micro_bosh_download.rb

@@ -0,0 +1,139 @@
+require "json" # for inline hashes within YAML
+
+module Bosh::Bootstrap::Stages
+  class MicroBoshDownload
+    attr_reader :settings
+
+    def initialize(settings)
+      @settings = settings
+    end
+
+    # TODO "aws_us_east_1" should come from settings.bosh_name
+    def commands
+      settings[:bosh_name] ||= "unnamed_bosh"
+
+      @commands ||= Bosh::Bootstrap::Commander::Commands.new do |server|
+        server.download "micro-bosh stemcell", script("download_micro_bosh_stemcell",
+                      "MICRO_BOSH_STEMCELL_NAME" => settings.micro_bosh_stemcell_name,
+                      "MICRO_BOSH_STEMCELL_TYPE" => settings.micro_bosh_stemcell_type,
+                      "PROVIDER" => settings.bosh_provider),
+                      :settings => settings,
+                      :save_output_to_settings_key => "micro_bosh_stemcell_name"
+      end
+    end
+
+    def stage_name
+      "stage_micro_bosh_download"
+    end
+
+    # Loads local script
+    # If +variables+, then injects KEY=VALUE environment
+    # variables into bash scripts.
+    def script(segment_name, variables={})
+      path = File.expand_path("../#{stage_name}/#{segment_name}", __FILE__)
+      if File.exist?(path)
+        script = File.read(path)
+        if variables.keys.size > 0
+          env_variables = variables.reject { |var| var.is_a?(Symbol) }
+
+          # inject variables into script if its bash script
+          inline_variables = "#!/usr/bin/env bash\n\n"
+          env_variables.each { |name, value| inline_variables << "#{name}='#{value}'\n" }
+          script.gsub!("#!/usr/bin/env bash", inline_variables)
+
+          # inject variables into script if its ruby script
+          inline_variables = "#!/usr/bin/env ruby\n\n"
+          env_variables.each { |name, value| inline_variables << "ENV['#{name}'] = '#{value}'\n" }
+          script.gsub!("#!/usr/bin/env ruby", inline_variables)
+        end
+        script
+      else
+        Thor::Base.shell.new.say_status "error", "Missing script lib/bosh-bootstrap/stages/#{stage_name}/#{segment_name}", :red
+        exit 1
+      end
+    end
+
+    def micro_bosh_manifest
+      name                       = settings.bosh_name
+      salted_password            = settings.bosh.salted_password
+      ipaddress                  = settings.bosh.ip_address
+      persistent_disk            = settings.bosh.persistent_disk
+      resources_cloud_properties = settings.bosh_resources_cloud_properties
+      cloud_plugin               = settings.bosh_provider
+
+      # aws:
+      #   access_key_id:     #{access_key}
+      #   secret_access_key: #{secret_key}
+      #   ec2_endpoint: ec2.#{region}.amazonaws.com
+      #   default_key_name: #{key_name}
+      #   default_security_groups: ["#{security_group}"]
+      #   ec2_private_key: /home/vcap/.ssh/#{key_name}.pem
+      cloud_properties = settings.bosh_cloud_properties
+
+      manifest = {
+        "name" => name,
+        "env" => { "bosh" => {"password" => salted_password}},
+        "logging" => { "level" => "DEBUG" },
+        "network" => { "type" => "dynamic", "vip" => ipaddress },
+        "resources" => {
+          "persistent_disk" => persistent_disk,
+          "cloud_properties" => resources_cloud_properties
+        },
+        "cloud" => {
+          "plugin" => cloud_plugin,
+          "properties" => cloud_properties
+        },
+        "apply_spec" => {
+          "agent" => {
+            "blobstore" => { "address" => ipaddress },
+            "nats" => { "address" => ipaddress }
+          },
+          "properties" => {
+            "#{cloud_plugin.downcase}_registry" => { "address" => ipaddress }
+          }
+        }
+      }
+
+      # Openstack settings
+      if cloud_plugin.downcase == "openstack"
+        # Delete OpenStack registry IP address
+        manifest["apply_spec"]["properties"].delete("openstack_registry")
+
+        # OpenStack private network label
+        if settings.network_label
+          manifest["network"]["label"] = settings.network_label
+        end
+      end
+
+      manifest.to_yaml.gsub(/\s![^ ]+$/, '')
+
+      # /![^ ]+\s/ removes object notation from the YAML which appears to cause problems when being interpretted by the
+      # Ruby running on the inception vm. A before and after example would look like;
+      #
+      #   properties: !map:Settingslogic
+      #     openstack: !map:Settingslogic
+      #       username: admin
+      #       api_key: xxxxxxxxxxxxxxxxxxx
+      #       tenant: CloudFoundry
+      #       auth_url: http://192.168.1.2:5000/v2.0/tokens
+      #       default_security_groups:
+      #       - !str:HighLine::String microbosh-openstack
+      #       default_key_name: !str:HighLine::String microbosh-openstack
+      #       private_key: /home/vcap/.ssh/microbosh-openstack.pem
+      #
+      # The regex strips the !Module::ClassName notation out and the result looks as it should
+      #
+      #   properties:
+      #     openstack:
+      #       username: admin
+      #       api_key: xxxxxxxxxxxxxxxxxxx
+      #       tenant: CloudFoundry
+      #       auth_url: http://192.168.1.2:5000/v2.0/tokens
+      #       default_security_groups:
+      #       - microbosh-openstack
+      #       default_key_name: microbosh-openstack
+      #       private_key: /home/vcap/.ssh/microbosh-openstack.pem
+
+    end
+  end
+end

data/lib/bosh-bootstrap/stages.rb

@@ -3,6 +3,7 @@ end
 
 require "bosh-bootstrap/stages/stage_validate_inception_vm"
 require "bosh-bootstrap/stages/stage_prepare_inception_vm"
+require "bosh-bootstrap/stages/stage_micro_bosh_download"
 require "bosh-bootstrap/stages/stage_micro_bosh_deploy"
 require "bosh-bootstrap/stages/stage_setup_new_bosh"
 require "bosh-bootstrap/stages/stage_salted_password"

data/lib/bosh-bootstrap/version.rb

@@ -1,5 +1,5 @@
 module Bosh
   module Bootstrap
-    VERSION = "0.9.0"
+    VERSION = "0.10.0"
   end
 end

data/spec/unit/aws_spec.rb

@@ -64,7 +64,7 @@ describe "AWS deployment" do
     @cmd.should_receive(:provision_and_mount_volume)
     @cmd.stub(:run_server).and_return(true)
     @cmd.stub(:sleep)
-    @cmd.should_receive(:deploy_stage_7_setup_new_bosh)
+    @cmd.should_receive(:deploy_stage_8_setup_new_bosh)
     @cmd.deploy
 
     fog.addresses.should have(1).item # assigned to inception VM
@@ -118,7 +118,7 @@ describe "AWS deployment" do
     @cmd.should_receive(:provision_and_mount_volume)
     @cmd.stub(:run_server).and_return(true)
     @cmd.stub(:sleep)
-    @cmd.should_receive(:deploy_stage_7_setup_new_bosh)
+    @cmd.should_receive(:deploy_stage_8_setup_new_bosh)
     @cmd.deploy
     @settings = nil # reload settings file
 
@@ -161,7 +161,7 @@ describe "AWS deployment" do
     @cmd.should_receive(:provision_and_mount_volume)
     @cmd.stub(:run_server).and_return(true)
     @cmd.stub(:sleep)
-    @cmd.should_receive(:deploy_stage_7_setup_new_bosh)
+    @cmd.should_receive(:deploy_stage_8_setup_new_bosh)
     @cmd.should_receive(:latest_prebuilt_microbosh_ami).and_return("ami-123456")
 
     setting "edge-prebuilt", true

data/spec/unit/cli_spec.rb

@@ -21,8 +21,9 @@ describe Bosh::Bootstrap do
       deploy_stage_3_create_allocate_inception_vm
       deploy_stage_4_prepare_inception_vm
       deploy_stage_5_salted_password
-      deploy_stage_6_deploy_micro_bosh
-      deploy_stage_7_setup_new_bosh
+      deploy_stage_6_download_micro_bosh
+      deploy_stage_7_deploy_micro_bosh
+      deploy_stage_8_setup_new_bosh
     ]
     stage_methods.each do |method|
       unless method =~ /#{stage}/
@@ -43,8 +44,9 @@ describe Bosh::Bootstrap do
       @cmd.should_receive(:deploy_stage_3_create_allocate_inception_vm)
       @cmd.should_receive(:deploy_stage_4_prepare_inception_vm)
       @cmd.should_receive(:deploy_stage_5_salted_password)
-      @cmd.should_receive(:deploy_stage_6_deploy_micro_bosh)
-      @cmd.should_receive(:deploy_stage_7_setup_new_bosh)
+      @cmd.should_receive(:deploy_stage_6_download_micro_bosh)
+      @cmd.should_receive(:deploy_stage_7_deploy_micro_bosh)
+      @cmd.should_receive(:deploy_stage_8_setup_new_bosh)
       @cmd.deploy
     end
 
@@ -68,8 +70,8 @@ describe Bosh::Bootstrap do
       @cmd.deploy
     end
 
-    it "stage 6 - download stemcell and deploy microbosh" do
-      testing_stage(6)
+    it "stage 7 - deploy microbosh" do
+      testing_stage(7)
       @cmd.should_receive(:recreate_local_ssh_keys_for_inception_vm)
       setting "bosh_provider", "aws"
       setting "micro_bosh_stemcell_name", "micro-bosh-stemcell-aws-0.8.1.tgz"
@@ -86,8 +88,8 @@ describe Bosh::Bootstrap do
       @cmd.deploy
     end
 
-    it "stage 7 - sets up new microbosh" do
-      testing_stage(7)
+    it "stage 8 - sets up new microbosh" do
+      testing_stage(8)
       setting "bosh_name", "microbosh-aws-us-east-1"
       setting "bosh_username", "drnic"
       setting "bosh_password", "password"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: bosh-bootstrap
 version: !ruby/object:Gem::Version
-  version: 0.9.0
+  version: 0.10.0
   prerelease: 
 platform: ruby
 authors:
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-04-11 00:00:00.000000000 Z
+date: 2013-04-20 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: thor
@@ -231,6 +231,7 @@ files:
 - .rspec
 - .travis.yml
 - ChangeLog.md
+- CleanupCi.md
 - Gemfile
 - Guardfile
 - LICENSE.txt
@@ -238,6 +239,8 @@ files:
 - Rakefile
 - bin/bosh-bootstrap
 - bosh-bootstrap.gemspec
+- docs/README.md
+- docs/devstack-openstack-tutorial.md
 - lib/bosh-bootstrap.rb
 - lib/bosh-bootstrap/cli.rb
 - lib/bosh-bootstrap/commander.rb
@@ -257,8 +260,9 @@ files:
 - lib/bosh-bootstrap/stages/stage_micro_bosh_delete/bosh_micro_delete
 - lib/bosh-bootstrap/stages/stage_micro_bosh_deploy.rb
 - lib/bosh-bootstrap/stages/stage_micro_bosh_deploy/bosh_micro_deploy
-- lib/bosh-bootstrap/stages/stage_micro_bosh_deploy/download_micro_bosh_stemcell
 - lib/bosh-bootstrap/stages/stage_micro_bosh_deploy/install_key_pair_for_user
+- lib/bosh-bootstrap/stages/stage_micro_bosh_download.rb
+- lib/bosh-bootstrap/stages/stage_micro_bosh_download/download_micro_bosh_stemcell
 - lib/bosh-bootstrap/stages/stage_prepare_inception_vm.rb
 - lib/bosh-bootstrap/stages/stage_prepare_inception_vm/configure_git
 - lib/bosh-bootstrap/stages/stage_prepare_inception_vm/create_vcap_user
@@ -308,16 +312,16 @@ files:
 - vendor/cache/blobstore_client-0.4.0.gem
 - vendor/cache/bosh_cli-1.0.3.gem
 - vendor/cache/bosh_common-0.5.4.gem
-- vendor/cache/builder-3.1.4.gem
+- vendor/cache/builder-3.2.0.gem
 - vendor/cache/coderay-1.0.8.gem
 - vendor/cache/diff-lcs-1.1.3.gem
 - vendor/cache/escape-0.0.4.gem
-- vendor/cache/excon-0.17.0.gem
+- vendor/cache/excon-0.20.1.gem
 - vendor/cache/fog-1.8.0.gem
 - vendor/cache/formatador-0.2.4.gem
 - vendor/cache/guard-1.6.2.gem
 - vendor/cache/guard-rspec-2.4.0.gem
-- vendor/cache/highline-1.6.15.gem
+- vendor/cache/highline-1.6.18.gem
 - vendor/cache/httpclient-2.2.4.gem
 - vendor/cache/i18n-0.6.1.gem
 - vendor/cache/json_pure-1.6.8.gem
@@ -325,14 +329,13 @@ files:
 - vendor/cache/log4r-1.1.10.gem
 - vendor/cache/lumberjack-1.0.2.gem
 - vendor/cache/method_source-0.8.1.gem
-- vendor/cache/mime-types-1.21.gem
+- vendor/cache/mime-types-1.22.gem
 - vendor/cache/multi_json-1.1.0.gem
 - vendor/cache/net-scp-1.0.4.gem
 - vendor/cache/net-ssh-2.2.2.gem
 - vendor/cache/net-ssh-gateway-1.1.0.gem
 - vendor/cache/netaddr-1.5.0.gem
-- vendor/cache/nokogiri-1.5.6-java.gem
-- vendor/cache/nokogiri-1.5.6.gem
+- vendor/cache/nokogiri-1.5.9.gem
 - vendor/cache/open4-1.3.0.gem
 - vendor/cache/progressbar-0.9.2.gem
 - vendor/cache/pry-0.9.11.4-java.gem
@@ -373,7 +376,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
       segments:
       - 0
-      hash: 2829308675066095393
+      hash: 1266362034070709822
 requirements: []
 rubyforge_project: 
 rubygems_version: 1.8.25