vcloud-launcher 0.0.3 → 0.0.4

data/.gitignore

@@ -2,15 +2,10 @@
 *.rbc
 *.swp
 *.un~
-vcloud_env.sh
-replace_variables.sh
 /.bundle/
 /.ruby-version
 /Gemfile.lock
 /bundle/
 /pkg/
-vendor/
-coverage/
-results.html
 fog_integration_test.config
 .idea

data/CHANGELOG.md

@@ -1,3 +1,7 @@
+## 0.0.4 (2014-05-01)
+
+  - Use pessimistic version dependency for vcloud-core
+
 ## 0.0.3 (2014-04-22)
 
 Features:

data/README.md

@@ -1,7 +1,7 @@
 vCloud Launcher
 ===============
-A tool that takes a YAML or JSON configuration file describing a vDC, and provisions
-the vApps and VMs contained within.
+A tool that takes a YAML or JSON configuration file describing a vDC, and
+provisions the vApps and VMs contained within.
 
 ### Supports
 
@@ -14,10 +14,15 @@ the vApps and VMs contained within.
 
 ### Limitations
 
-- Source vApp Template must contain a single VM. This is VMware's recommended 'simple' method of vApp creation. Complex multi-VM vApps are not supported.
+- Source vApp Template must contain a single VM. This is VMware's recommended
+'simple' method of vApp creation. Complex multi-VM vApps are not supported.
 - Org vDC Networks must be precreated.
-- IP addresses are assigned manually (recommended) or via DHCP. VM IP pools are not supported.
-- vCloud has some interesting ideas about the size of potential 'guest customisation scripts' (aka preambles). You may need to use an external minify tool to reduce the size, or speak to your provider to up the limit. 2048 bytes seems to be a practical default maximum.
+- IP addresses are assigned manually (recommended) or via DHCP. VM IP pools are
+not supported.
+- vCloud has some interesting ideas about the size of potential 'guest
+customisation scripts' (aka preambles). You may need to use an external minify
+tool to reduce the size, or speak to your provider to up the limit. 2048 bytes
+seems to be a practical default maximum.
 
 ## Installation
 
@@ -40,72 +45,70 @@ Or install it yourself as:
 
 ## Credentials
 
-vCloud Launcher uses [Fog](http://fog.io/). To use it you'll need to give it credentials that allow it to talk to a VMware environment. Fog offers two ways to do this.
+vCloud Launcher is based around [fog](http://fog.io/). To use it you'll need to give it
+credentials that allow it to talk to a vCloud Director environment.
 
-### 1. Create a `.fog` file containing your credentials
+1. Create a '.fog' file in your home directory.
 
-To use this method, you need a `.fog` file in your home directory.
+  For example:
 
-For example:
+      test_credentials:
+        vcloud_director_host: 'host.api.example.com'
+        vcloud_director_username: 'username@org_name'
+        vcloud_director_password: ''
 
-    test:
-      vcloud_director_username: 'username@org_name'
-      vcloud_director_password: 'password'
-      vcloud_director_host: 'host.api.example.com'
+2. Obtain a session token. First, curl the API:
 
-Unfortunately current usage of fog requires the password in this file. Multiple sets of credentials can be specified in the fog file, using the following format:
+        curl -D- -d '' \
+            -H 'Accept: application/*+xml;version=5.1' -u '<username>@<org_name>' \
+            https://<host.api.example.com>/api/sessions
 
-    test:
-      vcloud_director_username: 'username@org_name'
-      vcloud_director_password: 'password'
-      vcloud_director_host: 'host.api.example.com'
+  This will prompt for your password.
 
-    test2:
-      vcloud_director_username: 'username@org_name'
-      vcloud_director_password: 'password'
-      vcloud_director_host: 'host.api.vendor.net'
+  From the headers returned, the value of the `x-vcloud-authorization` header is your
+  session token, and this will be valid for 30 minutes idle - any activity will extend
+  its life by another 30 minutes.
 
-You can then pass the `FOG_CREDENTIAL` environment variable at the start of your command. The value of the `FOG_CREDENTIAL` environment variable is the name of the credential set in your fog file which you wish to use.  For instance:
+3. Specify your credentials and session token at the beginning of the command. For example:
 
-    FOG_CREDENTIAL=test2 bundle exec vcloud-launch node.yaml
+        FOG_CREDENTIAL=test_credentials \
+            FOG_VCLOUD_TOKEN=AAAABBBBBCCCCCCDDDDDDEEEEEEFFFFF= \
+            vcloud-launch node.yaml
 
-To understand more about `.fog` files, visit the 'Credentials' section here => http://fog.io/about/getting_started.html.
+  You may find it easier to export one or both of the values as environment variables.
 
-### 2. Log on externally and supply your session token
+  **NB** It is also possible to sidestep the need for the session token by saving your
+  password in the fog file. This is **not recommended**.
 
-You can choose to log on externally by interacting independently with the API and supplying your session token to the
-tool by setting the `FOG_VCLOUD_TOKEN` ENV variable. This option reduces the risk footprint by allowing the user to
-store their credentials in safe storage. The default token lifetime is '30 minutes idle' - any activity extends the life by another 30 mins.
+## Contributing
 
-A basic example of this would be the following:
+1. Fork it
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create new Pull Request
 
-    curl
-       -D-
-       -d ''
-       -H 'Accept: application/*+xml;version=5.1' -u '<user>@<org>'
-       https://host.com/api/sessions
+## Other settings
 
-This will prompt for your password.
+vCloud Launcher uses vCloud Core. If you want to use the latest version of
+vCloud Core, or a local version, you can export some variables. See the Gemfile
+for details.
 
-From the headers returned, select the header below
+## The vCloud API
 
-     x-vcloud-authorization: AAAABBBBBCCCCCCDDDDDDEEEEEEFFFFF=
+vCloud Tools currently use version 5.1 of the [vCloud API](http://pubs.vmware.com/vcd-51/index.jsp?topic=%2Fcom.vmware.vcloud.api.doc_51%2FGUID-F4BF9D5D-EF66-4D36-A6EB-2086703F6E37.html). Version 5.5 may work but is not currently supported. You should be able to access the 5.1 API in a 5.5 environment, and this *is* currently supported.
 
-Use token as ENV var FOG_VCLOUD_TOKEN
+The default version is defined in [Fog](https://github.com/fog/fog/blob/244a049918604eadbcebd3a8eaaf433424fe4617/lib/fog/vcloud_director/compute.rb#L32).
 
-    FOG_VCLOUD_TOKEN=AAAABBBBBCCCCCCDDDDDDEEEEEEFFFFF= bundle exec ...
+If you want to be sure you are pinning to 5.1, or use 5.5, you can set the API version to use in your fog file, e.g.
 
-## Contributing
+`vcloud_director_api_version: 5.1`
 
-1. Fork it
-2. Create your feature branch (`git checkout -b my-new-feature`)
-3. Commit your changes (`git commit -am 'Add some feature'`)
-4. Push to the branch (`git push origin my-new-feature`)
-5. Create new Pull Request
+## Debugging
 
-## Other settings
+`export EXCON_DEBUG=true` - this will print out the API requests and responses.
 
-vCloud Launcher uses vCloud Core. If you want to use the latest version of vCloud Core, or a local version, you can export some variables. See the Gemfile for details.
+`export DEBUG=true` - this will show you the stack trace when there is an exception instead of just the message.
 
 ## Testing
 
@@ -113,9 +116,12 @@ Default target: `bundle exec rake`
 Runs the unit and feature tests (pretty quick right now)
 
 * Unit tests only: `bundle exec rake spec`
+* Feature tests only: `bundle exec rake features`
 * Integration tests ('quick' tests): `bundle exec rake integration:quick`
 * Integration tests (all tests - takes 20mins+): `bundle exec rake integration:all`
 
+NB. `bundle exec rake integration` is an alias for `bundle exec rake integration:all`.
+
 You need access to a suitable vCloud Director organization to run the
 integration tests. It is not necessarily safe to run them against an existing
 environment, unless care is taken with the entities being tested.
@@ -140,7 +146,11 @@ Then run this before you run the integration tests.
 
 #### Storage profile tests
 
-There is an integration test to check storage profile behaviour, but it requires a lot of set-up so it is not called by the rake task. If you wish to run it you need access to an environment that has two VDCs, each one containing a storage profile with the same name. This named storage profile needs to be different from teh default storage profile.
+There is an integration test to check storage profile behaviour, but it requires
+a lot of set-up so it is not called by the rake task. If you wish to run it you
+need access to an environment that has two VDCs, each one containing a storage
+profile with the same name. This named storage profile needs to be different
+from the default storage profile.
 
 You will need to set the following environment variables:
 

data/Rakefile

@@ -1,25 +1,9 @@
-require 'rake/clean'
-require 'rake/testtask'
 require 'cucumber/rake/task'
 require 'rspec/core/rake_task'
+require 'gem_publisher'
 
-require 'vcloud/launcher/version'
-
-include Rake::DSL
-
-Bundler::GemHelper.install_tasks
-
-Rake::TestTask.new do |t|
-  t.pattern = 'test/tc_*.rb'
-end
-
-CUKE_RESULTS = 'results.html'
-CLEAN << CUKE_RESULTS
-
-Cucumber::Rake::Task.new(:features) do |t|
-  t.cucumber_opts = "features --format html -o #{CUKE_RESULTS} --format pretty --no-source -x"
-  t.fork = false
-end
+task :default => [:spec,:features]
+task :integration => ['integration:all']
 
 RSpec::Core::RakeTask.new(:spec) do |task|
   # Set a bogus Fog credential, otherwise it's possible for the unit
@@ -29,6 +13,11 @@ RSpec::Core::RakeTask.new(:spec) do |task|
   task.pattern = FileList['spec/vcloud/**/*_spec.rb']
 end
 
+Cucumber::Rake::Task.new(:features) do |t|
+  t.cucumber_opts = "--format pretty --no-source"
+  t.fork = false
+end
+
 RSpec::Core::RakeTask.new('integration:quick') do |t|
   t.rspec_opts = %w(--tag ~take_too_long)
   t.pattern = FileList['spec/integration/**/*_spec.rb']
@@ -38,9 +27,6 @@ RSpec::Core::RakeTask.new('integration:all') do |t|
   t.pattern = FileList['spec/integration/**/*_spec.rb']
 end
 
-task :default => [:spec,:features]
-
-require "gem_publisher"
 task :publish_gem do |t|
   gem = GemPublisher.publish_if_updated("vcloud-launcher.gemspec", :rubygems)
   puts "Published #{gem}" if gem

data/jenkins.sh

@@ -1,11 +1,8 @@
 #!/bin/bash -x
 set -e
 bundle install --path "${HOME}/bundles/${JOB_NAME}"
-bundle exec rake 
+bundle exec rake
 
-./scripts/generate_fog_conf_file.sh
-export FOG_RC=fog_integration_test.config
-bundle exec rake integration:all
-rm fog_integration_test.config
+RUBYOPT="-r ./tools/fog_credentials" bundle exec rake integration:all
 
 bundle exec rake publish_gem

data/jenkins_integration_tests.sh

@@ -1,7 +1,4 @@
 #!/bin/bash -x
 set -e
 bundle install --path "${HOME}/bundles/${JOB_NAME}"
-./scripts/generate_fog_conf_file.sh
-export FOG_RC=fog_integration_test.config
-bundle exec rake integration:all
-rm fog_integration_test.config
+RUBYOPT="-r ./tools/fog_credentials" bundle exec rake integration:all

data/lib/vcloud/launcher/version.rb

@@ -1,5 +1,5 @@
 module Vcloud
   module Launcher
-    VERSION = '0.0.3'
+    VERSION = '0.0.4'
   end
 end

data/spec/integration/launcher/data/storage_profile.yaml.erb

@@ -5,20 +5,44 @@ vapps:
     catalog: <%= catalog %>
     catalog_item: <%= vapp_template %>
     vm:
+      hardware_config: &hardware_config
+        memory: 4096
+        cpu: 2
+      bootstrap: &bootstrap
+        script_path: <%= bootstrap_script %>
+        vars:
+          message: hello world
       storage_profile: <%= storage_profile %>
   - name: <%= vapp_name_2 %>
     vdc_name: <%= vdc_name_2 %>
     catalog: <%= catalog %>
     catalog_item: <%= vapp_template %>
     vm:
+      hardware_config:
+        <<: *hardware_config
+      bootstrap:
+        <<: *bootstrap
       storage_profile: <%= storage_profile %>
   - name: <%= vapp_name_3 %>
     vdc_name: <%= vdc_name_1 %>
     catalog: <%= catalog %>
     catalog_item: <%= vapp_template %>
+    vm:
+      hardware_config:
+        <<: *hardware_config
+      metadata:
+        storage_profile_test_vm: true
+      bootstrap:
+        <<: *bootstrap
   - name: <%= vapp_name_4 %>
     vdc_name: <%= vdc_name_1 %>
     catalog: <%= catalog %>
     catalog_item: <%= vapp_template %>
     vm:
+      hardware_config:
+        <<: *hardware_config
+      metadata:
+        storage_profile_test_vm: true
+      bootstrap:
+        <<: *bootstrap
       storage_profile: <%= nonsense_storage_profile %>

data/spec/integration/launcher/storage_profile_integration_spec.rb

@@ -50,11 +50,24 @@ describe Vcloud::Launcher::Launch do
         @vm_3[:StorageProfile][:href].should == @test_data[:default_storage_profile_href]
     end
 
+   it "when a storage profile is not specified, customize continues with other customizations" do
+        @vm_3_id = @vm_3[:href].split('/').last
+        @vm_3_metadata = Vcloud::Core::Vm.get_metadata @vm_3_id
+        @vm_3_metadata[:storage_profile_test_vm].should == true
+    end
+
     it "when a storage profile specified does not exist, vm uses the default" do
         @vm_4[:StorageProfile][:name].should == @test_data[:default_storage_profile_name]
         @vm_4[:StorageProfile][:href].should == @test_data[:default_storage_profile_href]
     end
 
+    # This is a bug - if it has failed customization it should let the user know
+    it "when storage profile specified doesn't exist, it errors and continues" do
+        @vm_4_id = @vm_4[:href].split('/').last
+        @vm_4_metadata = Vcloud::Core::Vm.get_metadata @vm_4_id
+        @vm_4_metadata[:storage_profile_test_vm].should be_nil
+    end
+
     after(:all) do
       unless ENV['VCLOUD_TOOLS_RSPEC_NO_DELETE_VAPP']
         File.delete @config_yaml

data/tools/fog_credentials.rb

@@ -0,0 +1,17 @@
+# Initialiser for getting vCloud credentials into Fog from Jenkins build
+# parameters, without needing to write them to disk. To be used with:
+#
+#     RUBYOPT="-r ./tools/fog_credentials" bundle exec integration
+#
+# Replace with FOG_VCLOUD_TOKEN support when we have a tool:
+#
+#     https://www.pivotaltracker.com/story/show/68989754
+#
+require 'bundler/setup'
+require 'fog'
+
+Fog.credentials = {
+  :vcloud_director_host     => ENV['API_HOST'],
+  :vcloud_director_username => ENV['API_USERNAME'],
+  :vcloud_director_password => ENV['API_PASSWORD'],
+}

data/vcloud-launcher.gemspec

@@ -21,7 +21,7 @@ Gem::Specification.new do |s|
   s.required_ruby_version = '>= 1.9.2'
 
   s.add_runtime_dependency 'methadone'
-  s.add_runtime_dependency 'vcloud-core', '>= 0.0.12'
+  s.add_runtime_dependency 'vcloud-core', '~> 0.0.12'
   s.add_development_dependency 'aruba', '~> 0.5.3'
   s.add_development_dependency 'cucumber', '~> 1.3.10'
   s.add_development_dependency 'gem_publisher', '1.2.0'

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: vcloud-launcher
 version: !ruby/object:Gem::Version
-  version: 0.0.3
+  version: 0.0.4
   prerelease: 
 platform: ruby
 authors:
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2014-04-22 00:00:00.000000000 Z
+date: 2014-05-01 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: methadone
@@ -32,7 +32,7 @@ dependencies:
   requirement: !ruby/object:Gem::Requirement
     none: false
     requirements:
-    - - ! '>='
+    - - ~>
       - !ruby/object:Gem::Version
         version: 0.0.12
   type: :runtime
@@ -40,7 +40,7 @@ dependencies:
   version_requirements: !ruby/object:Gem::Requirement
     none: false
     requirements:
-    - - ! '>='
+    - - ~>
       - !ruby/object:Gem::Version
         version: 0.0.12
 - !ruby/object:Gem::Dependency
@@ -155,7 +155,6 @@ files:
 - lib/vcloud/launcher/version.rb
 - lib/vcloud/launcher/vm_orchestrator.rb
 - scripts/basic.erb
-- scripts/generate_fog_conf_file.sh
 - spec/erb_helper.rb
 - spec/integration/launcher/data/basic_preamble_test.erb
 - spec/integration/launcher/data/happy_path.yaml.erb
@@ -168,6 +167,7 @@ files:
 - spec/vcloud/launcher/launch_spec.rb
 - spec/vcloud/launcher/vapp_orchestrator_spec.rb
 - spec/vcloud/launcher/vm_orchestrator_spec.rb
+- tools/fog_credentials.rb
 - vcloud-launcher.gemspec
 homepage: https://github.com/alphagov/vcloud-launcher
 licenses:
@@ -190,7 +190,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
       segments:
       - 0
-      hash: 8814923118188880
+      hash: -3318848198983750692
 requirements: []
 rubyforge_project: 
 rubygems_version: 1.8.23

data/scripts/generate_fog_conf_file.sh

@@ -1,6 +0,0 @@
-cat <<EOF >fog_integration_test.config
-default:
-    vcloud_director_username: '$API_USERNAME'
-    vcloud_director_password: '$API_PASSWORD'
-    vcloud_director_host: '$API_HOST'
-EOF