bosh-stemcell 1.3215.4.0 → 1.3232.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 0d48198e84b48412db7a00636f437cb4e3ada0c0
-  data.tar.gz: 23578cec445e59b0078e09b600197a5bf7d2398b
+  metadata.gz: 7de7caef97cfee142cb1f9d1cbef3144cad49894
+  data.tar.gz: 79b67b96758237d97c6afb243e990213b1ad9e19
 SHA512:
-  metadata.gz: 41f614d46b02623924a34d56b4b421e523b8f45045e3984fc0cc1918ab80465c3cf40754255361fe6b71580e278e706a4861e5f0b6a6a4e4b05429c95568a0ed
-  data.tar.gz: 74e983d1d49c7922deb54ace2d82a2e0edb404798a5b0e5dcd1a3c1b9e8e324fa21edbf8ce1c1fa226c9c3f98e72eafb0e913c787619299e2c12b6b53e61924e
+  metadata.gz: 14c029edcba45d41d9710b8a34a53b6cff19a084413f999e7580a34e396be6b1a4c5d05d160ca3de2a5cac2a38e376b27316ed11aac82f99a65b094cfc6de95c
+  data.tar.gz: b0b0e2c4b6c80cd3c7da72994ce7135947479ba109ba69daedb4de386f597743d1fb9c33efd00487daf7d5e15cf88b410c11564ac10e4f54f649f87c4bba81ec

data/README.md

@@ -2,55 +2,28 @@
 
 Tools for creating stemcells
 
-## Bringing up stemcell building VM
+## Choosing execution environment
 
-### Once-off manual steps:
+Stemcells can be built using either a local Docker container or a Vagrant VM on AWS.
 
-Note: Use US East (Northern Virginia) region when using AWS in following steps. AMI (Amazon Machine Image) to be used for the stemcell building VM is in the US East (Northern Virginia) region.
-
-0. Upload a keypair called "bosh" to AWS that you'll use to connect to the remote vm later
-0. Create "bosh-stemcell" security group on AWS to allow SSH access to the stemcell (once per AWS account)
-0. Install the vagrant plugins we use:
-
-        vagrant plugin install vagrant-aws
-
-### Bring up the vagrant stemcell building VM
-
-From a fresh copy of the bosh repo:
-
-    git submodule update --init --recursive
-
-If you use AWS VPC environment, run:
-
-    export BOSH_VAGRANT_KEY_PATH=PATH-TO-YOUR-SSH-KEY
-    export BOSH_AWS_ACCESS_KEY_ID=YOUR-AWS-ACCESS-KEY
-    export BOSH_AWS_SECRET_ACCESS_KEY=YOUR-AWS-SECRET-KEY
-    export BOSH_AWS_SECURITY_GROUP=YOUR-AWS-SECURITY-GROUP-ID
-    export BOSH_AWS_SUBNET_ID=YOUR-AWS-SUBNET-ID
-    cd bosh-stemcell
-    vagrant up remote --provider=aws
-
-(Note: BOSH\_AWS\_SECURITY\_GROUP should be security group id (e.g. "sg-b799b9dc"), instead of name "bosh-stemcell")
-
-## Updating source code on stemcell building VM
+To run a given command in Docker:
 
-With existing stemcell building VM run:
+    # run this command from the root of the bosh project
+    docker run -it -v $PWD:/bosh -w /bosh -u ubuntu --privileged bosh/os-image-stemcell-builder
 
-    export BOSH_AWS_ACCESS_KEY_ID=YOUR-AWS-ACCESS-KEY
-    export BOSH_AWS_SECRET_ACCESS_KEY=YOUR-AWS-SECRET-KEY
-    cd bosh-stemcell
-    vagrant provision remote
-
-## Configure your local ssh and scp to communicate with the stemcell building VM
+    # from within docker container
+    bundle install --local
+    <STEMCELL_COMMAND>
 
-Once the stemcell-building machine is up, run:
+To run a command in Vagrant:
 
-    vagrant ssh-config remote
-
-Then copy the resulting output into your `~/.ssh/config` file.
+    vagrant ssh -c '
+      cd /bosh
+      bundle install --local
+      <STEMCELL_COMMAND>
+    ' remote
 
-Once this has been done, you can ssh into the stemcell building machine with `ssh remote`
-and you can copy files to and from it using `scp localfile remote:/path/to/destination`
+Jump to [Vagrant Setup](#vagrant-setup) for instructions on configuring the Vagrant box.
 
 ## Build an OS image
 
@@ -58,10 +31,7 @@ An OS image is a tarball that contains a snapshot of an entire OS filesystem tha
 
 If you have changes that will require new OS image you need to build one. A stemcell with a custom OS image can be built using the stemcell-building VM described above.
 
-    vagrant ssh -c '
-      cd /bosh
-      bundle exec rake stemcell:build_os_image[ubuntu,trusty,/tmp/ubuntu_base_image.tgz]
-    ' remote
+    bundle exec rake stemcell:build_os_image[ubuntu,trusty,$PWD/tmp/ubuntu_base_image.tgz]
 
 The arguments to `stemcell:build_os_image` are:
 
@@ -87,8 +57,7 @@ There are a few extra steps you need to do before building a RHEL OS image:
 
 5. On the stemcell building machine, run the stemcell building rake task:
 
-        $ cd /bosh
-        $ bundle exec rake stemcell:build_os_image[rhel,7,/tmp/rhel_7_base_image.tgz]
+        $ bundle exec rake stemcell:build_os_image[rhel,7,$PWD/tmp/rhel_7_base_image.tgz]
 
 See below [Building the stemcell with local OS image](#building-the-stemcell-with-local-os-image) on how to build stemcell with the new OS image.
 
@@ -105,8 +74,7 @@ There are a few extra steps you need to do before building a PhotonOS image:
 
 4. On the stemcell building machine, run the stemcell building rake task:
 
-        $ cd /bosh
-        $ bundle exec rake stemcell:build_os_image[photonos,TP2,/tmp/photon_TP2_base_image.tgz]
+        $ bundle exec rake stemcell:build_os_image[photonos,TP2,$PWD/tmp/photon_TP2_base_image.tgz]
 
 See below [Building the stemcell with local OS image](#building-the-stemcell-with-local-os-image) on how to build stemcell with the new OS image.
 
@@ -116,18 +84,12 @@ See below [Building the stemcell with local OS image](#building-the-stemcell-wit
 
 Substitute *\<current_build\>* with the current build number, which can be found by looking at [bosh.io/stemcells](https://bosh.io/stemcells). Note that the last two arguments to the rake command are the S3 bucket and key of the OS image to use (i.e. in the example below, the .tgz will be downloaded from [http://bosh-os-images.s3.amazonaws.com/bosh-centos-7-os-image.tgz](http://bosh-os-images.s3.amazonaws.com/bosh-centos-7-os-image.tgz)). More info at OS\_IMAGES.
 
-    vagrant ssh -c '
-      cd /bosh
-      CANDIDATE_BUILD_NUMBER=<current_build> bundle exec rake stemcell:build[vsphere,esxi,centos,7,go,bosh-os-images,bosh-centos-7-os-image.tgz]
-    ' remote
-
+    CANDIDATE_BUILD_NUMBER=<current_build> bundle exec rake \
+      stemcell:build[vsphere,esxi,centos,7,go,bosh-os-images,bosh-centos-7-os-image.tgz]
 
 ### Building the stemcell with local OS image
 
-    vagrant ssh -c '
-      cd /bosh
-      bundle exec rake stemcell:build_with_local_os_image[aws,xen,ubuntu,trusty,go,/tmp/ubuntu_base_image.tgz]
-    ' remote
+    bundle exec rake stemcell:build_with_local_os_image[aws,xen,ubuntu,trusty,go,$PWD/tmp/ubuntu_base_image.tgz]
 
 
 Public OS images can be obtained here:
@@ -137,14 +99,13 @@ Public OS images can be obtained here:
 
 ### Building light stemcell
 
+**Warning:** You must use Vagrant on AWS to build light stemcells.
+
 AWS stemcells can be shipped in light format which includes a reference to a public AMI. This speeds up the process of uploading the stemcell to AWS. To build a light stemcell:
 
-    vagrant ssh -c '
-      cd /bosh
-      export BOSH_AWS_ACCESS_KEY_ID=YOUR-AWS-ACCESS-KEY
-      export BOSH_AWS_SECRET_ACCESS_KEY=YOUR-AWS-SECRET-KEY
-      bundle exec rake stemcell:build_light[/tmp/bosh-stemcell.tgz,hvm]
-    ' remote
+    export BOSH_AWS_ACCESS_KEY_ID=YOUR-AWS-ACCESS-KEY
+    export BOSH_AWS_SECRET_ACCESS_KEY=YOUR-AWS-SECRET-KEY
+    bundle exec rake stemcell:build_light[$PWD/tmp/bosh-stemcell.tgz,hvm]
 
 To build for specific region specify `BOSH_AWS_REGION` environment variable.
 
@@ -167,5 +128,56 @@ If you find yourself debugging any of the above processes, here is what you need
    For example:
 
    ```
-   bundle exec rake stemcell:build_os_image[ubuntu,trusty,/tmp/ubuntu_base_image.tgz] resume_from=rsyslog_config
+   bundle exec rake stemcell:build_os_image[ubuntu,trusty,$PWD/tmp/ubuntu_base_image.tgz] resume_from=rsyslog_config
    ```
+
+## <a name="vagrant-setup"></a>Vagrant Setup
+
+**Warning:** You cannot building a stemcell using Vagrant on Virtualbox.
+
+### Bring up the vagrant stemcell building VM
+
+Note: Use US East (Northern Virginia) region when using AWS in following steps. AMI (Amazon Machine Image) to be used for the stemcell building VM is in the US East (Northern Virginia) region.
+
+0. Upload a keypair called "bosh" to AWS that you'll use to connect to the remote vm later
+0. Create "bosh-stemcell" security group on AWS to allow SSH access to the stemcell (once per AWS account)
+0. Install the vagrant plugins we use:
+
+        vagrant plugin install vagrant-aws
+
+
+From a fresh copy of the bosh repo:
+
+    git submodule update --init --recursive
+
+If you use AWS VPC environment, run:
+
+    export BOSH_VAGRANT_KEY_PATH=PATH-TO-YOUR-SSH-KEY
+    export BOSH_AWS_ACCESS_KEY_ID=YOUR-AWS-ACCESS-KEY
+    export BOSH_AWS_SECRET_ACCESS_KEY=YOUR-AWS-SECRET-KEY
+    export BOSH_AWS_SECURITY_GROUP=YOUR-AWS-SECURITY-GROUP-ID
+    export BOSH_AWS_SUBNET_ID=YOUR-AWS-SUBNET-ID
+    cd bosh-stemcell
+    vagrant up remote --provider=aws
+
+(Note: BOSH\_AWS\_SECURITY\_GROUP should be security group id (e.g. "sg-b799b9dc"), instead of name "bosh-stemcell")
+
+### Updating source code on stemcell building VM
+
+With existing stemcell building VM run:
+
+    export BOSH_AWS_ACCESS_KEY_ID=YOUR-AWS-ACCESS-KEY
+    export BOSH_AWS_SECRET_ACCESS_KEY=YOUR-AWS-SECRET-KEY
+    cd bosh-stemcell
+    vagrant provision remote
+
+### Configure your local ssh and scp to communicate with the stemcell building VM
+
+Once the stemcell-building machine is up, run:
+
+    vagrant ssh-config remote
+
+Then copy the resulting output into your `~/.ssh/config` file.
+
+Once this has been done, you can ssh into the stemcell building machine with `ssh remote`
+and you can copy files to and from it using `scp localfile remote:/path/to/destination`

data/lib/bosh/stemcell/archive.rb

@@ -59,7 +59,7 @@ module Bosh::Stemcell
     end
 
     def validate_stemcell
-      raise "Cannot find file `#{path}'" unless File.exists?(path)
+      raise "Cannot find file '#{path}'" unless File.exists?(path)
     end
   end
 end

data/lib/bosh/stemcell/build_environment.rb

@@ -172,6 +172,8 @@ module Bosh::Stemcell
         ' --tag ~exclude_on_azure'
       when 'softlayer'
         ' --tag ~exclude_on_softlayer'
+      when 'google'
+        ' --tag ~exclude_on_google'
       else
         ''
       end,

data/lib/bosh/stemcell/infrastructure.rb

@@ -6,6 +6,8 @@ module Bosh::Stemcell
           OpenStack.new
         when 'aws'
           Aws.new
+        when 'google'
+          Google.new
         when 'vsphere'
           Vsphere.new
         when 'warden'
@@ -101,6 +103,16 @@ module Bosh::Stemcell
       end
     end
 
+    class Google < Base
+      def initialize
+        super(name: 'google', hypervisor: 'kvm', default_disk_size: 3072, disk_formats: ['rawdisk'])
+      end
+
+      def additional_cloud_properties
+        {'root_device_name' => '/dev/sda1'}
+      end
+    end
+
     class Warden < Base
       def initialize
         super(name: 'warden', hypervisor: 'boshlite', default_disk_size: 2048, disk_formats: ['files'])

data/lib/bosh/stemcell/stage_collection.rb

@@ -44,6 +44,8 @@ module Bosh::Stemcell
       stages = case infrastructure
       when Infrastructure::Aws then
         aws_stages
+      when Infrastructure::Google then
+        google_stages
       when Infrastructure::OpenStack then
         openstack_stages
       when Infrastructure::Vsphere then
@@ -65,6 +67,8 @@ module Bosh::Stemcell
       case disk_format
         when 'raw' then
           raw_package_stages
+        when 'rawdisk' then
+          rawdisk_package_stages
         when 'qcow2' then
           qcow2_package_stages
         when 'ovf' then
@@ -108,7 +112,6 @@ module Bosh::Stemcell
       [
         :system_network,
         :system_open_vm_tools,
-        :password_policies,
         :system_vsphere_cdrom,
         :system_parameters,
         :bosh_clean,
@@ -136,6 +139,21 @@ module Bosh::Stemcell
       ]
     end
 
+    def google_stages
+      [
+        :system_network,
+        :system_google_modules,
+        :system_google_packages,
+        :system_parameters,
+        :bosh_clean,
+        :bosh_harden,
+        :bosh_google_agent_settings,
+        :bosh_clean_ssh,
+        :image_create,
+        :image_install_grub,
+      ]
+    end
+
     def warden_stages
       [
         :system_parameters,
@@ -145,6 +163,7 @@ module Bosh::Stemcell
         :bosh_enable_password_authentication,
         :bosh_clean_ssh,
         :image_create,
+        :image_install_grub,
       ]
     end
 
@@ -195,11 +214,14 @@ module Bosh::Stemcell
         :system_kernel_modules,
         :system_ixgbevf,
         bosh_steps,
+        :password_policies,
+        :tty_config,
         :rsyslog_config,
         :delay_monit_start,
         :system_grub,
         :cron_config,
-        :escape_ctrl_alt_del
+        :escape_ctrl_alt_del,
+        :bosh_audit
       ].flatten
     end
 
@@ -234,12 +256,15 @@ module Bosh::Stemcell
         :system_kernel_modules,
         :system_ixgbevf,
         bosh_steps,
+        :password_policies,
+        :tty_config,
         :rsyslog_config,
         :delay_monit_start,
         :system_grub,
         :vim_tiny,
         :cron_config,
         :escape_ctrl_alt_del,
+        :bosh_audit
       ].flatten.reject{ |s| Bosh::Stemcell::Arch.ppc64le? and s ==  :system_ixgbevf }
     end
 
@@ -263,8 +288,7 @@ module Bosh::Stemcell
         :bosh_monit,
         :bosh_ntpdate,
         :bosh_sudoers,
-        :password_policies,
-      ]
+      ].flatten
     end
 
     def raw_package_stages
@@ -273,6 +297,12 @@ module Bosh::Stemcell
       ]
     end
 
+    def rawdisk_package_stages
+      [
+        :prepare_rawdisk_image_stemcell,
+      ]
+    end
+
     def qcow2_package_stages
       [
         :prepare_qcow2_image_stemcell,

data/lib/bosh/stemcell/stemcell_packager.rb

@@ -48,7 +48,7 @@ module Bosh
       def manifest_cloud_properties(disk_format, infrastructure, stemcell_name)
         architecture = 'x86_64'
         if Bosh::Stemcell::Arch.ppc64le?
-          architecture = 'ppc64le'
+          architecture = 'ppc64'
         end
 
         {

data/lib/bosh/stemcell/version.rb

@@ -1,5 +1,5 @@
 module Bosh
   module Stemcell
-    VERSION = '1.3215.4.0'
+    VERSION = '1.3232.0'
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: bosh-stemcell
 version: !ruby/object:Gem::Version
-  version: 1.3215.4.0
+  version: 1.3232.0
 platform: ruby
 authors:
 - Pivotal
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2016-04-14 00:00:00.000000000 Z
+date: 2016-04-25 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bosh_aws_cpi
@@ -30,14 +30,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 1.3215.4.0
+        version: 1.3232.0
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 1.3215.4.0
+        version: 1.3232.0
 - !ruby/object:Gem::Dependency
   name: fakefs
   requirement: !ruby/object:Gem::Requirement