bosh-cloudfoundry 0.5.1 → 0.6.0

data/.travis.yml

@@ -1,6 +1,5 @@
 language: ruby
 rvm:
   - 1.9.3
-  - jruby-19mode # JRuby in 1.9 mode
   - rbx-19mode
-  - ruby-head
+  # - ruby-head - generates "Cannot find Syck parser for YAML"

data/ChangeLog.md

@@ -1,5 +1,17 @@
 # Change Log
 
+## v0.6
+
+There is now a write up of [the concepts](https://github.com/StarkAndWayne/bosh-cloudfoundry/blob/master/docs/concept.md) and constructs being deployed from Anders Sveen!
+
+Tests are running on Travis CI. We have green builds!
+
+Automatically uploading base stemcell 0.7.0; rather than the very new 1.5.0-pre2 which only work with the also very new microbosh 1.5.0-pre2.
+
+Faster when using a final release because it no longer pulls down all the submodules; and some better code for handling the switch between a development and final release; but...
+
+Still need to create development releases from cf-release's master branch. Hopefully all the required patches will be in cf-release's 130 release coming soon!
+
 ## v0.5
 
 Gerrit is dead. Long live gerrit. This release is for everyone who is getting started or wants to upgrade. Gerrit is dead.

data/README.md

@@ -4,6 +4,8 @@ You want Cloud Foundry? You can now create, scale, upgrade and patch one or more
 
 Currently supports AWS and OpenStack only. vSphere and vCloud support will require someone to tell me that they really want it.
 
+[![Build Status](https://travis-ci.org/StarkAndWayne/bosh-cloudfoundry.png?branch=master)](https://travis-ci.org/StarkAndWayne/bosh-cloudfoundry) [![Code Climate](https://codeclimate.com/github/StarkAndWayne/bosh-cloudfoundry.png)](https://codeclimate.com/github/StarkAndWayne/bosh-cloudfoundry)
+
 ## Zero to Cloud Foundry in a few hours
 
 If you have an AWS or OpenStack account then you can deploy a running Cloud Foundry VM by following this very simple tutorial. It first uses the very powerful [bosh-bootstrap](https://github.com/StarkAndWayne/bosh-bootstrap) tool to deploy a BOSH, and then using this project (a plugin for the BOSH CLI) it will do everything necessary to deploy Cloud Foundry via your shiny new, empty BOSH.

data/Rakefile

@@ -18,7 +18,7 @@ if defined?(RSpec)
     desc "Run Unit Tests"
     unit_rspec_task = RSpec::Core::RakeTask.new(:unit) do |t|
       t.pattern = "spec/unit/**/*_spec.rb"
-      t.rspec_opts = %w(--format progress --color -d)
+      t.rspec_opts = %w(--format progress --color)
     end
 
     desc "Run Integration Tests"
@@ -31,3 +31,5 @@ if defined?(RSpec)
   desc "Install dependencies and run tests"
   task :spec => %w(spec:unit spec:functional)
 end
+
+task :default => :spec

data/docs/concept.md

@@ -0,0 +1,141 @@
+If you're new to BOSH, Cloud Foundry and PaaS in general you might find this as confusing as I did. So I'll try to explain some of the concepts.
+
+# The parts
+There are several parts in action here. Let me take you for a quick tour.
+
+## Cloud Foundry
+
+[Cloud Foundry](http://cloudfoundry.org) is a Open Source PaaS solution. It is available as a public cloud at [http://cloudfoundry.com](http://cloudfoundry.com) operated by VMWare, but it is also a solution you can bring in-house and install on your own servers, or even operate on Amazon like [AppFog](http://appfog.com) does.
+
+* __Cloud Controller__ is the main component in Cloud Foundry and the server you point your vmc command to. It can be split up into separate VMs, but this bootstrap keeps it all on one instance.
+* __DEA__ is the compute instance. In the initial setup the Cloud Controller is also a DEA. But you can basically view it as VMs that will handle and serve your applications.
+* __Service__ is the concept Cloud Foundry uses to describe added functionality to your application like Postgres or MongoDB.
+
+
+## BOSH 
+
+[BOSH](https://github.com/cloudfoundry/bosh) is a cloud oriented tool chain for releasing, deployment and lifecycle management of applications. It abstracts away the underlying IaaS solution (AWS, OpenStack, VMWare) and performs requests for new virtual machines etc. It also handles the distribution of "Jobs" to the virtual machines in the BOSH setup. In some ways you can say that it eliminates the need for Puppet/Chef, but it uses some different concepts. It also covers more, like the provisioning of virtual machines.
+
+* __Job__ is BOSH's concept of grouping functionality together. In the normal instance you can think of a Job as a composition of installed software and configuration that a virtual machine will have. So two virtual machines with the same Job attached will be identical.
+* __Template__ is BOSH's concept for defining a software setup. CloudController is one such template, and dea is another one.
+* __Stemcell__ is BOSH's concept for a virtual machine image. For Amazon it equals and AMI. It is a template that is booted and becoms an instance of an virtual machine.
+* __Resource Pool__ is basically a collection of virtual machines, that you can reference/assign jobs to. The have the same stemcell and configuration (eg. AWS size).
+* __Compilation instance__ is instances that BOSH uses when deploying a new part. At first it is used to compile everything that is needed for VCAP and DEA. If you add a template like Postgres you'll see that it compiles the components used for that. I think this is something that resembles rpm or deb packages, and that they get compiled on your Stemcells to ensure compatibility when running on your cloud.
+
+## Inception VM
+Inception VM is a Virtual Machine used for bootstrapping BOSH.
+
+# Building a runnable Cloud Foundry
+This Bootstrap takes you throug the steps needed to have Cloud Foundry running and serving requests. It launches BOSH which then is used to deploy Cloud Foundry. BOSH is the only officially supported way of deploying Cloud Foundry, even though Chef scripts is maintained by the community.
+
+The way it operates is illustrated in the figure below:
+
+![BOSH-CF initial deploy](BOSH-cf-initial-create.png)
+
+1. The bosh-bootstrap gem on "My Computer" creates an Inception virtual machine from an Ubuntu image.
+2. The Inception VM creates the BOSH controller
+3. The BOSH controller creates virtual machines for Cloud Foundry
+4. The BOSH controller deploys Jobs to the separate instances
+
+In the initial configuration the Cloud Controller acts as both a controller and a compute instance where your applications run.
+
+# Scaling
+Now if the above description was crystal clear, you'll know where the scaling happens. ;) It happens to the far right. The figure below shows how everything looks when I've added a couple of compute instances.
+
+![BOSH-CF scaled deploy](BOSH-cf-scaled.png)
+
+In the case above we have three resource pools. They are called core, DEA and postgres. This is that standard way that the BOSH cf plugin does it. The following has been done here:
+
+* ```bosh cf change deas 3``` creates a pool with 3 virtual machines and assigns the DEA job to them. It also removes the DEA template from the core job.
+* ```bosh cf add service postgresql 2``` creates a pool with 2 virtual machines and assigns the postgresql job to them.
+* The dotted boxes above are compile instances that BOSH will create temporarily to compile the necessary components. See the BOSH description above.
+
+With BOSH it is possible to allocate the postgres templates on the already existing DEA instances and thus save costs on Amazon. It all comes down to what kind of scale you need, and also separation from load issues.
+
+It is the responsibility of the separate application to handle clustering and communication between the instances.
+
+## Implementation
+
+```bosh cf change deas 1``` will add a resource pool and set up jobs. This is from the cf plugin to BOSH.
+
+It adds a resource pool:
+
+```
+- name: dea
+  network: default
+  size: 2
+  stemcell:
+    name: bosh-stemcell
+    version: 0.7.0
+  cloud_properties:
+    region: eu-west-1
+    availability_zone: eu-west-1b
+    instance_type: m1.large
+```
+
+Size 2 specifies to launch two virtual machines for this pool.
+
+It then sets up a job:
+
+```
+- name: dea
+  template:
+  - dea
+  instances: 2
+  resource_pool: dea
+  networks:
+  - name: default
+    default:
+    - dns
+    - gateway
+```
+
+The instances setting tells it to use two instances in the dea resource pool. For our case this matches the size of the pool exactly.    
+
+After the changes has been done a ```bosh deploy``` will perform the needed changes to your BOSH setup.
+
+### Using your Cloud Controller as a compute instance
+
+If you take a look at the job named "core" before you do the adding of DEAs you will see that it has a template named dea. If you leave that in the list your Cloud Controller will also be a compute instance serving deployed applications.
+
+# Tips
+
+* Provision an Elastic IP from Amazon as early as possible and assign a DNS entry with wildcard. Example: *.cf.mydomain.com . It will take some time for the DNS to propagate (actually your old *.mydomain.com record to time out if you have one).
+* If you're deploying to a different region than us-east-1 check out [this bug.](https://github.com/StarkAndWayne/bosh-cloudfoundry/issues/100) Make the changes to the configuration just before you do ```bosh deploy```.
+* If you want to scale out by adding more instances you need to tweak the FW rules. Check [this bug.](https://github.com/StarkAndWayne/bosh-cloudfoundry/issues/112)
+* You might need to reduce the number of compile instances to avoid Amazon API errors. Check [this bug.](https://github.com/StarkAndWayne/bosh-cloudfoundry/issues/111)
+* If you're having problems with ```vmc push``` check out [this bug.](https://github.com/StarkAndWayne/bosh-cloudfoundry/issues/49)
+* Once you're up and running [test your setup with a simple Sinatra application](http://docs.cloudfoundry.com/tools/vmc/installing-vmc.html#creating-a-simple-sinatra-application).
+
+# Some resources
+
+Use these for some pointers for in depth understanding. The rest of this document tries to explain the basic concepts without getting into the really deep details.
+
+## Documentation
+
+* [Cloud Foundry documentation](http://cloudfoundry.github.com/)
+* [Cloud Foundry public cloud (commercial version) docs](http://docs.cloudfoundry.com/getting-started.html)
+* [Cloud Foundry community wiki](https://github.com/mrdavidlaing/cf-docs-contrib/wiki)
+
+## Community
+
+* [BOSH users Mailinglist](https://groups.google.com/a/cloudfoundry.org/forum/#!forum/bosh-users)
+* [Cloud Foundry Mailinglist](https://groups.google.com/a/cloudfoundry.org/forum/#!forum/vcap-dev)
+
+## Source code
+
+* [Cloud Foundry release](https://github.com/cloudfoundry/cf-release)
+* [Cloud Foundry jobs (within the release repo)](https://github.com/cloudfoundry/cf-release/tree/master/jobs)
+* [Cloud Foundry VCAP (core component)](https://github.com/cloudfoundry/vcap)
+
+# TODO/Questions
+
+* Is the BOSH controller a MicroBosh? Another name?
+* What is a microbosh?
+* Is the BOSH controller a part of BOSH, or something that just controls it?
+* Does the core Cloud Foundry instances scale out?
+* How do you enable HA for the core?
+* You run BOSH commands on the Inception VM, not BOSH Controller. Is it meant to be run on your computer? Is network the reason it isn't?
+* Add some nice to know commands
+* How can we lock this to a certain "release"?
+* Can we describe how the source is fetched and what happens to it?

data/lib/bosh-cloudfoundry/bosh_release_manager.rb

@@ -117,6 +117,10 @@ module Bosh::CloudFoundry::BoshReleaseManager
         end
       end
     end
+  end
+
+  # when creating a dev release, need to pull down submodules
+  def prepare_cf_release_for_dev_release
     chdir(cf_release_branch_dir) do
       say "Rewriting all git:// & git@ to https:// ..."
       # Snippet written by Mike Reeves <swampfoxmr@gmail.com> on bosh-users mailing list
@@ -139,12 +143,14 @@ module Bosh::CloudFoundry::BoshReleaseManager
   def switch_to_development_release
     system_config.release_name = default_dev_release_name(cf_release_branch)
     system_config.release_version = "latest"
+    system_config.release_type = "dev"
     system_config.save
   end
 
   def switch_to_final_release
     system_config.release_name = default_release_name
     system_config.release_version = "latest"
+    system_config.release_type = "final"
     system_config.save
   end
 end

data/lib/bosh-cloudfoundry/config/dea_config.rb

@@ -61,7 +61,7 @@ class Bosh::CloudFoundry::Config::DeaConfig
   #   size: 2
   #   stemcell: 
   #     name: bosh-stemcell
-  #     version: 0.6.4
+  #     version: 0.7.0
   #   cloud_properties: 
   #     instance_type: m1.xlarge
   def add_resource_pools_to_manifest(manifest)

data/lib/bosh-cloudfoundry/config/postgresql_service_config.rb

@@ -74,7 +74,7 @@ class Bosh::CloudFoundry::Config::PostgresqlServiceConfig
   #   size: 2
   #   stemcell: 
   #     name: bosh-stemcell
-  #     version: 0.6.4
+  #     version: 0.7.0
   #   cloud_properties: 
   #     instance_type: m1.xlarge
   def add_resource_pools_to_manifest(manifest)

data/lib/bosh-cloudfoundry/config/redis_service_config.rb

@@ -75,7 +75,7 @@ class Bosh::CloudFoundry::Config::RedisServiceConfig
   #   size: 2
   #   stemcell: 
   #     name: bosh-stemcell
-  #     version: 0.6.4
+  #     version: 0.7.0
   #   cloud_properties: 
   #     instance_type: m1.xlarge
   def add_resource_pools_to_manifest(manifest)

data/lib/bosh-cloudfoundry/config/system_config.rb

@@ -30,6 +30,7 @@ class Bosh::CloudFoundry::Config::SystemConfig < Bosh::Cli::Config
     :cf_release_branch,     # e.g. staging
     :cf_release_branch_dir, # e.g. /var/vcap/store/releases/cf-release/staging
     :release_name,     # e.g. 'appcloud'
+    :release_type,     # either 'final' or 'dev'
     :release_version,  # e.g. 'latest'
     :gerrit_changes,   # e.g. ['84/13084/4', '37/13137/4']
     :stemcell_name,    # e.g. 'bosh-stemcell'

data/lib/bosh-cloudfoundry/config_options.rb

@@ -28,7 +28,7 @@ module Bosh::CloudFoundry::ConfigOptions
   def common_config
     @common_config ||= begin
       config_file = options[:common_config] || DEFAULT_CONFIG_PATH
-      common_config = Bosh::CloudFoundry::Config:: CommonConfig.new(config_file)
+      common_config = Bosh::CloudFoundry::Config::CommonConfig.new(config_file)
       common_config.bosh_git_repo ||= DEFAULT_BOSH_GIT_REPO
       common_config.save
       common_config
@@ -245,6 +245,16 @@ module Bosh::CloudFoundry::ConfigOptions
     end
   end
 
+  # using/uploading a dev release; not a final release
+  def dev_release_type?
+    system_config.release_type == "dev"
+  end
+
+  # using/uploading a final release; not a dev release
+  def final_release_type?
+    !dev_release_type?
+  end
+
   def system_initialized?
     system_config.system_initialized
   end
@@ -384,7 +394,7 @@ module Bosh::CloudFoundry::ConfigOptions
     @bosh_stemcell_versions ||= begin
       # [{"name"=>"bosh-stemcell", "version"=>"0.6.7", "cid"=>"ami-9730bffe"}]
       stemcells = director.list_stemcells
-      stemcells.select! {|s| s["name"] == stemcell_name}
+      stemcells = stemcells.select {|s| s["name"] == stemcell_name}
       stemcells.map { |rel| rel["version"] }.sort { |v1, v2|
         version_cmp(v1, v2)
       }

data/lib/bosh-cloudfoundry/system_deployment_manifest_renderer.rb

@@ -274,6 +274,9 @@ class Bosh::CloudFoundry::SystemDeploymentManifestRenderer
          "port"=>8100,
          "catalina_opts"=>"-Xmx128m -Xms30m -XX:MaxPermSize=128m",
          "no_ssl"=>true,
+         "client"=>{
+           "autoapprove"=>[]
+         },
          "scim"=>{}},
        "uaadb"=>
         {"address"=>core_ip,

data/lib/bosh-cloudfoundry/version.rb

@@ -1,5 +1,5 @@
 module Bosh
   module Cloudfoundry
-    VERSION = "0.5.1"
+    VERSION = "0.6.0"
   end
 end

data/lib/bosh/cli/commands/cf.rb

@@ -38,10 +38,6 @@ module Bosh::Cli::Command
     option "--admin-emails email1,email2", Array, "Admin email accounts in created CloudFoundry"
     option "--skip-validations", "Skip all validations"
     def prepare_system(name=nil)
-      # FUTURE remove this when cf-release has a final release with
-      # all patches in https://github.com/StarkAndWayne/bosh-cloudfoundry/issues/42
-      options[:dev] = true
-
       setup_system_dir(name)
       confirm_or_prompt_all_defaults
       confirm_or_prompt_for_system_requirements
@@ -115,7 +111,8 @@ module Bosh::Cli::Command
     usage "cf upload stemcell"
     desc "download/create stemcell & upload to BOSH"
     # option "--stable", "Use latest stemcell; possibly not tagged stable"
-    option "--latest", "Use latest stemcell; possibly not tagged stable [default]"
+    option "--latest", "Use latest stemcell; possibly not tagged stable"
+    option "--version VERSION", "Use base stemcell with specific version [default: 0.7.0]"
     option "--custom", "Create custom stemcell from BOSH git source"
     def upload_stemcell
       stemcell_type = "stable" if options[:stable]
@@ -126,20 +123,27 @@ module Bosh::Cli::Command
 
     usage "cf upload release"
     desc "fetch & upload latest public cloudfoundry release to BOSH"
-    option "--branch branch", String, "Create development release from branch of cf-release [default: staging]"
-    option "--final", "Upload latest final release from very latest cf-release commits"
+    option "--branch branch", String, "Create development release from branch of cf-release"
+    option "--final", "Upload latest final release from very latest cf-release commits [default]"
     def upload_release
-      if options.delete(:final)
+      if new_branch = options.delete(:branch)
+        switch_to_development_release
+      elsif options.delete(:final)
+        switch_to_final_release
+      end
+      # TODO - default to final release when appcloud-130.yml is released
+      # switch_to_final_release unless system_config.release_type
+      switch_to_development_release unless system_config.release_type
+        
+      if dev_release_type?
+        new_branch ||= "master"
+        set_cf_release_branch(new_branch)
         clone_or_update_cf_release
-        upload_final_release
+        prepare_cf_release_for_dev_release
+        create_and_upload_dev_release
       else
-        # FUTURE once all patches from https://github.com/StarkAndWayne/bosh-cloudfoundry/issues/42
-        # are merged into cf-release, then no more gerrit merging required
-        if new_branch = options.delete(:branch)
-          set_cf_release_branch(new_branch)
-        end
         clone_or_update_cf_release
-        create_and_upload_dev_release
+        upload_final_release
       end
     end
 
@@ -277,7 +281,16 @@ module Bosh::Cli::Command
     # already uploaded to BOSH, else
     # proceeds to upload the release
     def confirm_or_upload_release
-      switch_to_development_release if options.delete(:edge) || options.delete(:custom) || options.delete(:dev)
+      # if flags overriding the current final/dev
+      if options.delete(:edge) || options.delete(:custom) || options.delete(:dev)
+        switch_to_development_release
+      elsif options.delete(:final)
+        switch_to_final_release
+      end
+      # TODO - default to final release when appcloud-130.yml is released
+      # switch_to_final_release unless system_config.release_type
+      switch_to_development_release unless system_config.release_type
+
       say "Using BOSH release name #{release_name_version} (#{effective_release_version})".green
       unless bosh_release_names.include?(release_name)
         say "BOSH does not contain release #{release_name.green}, uploading...".yellow
@@ -341,7 +354,7 @@ module Bosh::Cli::Command
       end
     end
 
-    # Largest version number BOSH stemcell ("bosh-stemcell")
+    # Largest version number BOSH stemcell ("bosh-stemcell") uploaded to BOSH
     # @return [String] version number, e.g. "0.6.7"
     def latest_bosh_stemcell_version
       @latest_bosh_stemcell_version ||= begin
@@ -427,12 +440,14 @@ module Bosh::Cli::Command
     # | bosh-stemcell-0.5.2.tgz                 | vsphere                |
     # | bosh-stemcell-aws-0.6.4.tgz             | aws, stable            |
     # | bosh-stemcell-aws-0.6.7.tgz             | aws                    |
+    #
+    # Ignores any stemcells with "pre" in their name
     def bosh_stemcell_name(stemcell_type)
       tags = [bosh_provider]
       tags << "stable" if stemcell_type == "stable" unless openstack?
       bosh_stemcells_cmd = "bosh public stemcells --tags #{tags.join(',')}"
       say "Locating bosh stemcell, running '#{bosh_stemcells_cmd}'..."
-      `#{bosh_stemcells_cmd} | grep ' bosh-stemcell-' | awk '{ print $2 }' | sort -r | head -n 1`.strip
+      `#{bosh_stemcells_cmd} | grep ' bosh-stemcell-' | grep -v pre | awk '{ print $2 }' | sort -r | head -n 1`.strip
     end
 
     def download_stemcell(stemcell_name)
@@ -535,7 +550,7 @@ module Bosh::Cli::Command
       common_password
       security_group
 
-      set_cf_release_branch("staging")
+      set_cf_release_branch("master")
     end
 
     # Renders the +SystemConfig+ model (+system_config+) into the system's

data/spec/assets/deployments/aws-core-1-m1.small-free-redis.yml

@@ -33,7 +33,7 @@ resource_pools:
   size: 1
   stemcell: 
     name: bosh-stemcell
-    version: 0.6.4
+    version: 0.7.0
   cloud_properties: 
     instance_type: m1.small
   persistent_disk: 16192
@@ -42,7 +42,7 @@ resource_pools:
   size: 1
   stemcell: 
     name: bosh-stemcell
-    version: 0.6.4
+    version: 0.7.0
   cloud_properties: 
     instance_type: m1.small
   persistent_disk: 16192
@@ -178,6 +178,9 @@ properties:
     port: 8100
     catalina_opts: -Xmx128m -Xms30m -XX:MaxPermSize=128m
     no_ssl: true
+    client: 
+      autoapprove: []
+
     scim: {}
 
   uaadb: 

data/spec/assets/deployments/aws-core-1-m1.xlarge-free-postgresql-2-m1.small-free-postgresql.yml

@@ -33,7 +33,7 @@ resource_pools:
   size: 1
   stemcell: 
     name: bosh-stemcell
-    version: 0.6.4
+    version: 0.7.0
   cloud_properties: 
     instance_type: m1.small
   persistent_disk: 16192
@@ -42,7 +42,7 @@ resource_pools:
   size: 1
   stemcell: 
     name: bosh-stemcell
-    version: 0.6.4
+    version: 0.7.0
   cloud_properties: 
     instance_type: m1.xlarge
   persistent_disk: 16192
@@ -51,7 +51,7 @@ resource_pools:
   size: 2
   stemcell: 
     name: bosh-stemcell
-    version: 0.6.4
+    version: 0.7.0
   cloud_properties: 
     instance_type: m1.small
   persistent_disk: 16192
@@ -198,6 +198,9 @@ properties:
     port: 8100
     catalina_opts: -Xmx128m -Xms30m -XX:MaxPermSize=128m
     no_ssl: true
+    client: 
+      autoapprove: []
+
     scim: {}
 
   uaadb: 

data/spec/assets/deployments/aws-core-2-m1.xlarge-dea.yml

@@ -33,7 +33,7 @@ resource_pools:
   size: 1
   stemcell: 
     name: bosh-stemcell
-    version: 0.6.4
+    version: 0.7.0
   cloud_properties: 
     instance_type: m1.small
   persistent_disk: 16192
@@ -42,7 +42,7 @@ resource_pools:
   size: 2
   stemcell: 
     name: bosh-stemcell
-    version: 0.6.4
+    version: 0.7.0
   cloud_properties: 
     instance_type: m1.xlarge
 jobs: 
@@ -174,6 +174,9 @@ properties:
     port: 8100
     catalina_opts: -Xmx128m -Xms30m -XX:MaxPermSize=128m
     no_ssl: true
+    client: 
+      autoapprove: []
+
     scim: {}
 
   uaadb: 

data/spec/assets/deployments/aws-core-only.yml

@@ -33,7 +33,7 @@ resource_pools:
   size: 1
   stemcell: 
     name: bosh-stemcell
-    version: 0.6.4
+    version: 0.7.0
   cloud_properties: 
     instance_type: m1.small
   persistent_disk: 16192
@@ -157,6 +157,9 @@ properties:
     port: 8100
     catalina_opts: -Xmx128m -Xms30m -XX:MaxPermSize=128m
     no_ssl: true
+    client: 
+      autoapprove: []
+
     scim: {}
 
   uaadb: 

data/spec/assets/deployments/aws-core-with-nfs.yml

@@ -33,7 +33,7 @@ resource_pools:
   size: 1
   stemcell: 
     name: bosh-stemcell
-    version: 0.6.4
+    version: 0.7.0
   cloud_properties: 
     instance_type: m1.small
   persistent_disk: 16192

data/spec/unit/bosh_release_manager_spec.rb

@@ -23,9 +23,6 @@ describe Bosh::CloudFoundry::BoshReleaseManager do
     self.cf_release_branch_dir = File.join(cf_release_dir, "master")
     mkdir_p(cf_release_branch_dir)
     should_receive(:sh).with("git pull origin master")
-    should_receive(:sh).with("sed -i 's#git@github.com:#https://github.com/#g' .gitmodules")
-    should_receive(:sh).with("sed -i 's#git://github.com#https://github.com#g' .gitmodules")
-    should_receive(:sh).with("git submodule update --init --recursive")
     clone_or_update_cf_release
   end
 
@@ -34,25 +31,32 @@ describe Bosh::CloudFoundry::BoshReleaseManager do
     self.cf_release_branch_dir = File.join(cf_release_dir, "staging")
     mkdir_p(cf_release_branch_dir)
     should_receive(:sh).with("git pull origin staging")
+    clone_or_update_cf_release
+  end
+
+  it "prepare_cf_release_for_dev_release" do
+    self.cf_release_branch     = "master"
+    self.cf_release_branch_dir = File.join(cf_release_dir, "master")
+    mkdir_p(cf_release_branch_dir)
     should_receive(:sh).with("sed -i 's#git@github.com:#https://github.com/#g' .gitmodules")
     should_receive(:sh).with("sed -i 's#git://github.com#https://github.com#g' .gitmodules")
     should_receive(:sh).with("git submodule update --init --recursive")
-    clone_or_update_cf_release
+    prepare_cf_release_for_dev_release
   end
 
   describe "switch release types" do
     it "from final to dev" do
-      self.cf_release_branch     = "staging"
+      self.cf_release_branch     = "master"
       @system_config.release_name = "appcloud"
       @system_config.release_version = "latest"
       @system_config.save
       switch_to_development_release
-      @system_config.release_name.should == "appcloud-staging"
+      @system_config.release_name.should == "appcloud-master"
       @system_config.release_version.should == "latest"
     end
 
     it "from dev to final" do
-      @system_config.release_name = "appcloud-staging"
+      @system_config.release_name = "appcloud-master"
       @system_config.release_version = "latest"
       @system_config.save
       switch_to_final_release

data/spec/unit/cf_command_spec.rb

@@ -43,11 +43,11 @@ describe Bosh::Cli::Command::Base do
       @cmd.stub!(:bosh_target_uuid).and_return("DIRECTOR_UUID")
       @cmd.stub!(:bosh_cpi).and_return("aws")
       @cmd.should_receive(:`).
-        with("bosh public stemcells --tags aws | grep ' bosh-stemcell-' | awk '{ print $2 }' | sort -r | head -n 1").
+        with("bosh public stemcells --tags aws | grep ' bosh-stemcell-' | grep -v pre | awk '{ print $2 }' | sort -r | head -n 1").
         and_return("bosh-stemcell-aws-0.6.7.tgz")
       # FIXME default to stable stemcells when 0.8.1 is marked stable
       # @cmd.should_receive(:`).
-      #   with("bosh public stemcells --tags aws,stable | grep ' bosh-stemcell-' | awk '{ print $2 }' | sort -r | head -n 1").
+      #   with("bosh public stemcells --tags aws,stable | grep ' bosh-stemcell-' | grep -v pre | awk '{ print $2 }' | sort -r | head -n 1").
       #   and_return("bosh-stemcell-aws-0.6.7.tgz")
       @cmd.should_receive(:sh).
         with("bosh -n --color download public stemcell bosh-stemcell-aws-0.6.7.tgz")
@@ -80,58 +80,51 @@ describe Bosh::Cli::Command::Base do
       @cmd.upload_stemcell
     end
 
-    it "updates/creates/uploads final cf-release"
-    #  do
-    #   cf_release_dir = File.join(@releases_dir, "cf-release")
-    #   FileUtils.mkdir_p(cf_release_dir)
-    #   @cmd.system_config.cf_release_dir = cf_release_dir
-    #   @cmd.system_config.cf_release_branch = "staging"
-    #   @cmd.system_config.cf_release_branch_dir = File.join(cf_release_dir, "staging")
-    # 
-    #   @cmd.should_receive(:sh).with("git pull origin master")
-    #   script = <<-BASH.gsub(/^      /, '')
-    #   grep -rI "github.com" * .gitmodules | awk 'BEGIN {FS=":"} { print($1) }' | uniq while read file
-    #   do
-    #     echo "changing - $file"
-    #     sed -i 's#git://github.com#https://github.com#g' $file
-    #     sed -i 's#git@github.com:#https://github.com:#g' $file
-    #   done
-    #   BASH
-    #   @cmd.should_receive(:sh).with("sed -i 's#git@github.com:#https://github.com/#g' .gitmodules")
-    #   @cmd.should_receive(:sh).with("sed -i 's#git://github.com#https://github.com#g' .gitmodules")
-    #   @cmd.should_receive(:sh).with("git submodule update --init --recursive")
-    #   @cmd.should_receive(:`).with("tail -n 1 releases/index.yml | awk '{print $2}'").
-    #     and_return("126")
-    #   @cmd.should_receive(:sh).with("bosh -n --color upload release releases/appcloud-126.yml")
-    #   @cmd.add_option(:final, true)
-    #   @cmd.upload_release
-    # end
+    it "updates/creates/uploads final cf-release" do
+      generate_new_system
 
-    it "updates/creates/uploads development/edge cf-release (requires system setup)" 
-    # TODO turn this into a unit test for the specific methods
-    # do
-    #   cf_release_dir = File.join(@releases_dir, "cf-release")
-    #   FileUtils.mkdir_p(cf_release_dir)
-    #   @cmd.common_config.cf_release_dir = cf_release_dir
-    #   @cmd.add_option(:dev, true)
-    # 
-    #   @cmd.should_receive(:sh).with("git pull origin master")
-    #   script = <<-BASH.gsub(/^      /, '')
-    #   grep -rI "github.com" * .gitmodules | awk 'BEGIN {FS=":"} { print($1) }' | uniq while read file
-    #   do
-    #     echo "changing - $file"
-    #     sed -i 's#git://github.com#https://github.com#g' $file
-    #     sed -i 's#git@github.com:#https://github.com:#g' $file
-    #   done
-    #   BASH
-    #   @cmd.should_receive(:sh).with("sed -i 's#git@github.com:#https://github.com/#g' .gitmodules")
-    #   @cmd.should_receive(:sh).with("sed -i 's#git://github.com#https://github.com#g' .gitmodules")
-    #   @cmd.should_receive(:sh).with("git submodule update --init --recursive")
-    #   @cmd.should_receive(:write_dev_config_file).with("appcloud-staging")
-    #   @cmd.should_receive(:sh).with("bosh create release --with-tarball --force")
-    #   @cmd.should_receive(:sh).with("bosh -n --color upload release")
-    #   @cmd.upload_release
-    # end
+      cf_release_dir = File.join(@releases_dir, "cf-release")
+      @cmd.system_config.cf_release_dir = cf_release_dir
+      @cmd.system_config.cf_release_branch = "master"
+      @cmd.system_config.cf_release_branch_dir = File.join(cf_release_dir, "master")
+      FileUtils.mkdir_p(@cmd.system_config.cf_release_branch_dir)
+
+      @cmd.should_receive(:sh).with("git pull origin master")
+      @cmd.should_receive(:`).with("tail -n 1 releases/index.yml | awk '{print $2}'").and_return("128")
+      @cmd.should_receive(:sh).with("bosh -n --color upload release releases/appcloud-128.yml")
+
+      @cmd.add_option(:final, true)
+      @cmd.upload_release
+    end
+
+    it "updates/creates/uploads development/edge cf-release (requires system setup)" do
+      generate_new_system
+
+      cf_release_dir = File.join(@releases_dir, "cf-release")
+      @cmd.system_config.cf_release_dir = cf_release_dir
+      @cmd.system_config.cf_release_branch = "master"
+      @cmd.system_config.cf_release_branch_dir = File.join(cf_release_dir, "master")
+      FileUtils.mkdir_p(@cmd.system_config.cf_release_branch_dir)
+
+      @cmd.should_receive(:sh).with("git pull origin master")
+      script = <<-BASH.gsub(/^      /, '')
+      grep -rI "github.com" * .gitmodules | awk 'BEGIN {FS=":"} { print($1) }' | uniq while read file
+      do
+        echo "changing - $file"
+        sed -i 's#git://github.com#https://github.com#g' $file
+        sed -i 's#git@github.com:#https://github.com:#g' $file
+      done
+      BASH
+      @cmd.should_receive(:sh).with("sed -i 's#git@github.com:#https://github.com/#g' .gitmodules")
+      @cmd.should_receive(:sh).with("sed -i 's#git://github.com#https://github.com#g' .gitmodules")
+      @cmd.should_receive(:sh).with("git submodule update --init --recursive")
+      @cmd.should_receive(:write_dev_config_file).with("appcloud-master")
+      @cmd.should_receive(:sh).with("bosh -n --color create release --with-tarball --force")
+      @cmd.should_receive(:sh).with("bosh -n --color upload release")
+
+      @cmd.add_option(:branch, "master")
+      @cmd.upload_release
+    end
 
     def generate_new_system(cmd = nil)
       needs_initial_release_uploaded = true
@@ -154,16 +147,22 @@ describe Bosh::Cli::Command::Base do
 
       if needs_initial_release_uploaded
         cmd.should_receive(:bosh_releases).exactly(1).times.and_return([])
+
+        # TODO revert to these when appcloud-130 is released; and we go to final release
+        # cmd.should_receive(:clone_or_update_cf_release)
+        # cmd.should_receive(:upload_final_release)
+        cmd.should_receive(:set_cf_release_branch).with("master").exactly(2).times
         cmd.should_receive(:clone_or_update_cf_release)
+        cmd.should_receive(:prepare_cf_release_for_dev_release)
         cmd.should_receive(:create_and_upload_dev_release)
       else
         cmd.should_receive(:bosh_releases).exactly(1).times.and_return([
-          {"name"=>"appcloud", "versions"=>["124", "126"], "in_use"=>[]},
-          {"name"=>"appcloud-staging", "versions"=>["124.1-dev", "126.1-dev"], "in_use"=>[]},
+          {"name"=>"appcloud", "versions"=>["124", "126", "129"], "in_use"=>[]},
+          {"name"=>"appcloud-master", "versions"=>["124.1-dev", "126.1-dev"], "in_use"=>[]},
         ])
       end
 
-      cmd.should_receive(:bosh_stemcell_versions).exactly(4).times.and_return(['0.6.4'])
+      cmd.should_receive(:bosh_stemcell_versions).exactly(4).times.and_return(['0.7.0'])
       cmd.should_receive(:render_system)
 
       provider = Bosh::CloudFoundry::Providers::AWS.new
@@ -196,11 +195,13 @@ describe Bosh::Cli::Command::Base do
       File.basename(@cmd.system).should == "production"
     end
 
-    it "temporarily uploads latest stemcell & patched cf-release by default" do
-      generate_new_system(@cmd)
-      File.basename(@cmd.system).should == "production"
-      @cmd.system_config.release_name.should == "appcloud-staging"
-    end
+    # TODO restore when appcloud-130 released
+    it "uploads latest stemcell & final cf-release by default" 
+    # do
+    #   generate_new_system(@cmd)
+    #   File.basename(@cmd.system).should == "production"
+    #   @cmd.system_config.release_name.should == "appcloud"
+    # end
 
     it "new system has common random password" do
       generate_new_system(@cmd)

data/spec/unit/config_options_spec.rb

@@ -17,10 +17,6 @@ describe Bosh::CloudFoundry::ConfigOptions do
     mkdir_p(@options[:system])
   end
 
-  describe "common_config attribute" do
-    
-  end
-
   describe "system_config attribute" do
     it "release_name can be overridden but is stored in system_config" do
       options[:release_name] = "CHANGED"

data/spec/unit/system_deployment_manifest_renderer_spec.rb

@@ -25,7 +25,7 @@ describe Bosh::CloudFoundry::SystemDeploymentManifestRenderer do
     @system_config.release_name = 'appcloud'
     @system_config.release_version = 'latest'
     @system_config.stemcell_name = 'bosh-stemcell'
-    @system_config.stemcell_version = '0.6.4'
+    @system_config.stemcell_version = '0.7.0'
     @system_config.core_server_flavor = 'm1.small'
     @system_config.core_ip = '1.2.3.4'
     @system_config.root_dns = 'mycompany.com'

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: bosh-cloudfoundry
 version: !ruby/object:Gem::Version
-  version: 0.5.1
+  version: 0.6.0
   prerelease: 
 platform: ruby
 authors:
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-02-12 00:00:00.000000000 Z
+date: 2013-02-27 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bosh_cli
@@ -109,6 +109,9 @@ files:
 - Rakefile
 - bosh-cloudfoundry.gemspec
 - config/defaults.yml
+- docs/BOSH-cf-initial-create.png
+- docs/BOSH-cf-scaled.png
+- docs/concept.md
 - lib/bosh-cloudfoundry.rb
 - lib/bosh-cloudfoundry/bosh_release_manager.rb
 - lib/bosh-cloudfoundry/config.rb
@@ -160,7 +163,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '0'
       segments:
       - 0
-      hash: -2872187592937084659
+      hash: 1064713131226492709
 required_rubygems_version: !ruby/object:Gem::Requirement
   none: false
   requirements:
@@ -169,10 +172,10 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
       segments:
       - 0
-      hash: -2872187592937084659
+      hash: 1064713131226492709
 requirements: []
 rubyforge_project: 
-rubygems_version: 1.8.24
+rubygems_version: 1.8.25
 signing_key: 
 specification_version: 3
 summary: Create and manage your Cloud Foundry deployments on AWS and OpenStack with