bosh-cloudfoundry 0.7.0.alpha.11 → 0.7.0

data/ChangeLog.md

@@ -5,17 +5,16 @@
 `bosh-cloudfoundry` is back, rewritten, and supporting Cloud Foundry v2.
 
 ```
-$ gem install bosh-cloudfoundry -v "~> 0.7.0.alpha"
+$ gem install bosh-cloudfoundry
 $ bosh prepare cf
 $ bosh create cf --dns mycloud.com --public-ip 1.2.3.4
 ```
 
-As a rewrite, v0.7 initially implements fewer features than were in v0.6. It takes advantage of the long-awaited first final release of [cf-release](https://github.com/cloudfoundry/cf-release) for Cloud Foundry v2 (v132).
-
 The rewrite introduces some new implementation/feature concepts:
 
-* using `bosh diff` (aka biff) to generate the deployment file
+* takes advantage of the long-awaited first final release of [cf-release](https://github.com/cloudfoundry/cf-release) for Cloud Foundry v2 (v132).
 * bundles all final releases into the project & distributed rubygem/plugin (no runtime dependency on cf-release git repository; only the public blobstore)
+* using `bosh diff` (aka biff) to generate the deployment file
 * templates are versioned for each final release (unless new templates not required for new release)
 * different sizes of deployments (orders of magnitude), such as small, medium & large: `bosh create cf --deployment-size large`
 * mutable/changable properties (and immutable properties) for each template version: `bosh change cf attributes persistent_disk=8192`

data/README.md

@@ -15,7 +15,7 @@ $ bosh delete cf
 
 The deployed Cloud Foundry does not include any data or messaging services for the user applications. These are available as add-ons coming soon.
 
-[![Build Status](https://travis-ci.org/cloudfoundry-community/bosh-cloudfoundry.png?branch=v0.7)](https://travis-ci.org/cloudfoundry-community/bosh-cloudfoundry) [![Stories in Ready](http://badge.waffle.io/cloudfoundry-community/bosh-cloudfoundry.png)](http://waffle.io/cloudfoundry-community/bosh-cloudfoundry)
+[![Build Status](https://travis-ci.org/cloudfoundry-community/bosh-cloudfoundry.png?branch=v0.7)](https://travis-ci.org/cloudfoundry-community/bosh-cloudfoundry) [![Stories in Ready](http://badge.waffle.io/cloudfoundry-community/bosh-cloudfoundry.png)](http://waffle.io/cloudfoundry-community/bosh-cloudfoundry) [![Code Climate](https://codeclimate.com/github/cloudfoundry-community/bosh-cloudfoundry.png)](https://codeclimate.com/github/cloudfoundry-community/bosh-cloudfoundry)
 
 ## What gets created?
 
@@ -85,7 +85,7 @@ Install via RubyGems:
 
 ```
 $ gem install bosh_cli -v "~> 1.5.0.pre" --source https://s3.amazonaws.com/bosh-jenkins-gems/ 
-$ gem install bosh-cloudfoundry -v "~> 0.7.0.alpha"
+$ gem install bosh-cloudfoundry
 ```
 
 The `bosh_cli` gem is currently only available from S3, rather than RubyGem itself. So it needs to be installed first.

data/bosh-cloudfoundry.gemspec

@@ -2,7 +2,7 @@
 
 Gem::Specification.new do |spec|
   spec.name          = "bosh-cloudfoundry"
-  spec.version       = "0.7.0.alpha.11"
+  spec.version       = "0.7.0"
   spec.authors       = ["Dr Nic Williams"]
   spec.email         = ["drnicwilliams@gmail.com"]
   spec.description   = %q{Create & manage Cloud Foundry deployments}

data/spec/assets/v133/aws/large.yml

@@ -274,6 +274,7 @@ properties:
       app_package_directory_key: cc-packages
     droplets:
       droplet_directory_key: cc-droplets
+    default_quota_definition: runaway
 
   ccng: *cc
 

data/spec/assets/v133/aws/medium.yml

@@ -151,6 +151,10 @@ properties:
 
   dea_next: *dea
 
+  syslog_aggregator:
+    address: 0.syslog-aggregator.default.demo.microbosh
+    port: 54321
+
   nfs_server:
     address: 0.data.default.demo.microbosh
     #network: "*.demo.microbosh"
@@ -229,6 +233,7 @@ properties:
       app_package_directory_key: cc-packages
     droplets:
       droplet_directory_key: cc-droplets
+    default_quota_definition: runaway
 
   ccng: *cc
 

data/templates/v133/aws/large/deployment_file.yml.erb

@@ -311,6 +311,7 @@ properties:
       app_package_directory_key: cc-packages
     droplets:
       droplet_directory_key: cc-droplets
+    default_quota_definition: runaway
 
   ccng: *cc
 

data/templates/v133/aws/medium/deployment_file.yml.erb

@@ -188,6 +188,10 @@ properties:
 
   dea_next: *dea
 
+  syslog_aggregator:
+    address: 0.syslog-aggregator.default.<%= name %>.microbosh
+    port: 54321
+
   nfs_server:
     address: 0.data.default.<%= name %>.microbosh
     #network: "*.<%= name %>.microbosh"
@@ -266,6 +270,7 @@ properties:
       app_package_directory_key: cc-packages
     droplets:
       droplet_directory_key: cc-droplets
+    default_quota_definition: runaway
 
   ccng: *cc
 

data/templates/v133/openstack/large/deployment_file.yml.erb

@@ -311,6 +311,7 @@ properties:
       app_package_directory_key: cc-packages
     droplets:
       droplet_directory_key: cc-droplets
+    default_quota_definition: runaway
 
   ccng: *cc
 

data/templates/v133/openstack/medium/deployment_file.yml.erb

@@ -188,6 +188,10 @@ properties:
 
   dea_next: *dea
 
+  syslog_aggregator:
+    address: 0.syslog-aggregator.default.<%= name %>.microbosh
+    port: 54321
+
   nfs_server:
     address: 0.data.default.<%= name %>.microbosh
     #network: "*.<%= name %>.microbosh"
@@ -266,6 +270,7 @@ properties:
       app_package_directory_key: cc-packages
     droplets:
       droplet_directory_key: cc-droplets
+    default_quota_definition: runaway
 
   ccng: *cc
 

data/templates/v134/aws/large/deployment_file.yml.erb

@@ -311,6 +311,7 @@ properties:
       app_package_directory_key: cc-packages
     droplets:
       droplet_directory_key: cc-droplets
+    default_quota_definition: runaway
 
   ccng: *cc
 

data/templates/v134/aws/medium/deployment_file.yml.erb

@@ -188,6 +188,10 @@ properties:
 
   dea_next: *dea
 
+  syslog_aggregator:
+    address: 0.syslog-aggregator.default.<%= name %>.microbosh
+    port: 54321
+
   nfs_server:
     address: 0.data.default.<%= name %>.microbosh
     #network: "*.<%= name %>.microbosh"
@@ -266,6 +270,7 @@ properties:
       app_package_directory_key: cc-packages
     droplets:
       droplet_directory_key: cc-droplets
+    default_quota_definition: runaway
 
   ccng: *cc
 

data/templates/v134/openstack/large/deployment_file.yml.erb

@@ -311,6 +311,7 @@ properties:
       app_package_directory_key: cc-packages
     droplets:
       droplet_directory_key: cc-droplets
+    default_quota_definition: runaway
 
   ccng: *cc
 

data/templates/v134/openstack/medium/deployment_file.yml.erb

@@ -188,6 +188,10 @@ properties:
 
   dea_next: *dea
 
+  syslog_aggregator:
+    address: 0.syslog-aggregator.default.<%= name %>.microbosh
+    port: 54321
+
   nfs_server:
     address: 0.data.default.<%= name %>.microbosh
     #network: "*.<%= name %>.microbosh"
@@ -266,6 +270,7 @@ properties:
       app_package_directory_key: cc-packages
     droplets:
       droplet_directory_key: cc-droplets
+    default_quota_definition: runaway
 
   ccng: *cc
 

data/tutorials/README.md

@@ -2,4 +2,14 @@
 
 ## End-to-end tutorials
 
-* [Build Your Own Heroku With Open Source Cloud Foundry](https://github.com/cloudfoundry-community/bosh-cloudfoundry/blob/master/tutorials/build-your-own-heroku-with-cloudfoundry.md)
+* [Build Your Own Heroku With Open Source Cloud Foundry](https://github.com/cloudfoundry-community/bosh-cloudfoundry/blob/master/tutorials/build-your-own-heroku-with-cloudfoundry.md)
+
+## Practice activities
+
+Challenges and tasks for you to perform that will help you learn more about running your own Cloud Foundry.
+
+* [Which bits are running where?](https://github.com/cloudfoundry-community/bosh-cloudfoundry/blob/master/tutorials/activities/which-bits-are-running-where.md)
+* [Run more applications by adding DEA servers](https://github.com/cloudfoundry-community/bosh-cloudfoundry/blob/master/tutorials/activities/add-more-deas.md)
+* [View system health](https://github.com/cloudfoundry-community/bosh-cloudfoundry/blob/master/tutorials/activities/view-system-health.md)
+* [Move the Cloud Controller into its own VMs and scale it](https://github.com/cloudfoundry-community/bosh-cloudfoundry/blob/master/tutorials/activities/move-cc-into-own-job-and-scale.md)
+

data/tutorials/activities/add-more-deas.md

@@ -0,0 +1,178 @@
+# Add more DEA servers
+
+In the initial "tutorial" deployment there is a single DEA server - a small instance with approximately 2G of RAM. This isn't very much and you won't be able to run many applications.
+
+One quick way to increase the amount of RAM available for applications is to run more DEA servers.
+
+## Activity
+
+Scale the "env" tutorial application to exceed the 2G of RAM available and see the error.
+
+Edit the deployment file (as of writing there is not yet a nice way to do this via bosh-cloudfoundry) to increase the number of DEA servers to 3.
+
+## Tips
+
+1. Scale a running Cloud Foundry app using `cf scale`
+1. To edit your deployment file run `bosh edit deployment`
+1. If you increase the `instances:` of a job, also increase the `size:` of the corresponding `resource_pool`
+
+## Solution
+
+To scale the app and run out of RAM:
+
+```
+$ cf scale env
+Instances> 10
+
+1: 128M
+2: 256M
+3: 512M
+4: 1G
+Memory Limit> 128M
+
+Scaling env2... OK
+Stopping env2... OK
+
+Preparing to start env2... OK
+Checking status of app 'env2'...
+   0 of 10 instances running (10 down)
+   0 of 10 instances running (10 down)
+   0 of 10 instances running (10 down)
+   0 of 10 instances running (10 down)
+   0 of 10 instances running (10 down)
+   0 of 10 instances running (10 down)
+   0 of 10 instances running (10 down)
+   0 of 10 instances running (10 starting)
+   0 of 10 instances running (6 starting, 4 down)
+   0 of 10 instances running (3 starting, 7 down)
+   3 of 10 instances running (3 running, 7 starting)
+   3 of 10 instances running (3 running, 7 starting)
+   6 of 10 instances running (6 running, 4 down)
+   8 of 10 instances running (8 running, 2 starting)
+   9 of 10 instances running (9 running, 1 starting)
+   7 of 10 instances running (7 running, 3 down)
+   6 of 10 instances running (6 running, 4 down)
+   7 of 10 instances running (7 running, 3 down)
+```
+
+And it never quite finishes. This is the current sign that there isn't enough capacity for your application's scale.
+
+To scale up the number of DEA servers, edit the following sections of your deployment file (via `bosh edit deploment`).
+
+Increase the `small` resource pool by 2.
+
+``` yaml
+resource_pools:
+  - name: small
+    network: default
+    size: 6
+    stemcell:
+      name: bosh-stemcell
+      version: latest
+    cloud_properties:
+      instance_type: m1.small
+```
+
+Increase the 'dea' job by 2.
+
+In `jobs:`
+
+``` yaml
+- name: dea
+  release: cf-release
+  template:
+    - dea_next
+  instances: 3
+  resource_pool: small
+  networks:
+    - name: default
+      default: [dns, gateway]
+```
+
+To apply the changes run `bosh deploy`.
+
+```
+$ bosh deploy
+...
+Resource pools
+small
+  changed size: 
+    - 4
+    + 6
+
+Networks
+No changes
+
+Jobs
+dea
+  changed instances: 
+    - 1
+    + 3
+```
+
+Type "yes" and Enter.
+
+It will boot up two new servers and run the DEA job on them. Each DEA job will automatically join Cloud Foundry by announcing itself to the Cloud Controller.
+
+```
+Creating bound missing VMs
+small/0, small/1                    |                        | 0/2 00:00:04  ETA: --:--:--          
+```
+
+And later...
+
+```
+Updating job dea
+dea/1 (canary)                      |ooooooooo               | 0/2 00:00:04  ETA: --:--:--          
+```
+
+Now scale the app again to 10 instances:
+
+```
+$ cf scale env
+Instances> 10
+
+1: 128M
+2: 256M
+3: 512M
+4: 1G
+Memory Limit> 1 
+
+Scaling env2... OK
+Stopping env2... OK
+
+Preparing to start env... OK
+Checking status of app 'env2'...
+   0 of 10 instances running (10 down)
+   0 of 10 instances running (10 down)
+   0 of 10 instances running (10 down)
+   0 of 10 instances running (10 down)
+   0 of 10 instances running (10 down)
+   0 of 10 instances running (10 down)
+   0 of 10 instances running (10 down)
+   0 of 10 instances running (10 starting)
+   0 of 10 instances running (6 starting, 4 down)
+   0 of 10 instances running (3 starting, 7 down)
+   3 of 10 instances running (3 running, 7 starting)
+   3 of 10 instances running (3 running, 7 starting)
+   6 of 10 instances running (6 running, 4 down)
+   8 of 10 instances running (8 running, 2 starting)
+   9 of 10 instances running (9 running, 1 starting)
+   7 of 10 instances running (7 running, 3 down)
+   6 of 10 instances running (6 running, 4 down)
+   7 of 10 instances running (7 running, 3 down)
+   6 of 10 instances running (6 running, 4 down)
+   7 of 10 instances running (7 running, 3 down)
+   6 of 10 instances running (6 running, 4 down)
+   7 of 10 instances running (7 running, 3 down)
+   6 of 10 instances running (6 running, 4 down)
+  10 of 10 instances running (10 running)
+Push successful! App 'env2' available at http://env.1.2.3.4.xip.io
+```
+
+
+## Bonus
+
+Why did the two identical DEA servers start one at a time?
+
+Learn about "canaries" in the educational video [BOSH: What, How, When](http://drnicwilliams.com/2012/05/15/bosh-what-how-when/ "Dr Nic's   BOSH: What, How, When").

data/tutorials/activities/move-cc-into-own-job-and-scale.md

@@ -0,0 +1,111 @@
+# Move the Cloud Controller into its own VMs and scale it
+
+In the default "medium" deployment (used in the step-by-step tutorial), the Cloud Controller is collocated with the front-end router on the `api/0` server.
+
+## Activity
+
+Manually edit your deployment to run the Cloud Controller on its own dedicated server. Then run two Cloud Controller servers. Then revert back to running it on the same server as the router (and which also has the public IP attached).
+
+You will see the following when running `bosh vms`:
+
+```
+$ bosh vms
++--------------------+---------+---------------+------------------------------+
+| Job/index          | State   | Resource Pool | IPs                          |
++--------------------+---------+---------------+------------------------------+
+| api/0              | running | small         | 10.159.41.142, 50.19.127.213 |
+| cloud_controller/0 | running | small         | 10.85.1.115                  |
+| cloud_controller/1 | running | small         | 10.116.133.188               |
+| core/0             | running | small         | 10.170.15.85                 |
+| data/0             | running | small         | 10.168.20.214                |
+| dea/0              | running | small         | 10.158.75.168                |
++--------------------+---------+---------------+------------------------------+
+```
+
+## Tips
+
+1. to edit your deployment file run `bosh edit deployment`
+1. to make a backup of your deployment file, get its path from `bosh deployment`
+1. to apply the new deployment file changes run `bosh deploy`
+1. the Cloud Controller is the `cloud_controller_ng` job template within the `api` job in the deployment file.
+1. remove `- cloud_controller_ng` from the `name: api` job; and create a new job that looks similar to the `name: core` job
+1. your new `cloud_controller` job needs a `properties.db` set to `databases` to tell it where in the `properties` to find the database connections. The `api` job no longer needs these properties.
+
+
+## Solution
+
+Add the `cloud_controller` job above the `api` job; and remove `cloud_controller_ng` from the `api` job.
+
+Convert the following `api` job:
+
+``` yaml
+- name: api
+  release: cf-release
+  template:
+    - cloud_controller_ng
+    - gorouter
+  instances: 1
+  resource_pool: small
+  networks:
+  - name: default
+    default:
+    - dns
+    - gateway
+  - name: floating
+    static_ips:
+    - 50.19.127.213
+  properties:
+    db: databases
+```
+
+Into the following two jobs:
+
+``` yaml
+jobs:
+...
+
+- name: cloud_controller
+  release: cf-release
+  template:
+    - cloud_controller_ng
+  instances: 2
+  resource_pool: small
+  networks:
+  - name: default
+    default:
+    - dns
+    - gateway
+  properties:
+    db: databases
+
+- name: api
+  release: cf-release
+  template:
+    - gorouter
+  instances: 1
+  resource_pool: small
+  networks:
+  - name: default
+    default:
+    - dns
+    - gateway
+  - name: floating
+    static_ips:
+    - 1.2.3.4
+```
+
+And increase the `small` resource pool by two (for the two `cloud_controller` job servers above):
+
+```
+resource_pools:
+  - name: small
+    network: default
+    size: 6
+    stemcell:
+      name: bosh-stemcell
+      version: latest
+    cloud_properties:
+      instance_type: m1.small
+```
+
+To change back to collocated Cloud Controller and Router, copy back in your original deployment file; and `bosh deploy`.

data/tutorials/activities/view-system-health.md

@@ -0,0 +1,30 @@
+# View system health
+
+Bosh CLI includes the ability to view some health metrics of each server in a deployment.
+
+## Activity
+
+Display the health (CPU, RAM, persistent disk space) of the servers in your Cloud Foundry deployment.
+
+## Tips
+
+1. `bosh vms` shows the list of servers in all deployments
+1. `bosh vms tutorial` shows the list of servers in the deployment called `tutorial`
+1. `bosh help vms` shows the available options for `bosh vms`
+
+## Solution
+
+Invoke `bosh vms --vitals`:
+
+```
+$ bosh vms --vitals
++-----------+---------+---------------+------------------------------+-----------------------+------+------+------+----------------+------------+------------+------------+------------+
+| Job/index | State   | Resource Pool | IPs                          |         Load          | CPU  | CPU  | CPU  | Memory Usage   | Swap Usage | System     | Ephemeral  | Persistent |
+|           |         |               |                              | (avg01, avg05, avg15) | User | Sys  | Wait |                |            | Disk Usage | Disk Usage | Disk Usage |
++-----------+---------+---------------+------------------------------+-----------------------+------+------+------+----------------+------------+------------+------------+------------+
+| api/0     | running | small         | 10.159.41.142, 50.19.127.213 | 0.13%, 0.14%, 0.09%   | 0.0% | 0.0% | 0.1% | 14.2% (236.6M) | 0.0% (0B)  | 49%        | 1%         | n/a        |
+| core/0    | running | small         | 10.170.15.85                 | 0.88%, 1.04%, 0.66%   | 0.0% | 0.0% | 0.1% | 33.9% (561.9M) | 0.0% (0B)  | 49%        | 1%         | n/a        |
+| data/0    | running | small         | 10.168.20.214                | 0.00%, 0.06%, 0.09%   | 0.0% | 0.0% | 0.0% | 7.1% (119.2M)  | 0.0% (0B)  | 49%        | 1%         | 4%         |
+| dea/0     | running | small         | 10.158.75.168                | 0.00%, 0.01%, 0.05%   | 0.0% | 0.0% | 0.1% | 19.8% (329.2M) | 0.0% (0B)  | 49%        | 2%         | n/a        |
++-----------+---------+---------------+------------------------------+-----------------------+------+------+------+----------------+------------+------------+------------+------------+
+```

data/tutorials/activities/which-bits-are-running-where.md

@@ -0,0 +1,43 @@
+# Which bits are running where?
+
+Within your running "tutorial" deployment are the following components of Cloud Foundry:
+
+* Router
+* Cloud Controller
+* DEA
+* NATS (message bus)
+* UAA (User Account and Authentication)
+* Health manager
+* Postgresql
+* NFS
+
+## Activity
+
+Your "tutorial" deployment has 4 VMs:
+
+```
+$ bosh vms
++-----------+---------+---------------+-------------------------+
+| Job/index | State   | Resource Pool | IPs                     |
++-----------+---------+---------------+-------------------------+
+| api/0     | running | small         | 10.159.41.142, 1.2.3.4  |
+| core/0    | running | small         | 10.170.15.85            |
+| data/0    | running | small         | 10.168.20.214           |
+| dea/0     | running | small         | 10.158.75.168           |
++-----------+---------+---------------+-------------------------+
+```
+
+Which components are running on which VMs?
+
+## Tips
+
+1. to edit your deployment file run `bosh edit deployment`
+1. the VMs in the running deployment above are called `jobs`
+
+## Solution
+
+* `data/0` - `postgres`, `debian_nfs_server`
+* `core/0` - `nats`, `health_manager_next`, `uaa`
+* `api/0` - `cloud_controller_ng`, `gorouter`
+* `dea/0` - `dea_ng`
+

data/tutorials/build-your-own-heroku-with-cloudfoundry.md

@@ -9,9 +9,10 @@ There are some requirements:
 * AWS account, with access credentials, and capacity to provision 6 servers & 2 elastic IPs
 * Credit on the credit card attached to your AWS account for $1 or so
 * Ruby 1.9.3 or Ruby 2.0.0 installed on your local machine
+* Git (1.8+) installed on your local machine
 * Internet access
 * About an hour of your time (about 5 minutes of your human activity)
-* Possibly doesn't support Windows (please let me know if you have any success or failure)
+* Doesn't support Windows (lack of rsync for windows)
 
 Optionally:
 
@@ -63,7 +64,7 @@ The inception server above already installs the following gems. If you skipped t
 ```
 $ gem install bosh_cli -v "~> 1.5.0.pre" --source https://s3.amazonaws.com/bosh-jenkins-gems/ 
 $ gem install bosh-bootstrap
-$ gem install bosh-cloudfoundry --pre
+$ gem install bosh-cloudfoundry
 ```
 
 
@@ -189,6 +190,32 @@ $ cd cf-env
 $ bundle
 
 $ cf push env
+Instances> 1
+
+1: 128M
+2: 256M
+3: 512M
+4: 1G
+Memory Limit> 1
+
+Creating env... OK
+
+1: env
+2: none
+Subdomain> env
+
+1: 1.2.3.4.xip.io
+2: none
+Domain> 1.2.3.4.xip.io
+
+Creating route env.1.2.3.4.xip.io... OK
+Binding env.1.2.3.4.xip.io to env... OK
+
+Create services for application?> n
+
+Save configuration?> n
+...
+
 ```
 
 Open in a browser: http://env.1.2.3.4.xip.io

metadata

@@ -1,8 +1,8 @@
 --- !ruby/object:Gem::Specification
 name: bosh-cloudfoundry
 version: !ruby/object:Gem::Version
-  version: 0.7.0.alpha.11
-  prerelease: 6
+  version: 0.7.0
+  prerelease: 
 platform: ruby
 authors:
 - Dr Nic Williams
@@ -499,6 +499,10 @@ files:
 - templates/v134/openstack/spec
 - templates/v134/spec
 - tutorials/README.md
+- tutorials/activities/add-more-deas.md
+- tutorials/activities/move-cc-into-own-job-and-scale.md
+- tutorials/activities/view-system-health.md
+- tutorials/activities/which-bits-are-running-where.md
 - tutorials/build-your-own-heroku-with-cloudfoundry.md
 homepage: ''
 licenses:
@@ -515,13 +519,16 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '0'
       segments:
       - 0
-      hash: -725527410514912840
+      hash: 2077461492105591398
 required_rubygems_version: !ruby/object:Gem::Requirement
   none: false
   requirements:
-  - - ! '>'
+  - - ! '>='
     - !ruby/object:Gem::Version
-      version: 1.3.1
+      version: '0'
+      segments:
+      - 0
+      hash: 2077461492105591398
 requirements: []
 rubyforge_project: 
 rubygems_version: 1.8.25