RubyGems - bosh-gen - Versions diffs - 0.24.0 → 0.90.0 - Mend

bosh-gen 0.24.0 → 0.90.0

Files changed (124) hide show

checksums.yaml CHANGED Viewed

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: fbd5f5ae54cc9d6045654c2545801a6c4f1f3d05
-  data.tar.gz: 1763c8535dbd032b94009dc164e1d178d983de8c
+  metadata.gz: a6628b3382da0dde67f0e955fe2c1d865f74c2c7
+  data.tar.gz: 144baff46e83bd9cbc98f38fb784198a951c6a32
 SHA512:
-  metadata.gz: bc70d4413f65711c68f4644f63fbec7c02f6c9a66865adba53bcc7e946416bfe206af44ab5ed5a0c709cc21c317402f9c328412d5fb74145191e774dc38227c7
-  data.tar.gz: 41a3a15b2d28a5e60cc33d076e8eebb24893a915e8e47ecdd4b82fba86d16dc7c49aab1b7752b947b5199da2fba98426c9fd591b1333665d0b69c0505ceb3583
+  metadata.gz: 9c0576c0e4c0c7ac85d70b6f5909423be564cb2039ffe16f582093eee02d200729cf286f310dcc13918900b342ae693b5f49d320297ea3d865e9cd7936531f84
+  data.tar.gz: 9d44ad668f5537a47d3abc31d314923d855a8d6acad835cfa18cb8627188ebc91d5b4548e06531d54b83dea0e14bfd8be98cfea0ec307de3f43835ae9c6277a7

data/README.md CHANGED Viewed

@@ -3,366 +3,6 @@ BOSH Generators
 Generators for creating and sharing BOSH releases.
-New in 0.20: Create packages from embedded Docker images
+New in 0.90: `bosh2` support - single `manifests/thing.yml` deployment file
-New in 0.17: Creates blobstore/bucket when creating new release. AWS bucket is publicly readable.
-If you would like to share your BOSH release with the world, you can use the [BOSH Community AWS S3 account](#share-bosh-releases).
-![example](http://f.cl.ly/items/3v2F43020a3N0Q1g3Z0E/bosh-gen-new.gif)
-Installation
-------------
-This application requires Ruby 1.9 or 2.0 and is installed via RubyGems:
-```
-$ gem install bosh-gen
-```
-Usage
------
-```
-$ bosh-gen new $(whoami)-project
-      create
-Auto-detected infrastructure API credentials at ~/.fog (override with $FOG)
-1. AWS (community)
-2. Alternate credentials
-Choose an auto-detected infrastructure: 2
-Choose AWS region: 1
-      create  README.md
-      create  Rakefile
-      create  jobs
-      create  jobs/my-project/templates/bin/my_project_ctl
-      ...
-      create  config/blobs.yml
-      create  config/dev.yml
-      create  config/private.yml
-      create  config/final.yml
-      create  .gitignore
-         run  git init from "."
-Next, change to BOSH release location:
-cd ./my-project-boshrelease
-Finally...
-Attempting to create blobstore my-project-boshrelease... done
-Confirming: Using blobstore my-project-boshrelease
-```
-Your project is now in the folder mentioned above:
-```
-$ cd ./my-project-boshrelease
-```
-```
-$ wget -P /tmp http://ftp.ruby-lang.org/pub/ruby/2.2/ruby-2.2.2.tar.gz
-$ bosh-gen package ruby -f /tmp/ruby-2.2.2.tar.gz
-$ bosh-gen job some-ruby-job -d ruby
-$ bosh create release --force
-```
-To test out each iteration of your release, you can create a manifest, upload your release, and deploy it:
-```
-./templates/make_manifest warden
-bosh upload release
-bosh -n deploy
-```
-The large ruby tarball is automatically placed in the `blobs/` folder. Before you share your boshrelease with other developers you will want to sync it to your blobstore (the S3 bucket created via `bosh-gen new`\):
-```
-$ bosh upload blobs
-```
-Quickly creating packages
--------------------------
-There is a slow way to create a package, and there are three faster ways. Slow vs fast is not a debated about best vs worse. But sometimes you're in a hurry.
-### Slow way
-```
-$ bosh-gen package apache2
-create  packages/apache2/packaging
-create  packages/apache2/spec
-```
-The slowest way to create a package is to run the command above, then get the source, read the "install from source" instructions, and create a package.
-### Slightly faster way
-As above, when we created the `ruby` package we included a pre-downloaded asset:
-```
-$ bosh-gen package ruby -f /tmp/ruby-2.2.2.tar.gz
-```
-If you download the source files first, and reference them with the `bosh-gen package` generator, then it will attempt to guess how to install the package. The generated `packaging` script will include some starting commands that might work.
-The command above will also copy the target file into the `blobs/ruby/` folder. One less thing for you to do.
-You still need to look up "how to install from source" instructions and put them in `packages/ruby/packaging` script.
-### Fastest way - reuse existing packages
-```
-$ bosh-gen extract-pkg ../cf-release/packages/postgres
-```
-The fastest way is to reuse an existing, working package from another BOSH release that you have on your local machine.
-This command will copy across the `packages/postgres/spec` & `packages/postgres/packaging` files, as well as any blobs or src files that are referenced in the original BOSH release.
-This is a great command to use. There are a growing number of BOSH releases on GitHub from which to steal, err, extract packages into your own BOSH releases.
-Remember, first run `bosh sync blobs` in the target BOSH release project. Otherwise it will not be able to copy over the blobs.
-### Fast way - embedded Docker images
-This use case assumes you have `docker` CLI installed and access to a Docker daemon.
-It will also make your BOSH release dependent upon the [cf-platform-eng/docker-boshrelease](https://bosh.io/d/github.com/cf-platform-eng/docker-boshrelease) release which installs the Docker daemon on VMs; and offers a simple way to run Docker containers if required.
-```
-$ bosh-gen package tmate --docker-image nicopace/tmate-docker
-       exist  jobs
-      create  jobs/nicopace_tmate_docker_image/monit
-      create  jobs/nicopace_tmate_docker_image/spec
-      create  jobs/nicopace_tmate_docker_image/templates/bin/install_ctl
-      create  jobs/nicopace_tmate_docker_image/templates/bin/monit_debugger
-      create  jobs/nicopace_tmate_docker_image/templates/helpers/ctl_setup.sh
-      create  jobs/nicopace_tmate_docker_image/templates/helpers/ctl_utils.sh
-       exist  packages
-      create  packages/tmate/packaging
-      create  packages/tmate/spec
-docker pull nicopace/tmate-docker
-Pulling repository nicopace/tmate-docker
-7b9df453c66b: Download complete
-...
-6df853718c80: Download complete
-Status: Image is up to date for nicopace/tmate-docker:latest
-docker save nicopace/tmate-docker > blobs/docker-images/nicopace_tmate_docker.tgz
-$ bosh create release --force
-...
-Release name: tmate-server
-Release version: 0+dev.1
-```
-The `package --docker-image` flag will display the next steps help as well:
-```
-Next steps:
-  1. To use this BOSH release, first upload it and the docker release to your BOSH:
-    bosh upload release https://bosh.io/releases/cloudfoundry-community/consul-docker
-    bosh upload release https://bosh.io/d/github.com/cf-platform-eng/docker-boshrelease
-  2. To use the docker image, your deployment job needs to start with the following:
-    jobs:
-    - name: some_job
-    templates:
-      # run docker daemon
-      - {name: docker, release: docker}
-      # warm docker image cache from bosh package
-      - {name: nicopace_tmate_docker_image, release: tmate-server}
-  3. To simply run a single container, try the 'containers' job from 'docker' release
-    https://github.com/cloudfoundry-community/consul-docker-boshrelease/blob/master/templates/jobs.yml#L18-L40
-```
-### Fast way - reuse Aptitude/Debian packages
-```
-$ bosh-gen package apache2 --apt
-$ vagrant up
-$ vagrant ssh -c '/vagrant/src/apt/fetch_debs.sh apache2'
-$ vagrant destroy
-```
-You can add/change the Debian packages to install by editing `src/apt/apache2/aptfile` and re-running the `fetch_debs.sh` command above. You might want to delete `blobs/apt/apache2` first to ensure that only the fetched `.deb` files are subsequently included during package compilation.
-It is possible now to download one or more `.deb` files into the `blobs/apt/` folder, and have them installed during package compilation time.
-The installed .deb packages will be available at `/var/vcap/packages/apache2/apt`; rather than within the root folder system.
-Your job monit control scripts can source a provided `profile.sh` to setup environment variables:
-```
-source /var/vcap/packages/apache2/profile.sh
-```
-This is the last option, and it is not the best option. Many Debian packages will also start processes that have default configuration that is not correct for your use case. It may be fast to get the Debian packages; but additional work may be required by your jobs to stop and unhook the processes that are automatically started upon installation.
-Tutorial
---------
-To see how the various commands work together, let's create a new bosh release for [Cassandra](http://cassandra.apache.org/).
-```
-$ bosh-gen new cassandra
-$ cd cassandra
-$ bosh-gen extract-pkg ../cf-release/packages/dea_jvm7
-      create  packages/dea_jvm7
-      create  packages/dea_jvm7/packaging
-      create  packages/dea_jvm7/spec
-      create  blobs/java/jre-7u4-linux-i586.tar.gz
-      create  blobs/java/jre-7u4-linux-x64.tar.gz
-      readme  Upload blobs with 'bosh upload blobs'
-$ mv packages/dea_jvm7 packages/java7
-```
-In `packages/java7/spec`, rename it to `java7`.
-```
-$ bosh-gen package cassandra -d java7 -f ~/Downloads/apache-cassandra-1.0.11-bin.tar.gz
-      create  packages/cassandra/packaging
-      create  blobs/cassandra/apache-cassandra-1.0.11-bin.tar.gz
-      create  packages/cassandra/spec
-```
-Change `packages/cassandra/packaging` to:
-```
-tar xfv cassandra/apache-cassandra-1.0.11-bin.tar.gz
-cp -a apache-cassandra-1.0.11/* $BOSH_INSTALL_TARGET
-```
-Now create a stub for running cassandra as a job:
-```
-$ bosh-gen job cassandra -d java7 cassandra
-      create  jobs/cassandra
-      create  jobs/cassandra/monit
-      create  jobs/cassandra/templates/bin/cassandra_ctl
-      create  jobs/cassandra/templates/bin/monit_debugger
-      create  jobs/cassandra/templates/data/properties.sh.erb
-      create  jobs/cassandra/templates/helpers/ctl_setup.sh
-      create  jobs/cassandra/templates/helpers/ctl_utils.sh
-      create  jobs/cassandra/spec
-```
-Look at all that goodness!
-A quick summary of these files:
--	The `monit` script uses `bin/monit_debugger` to help you debug any glitches in starting/stopping processes.
--	`ctl_setup.sh` setups up lots of common folders and env vars.
--	`ctl_utils.sh` comes from cf-release's common/utils.sh with some extra helper functions
--	`data/properties.sh.erb` is where you extract any `<%= properties.cassandra... %>` values from the deployment manifest.
--	`bin/cassandra_ctl` no longer needs to be an unreadable ERb template! Use the env variables you create in `data/properties.sh.erb` and normal bash if statements instead of ERb `<% if ... %>` templates.
--	`examples/...` is a folder for documenting example, valid deployment manifest properties for the release.
-In `bin/cassandra_ctl` you now change "TODO" to `cassandra` and the rest of the tutorial is left to you, dear cassandra lover.
-Your release is now ready to build, test and deploy:
-```
-bosh create release --force
-bosh upload release
-```
-When you create a final release, you will first need to setup your AWS credentials in `config/final.yml`
-Share BOSH releases
--------------------
-To share your BOSH release with other BOSH users you need ONLY:
--	Use a public blobstore (such as AWS S3)
--	Use a public source control repository
--	Optionally, publicly share pre-created final release tarballs via a HTTP URL.
-### Share release tarballs via HTTP
-bosh-gen includes a BOSH CLI plugin to upload dev or final tarballs to your blobstore, and get a public URL back (if supported by your blobstore)
-```
-$ bosh create release --with-tarball
-$ bosh share release releases/my-project-1.tgz
-https://my-project-boshrelease.s3.amazonaws.com/boshrelease-my-project-1.tgz
-```
-The URL is displayed and can be given to other users and uploaded directly to their BOSH:
-```
-$ bosh upload release https://my-project-boshrelease.s3.amazonaws.com/boshrelease-my-project-1.tgz
-```
-They no longer require your BOSH release repo to access the BOSH release.
-![share-http-tarball](http://f.cl.ly/items/0R3A1w3k2E3h3a2d1g2U/bosh-gen-share-release.gif)
-### BOSH community facilities
-You are welcome to re-use the BOSH user community facilities:
--	Use the shared AWS S3 account (currently over 30 BOSH release blobstores).
--	Place your release git repository in the [@cloudfoundry-community](https://github.com/cloudfoundry-community) GitHub account (over 50 people have access).
-One time only, please email [Dr Nic Williams](mailto:&#x64;&#x72;&#x6E;&#x69;&#x63;&#x77;&#x69;&#x6C;&#x6C;&#x69;&#x61;&#x6D;&#x73;&#x40;&#x67;&#x6D;&#x61;&#x69;&#x6C;&#x2E;&#x63;&#x6F;&#x6D;) and he will set you up with access:
--	Read/write credentials to the AWS S3 account for your BOSH release blobstores/buckets
--	Access to create [@cloudfoundry-community](https://github.com/cloudfoundry-community) GitHub repositories for your BOSH releases
-When he gives you the AWS S3 credentials, place them in the `~/.fog` file and you'll easily be able to reuse them for each new BOSH release:
-```yaml
-:community:
-  :aws_access_key_id:     ACCESS
-  :aws_secret_access_key: SECRET
-```
-Then for your next BOSH release:
-```
-$ bosh-gen new my-project
-      create
-Auto-detected infrastructure API credentials at ~/.fog (override with $FOG)
-1. AWS (community)
-2. Alternate credentials
-Choose an auto-detected infrastructure: 2
-```
-You'll only need to do this once. Yes, it would be awesome if there was some public service to do this nicely. Want to build it?
-Create BOSH CLI plugins
------------------------
-You can now (v0.19.0+) quickly create a BOSH CLI plugin within a RubyGem (to share), BOSH workspace or release (local to that project).
-Inside your RubyGem, BOSH workspace or release:
-```
-$ bosh-gen cli-plugin setup-deployment
-    create  lib
-    create  lib/bosh/cli/commands/setup_deployment.rb
-```
-The `setup_deployment.rb` file will now be automatically picked up by `bosh` CLI locally OR if you distribute your project as a RubyGem:
-```
-$ bosh setup deployment
-WARNING: loading local plugin: lib/bosh/cli/commands/setup_deployment.rb
-TODO
-```
-Note: the hyphenated name `setup-deployment` becomes a space-separated command `setup deployment`.
-Contributing
-------------
-1.	Fork it
-2.	Create your feature branch (`git checkout -b my-new-feature`\)
-3.	Commit your changes (`git commit -am 'Added some feature'`\)
-4.	Push to the branch (`git push origin my-new-feature`\)
-5.	Create new Pull Request
+NOTE: this `bosh2` branch is under construction. README will be re-written as we get towards v1.0

data/bosh-gen.gemspec CHANGED Viewed

@@ -22,7 +22,8 @@ Gem::Specification.new do |gem|
   gem.add_dependency "cyoi", "~> 0.10"
   gem.add_dependency "fog", "~> 1.11"
-  gem.add_dependency "fog-aws", "0.1.1" # 0.1.2 causes issues
+  gem.add_dependency "fog-aws"
+  gem.add_dependency "mime-types"
   gem.add_dependency "readwritesettings", "~> 3.0"
   gem.add_dependency "activesupport", ">= 4.0", "< 5.0"

data/lib/bosh/gen/cli.rb CHANGED Viewed

@@ -5,8 +5,6 @@ module Bosh; end
 require "cli/config"
 require "cli/core_ext"
-require 'bosh/gen/models'
 module Bosh
   module Gen
     class Command < Thor
@@ -119,12 +117,6 @@ module Bosh
         Bosh::Gen::Generators::ExtractPackageGenerator.start([source_package_path])
       end
-      desc "cli-plugin NAME", "Add a BOSH CLI plugin into this project"
-      def cli_plugin(plugin_name)
-        require 'bosh/gen/generators/bosh_cli_plugin_generator'
-        Bosh::Gen::Generators::BoshCliPluginGenerator.start([plugin_name])
-      end
       no_tasks do
         def cyan; "\033[36m" end
         def clear; "\033[0m" end

data/lib/bosh/gen/generators/errand_generator/templates/jobs/%job_name%/templates/bin/run.tt CHANGED Viewed

@@ -1,5 +1,7 @@
 #!/bin/bash
+exec 2>&1
 set -e # exit immediately if a simple command exits with a non-zero status
 set -u # report the usage of uninitialized variables
@@ -9,4 +11,3 @@ source /var/vcap/jobs/<%= job_name %>/helpers/ctl_setup.sh '<%= job_name %>'
 ############################################################################
 # put your errand steps here...

data/lib/bosh/gen/generators/job_generator.rb CHANGED Viewed

@@ -60,11 +60,6 @@ module Bosh::Gen
         end
       end
-      def job_specification
-        config = { "name" => job_name, "packages" => dependencies, "templates" => @template_files }
-        create_file job_dir("spec"), YAML.dump(config)
-      end
       private
       def filenames
         files.map {|f| File.basename(f) }

data/lib/bosh/gen/generators/job_generator/templates/jobs/%job_name%_simple/monit.tt CHANGED Viewed

@@ -1,5 +1,5 @@
 check process <%= job_name %>
   with pidfile /var/vcap/sys/run/<%= job_name %>/<%= job_name %>.pid
-  start program "/var/vcap/jobs/<%= job_name %>/bin/monit_debugger <%= job_name %>_ctl '/var/vcap/jobs/<%= job_name %>/bin/ctl start'"
-  stop program "/var/vcap/jobs/<%= job_name %>/bin/monit_debugger <%= job_name %>_ctl '/var/vcap/jobs/<%= job_name %>/bin/ctl stop'"
+  start program "/var/vcap/jobs/<%= job_name %>/bin/monit_debugger ctl '/var/vcap/jobs/<%= job_name %>/bin/ctl start'"
+  stop program "/var/vcap/jobs/<%= job_name %>/bin/monit_debugger ctl '/var/vcap/jobs/<%= job_name %>/bin/ctl stop'"
   group vcap