baha 0.0.1

checksums.yaml

@@ -0,0 +1,15 @@
+---
+!binary "U0hBMQ==":
+  metadata.gz: !binary |-
+    M2E1OWQwZWYyYmQxMzc3NmM1ZjAzZGI3ZTA5NmM3NGQyYTBjZGIyOA==
+  data.tar.gz: !binary |-
+    YWJhZDkxNDNhYmNhMmYyMTBiZDNmNzliYTc2YmNmYTk4M2JmYTlmNQ==
+SHA512:
+  metadata.gz: !binary |-
+    MmVhNjgyMWFmOTJjZGRiYzE0Zjc1ZjdkOTBiZTlkY2YzYzhhMWRkN2I2ODdl
+    MzQ3N2IwMzhkMDQyNDJiZjU4ZjNmYWZkNzU3YjE0OWE3OThjNTkxZjQwMDZl
+    MDA2Zjg2ZTRhZjg2M2MwODZhNGRmMDM1ZjUwYmJhMDA1Y2E1ZGI=
+  data.tar.gz: !binary |-
+    MTJhOGNiYmYwOGI0YmE5NDI3Y2EwODFiZDU5MzNjYzE4ZmI4NTA0YTlkN2M0
+    ZTgxNmQ5Mzc1ZmZlM2M5NzJiNDg2YjkzYjdmZDlkNjkwYjA5NGUzNzE3YTQz
+    NGVkMTkzYTJhMzgwNGQwMDFjYzNlMWE2YzA1ZjdlZDZjYmJkMTk=

data/.gitignore

@@ -0,0 +1,15 @@
+/.bundle/
+/.yardoc
+/Gemfile.lock
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/spec/fixtures/workspace
+/tmp/
+*.bundle
+*.so
+*.o
+*.a
+mkmf.log

data/.travis.yml

@@ -0,0 +1,11 @@
+---
+language: ruby
+before_install: 
+  - gem update --system
+  - gem update
+script: bundle exec rake spec
+rvm:
+  - 1.9.2
+  - 1.9.3
+  - 2.0.0
+  - 2.1.0

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in Baha.gemspec
+gemspec

data/LICENSE.txt

@@ -0,0 +1,22 @@
+Copyright (c) 2014 Justen Walker
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,156 @@
+Baha 
+=======
+
+[![Gem Version](https://badge.fury.io/rb/baha.png)](http://badge.fury.io/rb/baha)
+[![Build Status](https://travis-ci.org/justenwalker/baha.png?branch=master)](https://travis-ci.org/justenwalker/baha)
+
+Introduction
+------------
+
+Baha is a command-line utility that assists in the creation of docker images.
+It addresses some of `Dockerfile`'s' shortcomings and encourages smaller, reusable, tagged images.
+
+Why not Dockerfiles?
+--------------------
+
+Dockerfiles are simple. They are an easy way to make a docker image without needing any other dependencies. However, their simplicity comes at a cost: redundancy.  This redundancy is present in a few ways that make more complicated images difficult to make and maintain.
+
+So why would I use Baha?
+------------------------
+
+### 1. Baha forbids more than 1 layer per image
+
+If you split statements across multiple RUN statements, each of these results in a new layer.
+The more commands you run, the more layers you create.  If you want to *minimize the number of layers* (See [Official Recommendations](https://docs.docker.com/articles/dockerfile_best-practices/)) 
+then you must ensure that all statements can be condensed into one line - sacrificing maintainability in the process.
+
+Baha encourages using scripts instead of `RUN` statements to ensure that only one layer is created per image.
+
+### 2. Baha encourages smaller images
+
+The nature of the way the dockerfiles are processed means that each command you run commits a new image.
+This means, that if you have a `RUN` statement that downloads packages for installation. 
+This will commit a layer with the installation files.  Later on if you clean up these files with further RUN commands, they will still exist in your image lineage - thus having no space savings. Without proper precautions, you'll end up having unnecessarily large images.
+
+Baha ensures that all setup tasks happen in a single commit - so you can write cleanup statements and be assured that they will indeed be absent in the resulting images.
+
+### 3. Baha understands the bigger picture
+
+Another best practice (2, #7) recommends that you use your own base image.
+Dockerfiles make it simple to create your own 'base' image, but how about updates?
+
+If you were to just rebuild all of your Dockerfiles, you would create an entirely new tree - even if nothing changed.
+
+Baha will rebuild your entire lineage if the base images changes, but will not rebuild base images if only the children change.
+
+**Caveat**: 
+
+Baha relies on tagging your releases, noticing when the tag has changed, and treating tags as immutable. 
+This is analogous to how **git** treats tags.
+
+Tagging your images is another best-practice (2, #5) anyway, so this is encouraged by design. 
+
+Bottom line is: If you change your image, you should change the tag/version.
+
+#### 4. Baha lets you cache build dependencies
+
+Most of the time, dependencies are downloaded and installed in the context of the docker container itself.
+This means that rebuilding an image results in redundant downloads if files haven't changed.
+
+Baha has pre-build steps that can be used to download files and prepare scripts on host machine.
+This workspace is made available to the image via a bind mount during build-time. Not only do these dependencies stick around between builds, but they do not need to be cleaned up after the image is committed, since they are never persisted to the container.
+
+**References**
+
+1. [Official Dockerfile Best Practices](https://docs.docker.com/articles/dockerfile_best-practices/)<a name="r1"></a>
+2. [Dockerfile Best Practices - take 2, by Michael Crosby](http://crosbymichael.com/dockerfile-best-practices-take-2.html)<a name="r2"></a>
+
+Disclaimer
+----------
+
+This gem was just released (pre 1.0) and is not ready for production use yet.
+
+During pre 1.0, things may change that break backwards compatibility between releases. Most likely these breaking
+changes would be related to the YAML file syntax.
+
+To Do
+-----
+See the [Issue Tracker](https://github.com/justenwalker/baha/issues)
+
+Installation
+------------
+
+```
+$ gem install baha
+```
+### gem install baha
+
+Usage
+-----
+
+```
+Baha Commands:
+  baha build [options] CONFIG  # Builds all docker images based on the given config
+  baha help [COMMAND]          # Describe available commands or one specific command
+  baha version                 # Print version and exit
+```
+
+**build**
+
+```
+Usage:
+  baha build [options] CONFIG
+
+Options:
+      [--logfile=LOGFILE]         # Log output to LOGFILE. Omit to log to stdout.
+  d, [--debug], [--no-debug]      # Toggle debug logging. Includes verbose.
+  v, [--verbose], [--no-verbose]  # Toggle verbose logging
+  q, [--quiet], [--no-quiet]      # Suppress all logging
+
+Description:
+  Reads the CONFIG file and builds all of the docker images in the order they appear.
+```
+
+Check out the `example` directory for a sample CONFIG file.
+
+How it works
+------------
+
+Baha will read a config.yml file first and load each image configuration.
+It will build each image in the order they appear by doing the following.
+
+### Check if the image needs updating
+1. Check to see if the parent image changed
+2. Check to see if the tag does not exist in the repository
+
+### Prepare the image's workspace (bind mount)
+1. Create the workspace directory if it doesn't exist
+2. Run any `pre_build` tasks to prepare dependencies
+
+### Run the image's `command` inside the image
+1. Creates a new container and runs it with the `command` given
+2. Commits the container with the run options specified in the images `config` section.
+
+How to Contribute
+-----------------
+
+### Running the tests
+
+    $ bundle
+    $ bundle exec rake spec
+
+### Installing locally
+
+    $ bundle
+    $ [bundle exec] rake install
+
+### Reporting Issues
+
+Please include a reproducible test case.
+
+License
+-------
+
+Copyright (c) 2014 Justen Walker.
+
+Released under the terms of the MIT License. For further information, please see the file `LICENSE.md`.

data/Rakefile

@@ -0,0 +1,21 @@
+require "bundler/gem_tasks"
+require 'rake/clean'
+require 'rspec/core/rake_task'
+
+require 'rake/clean'
+CLEAN.include('pkg/', 'example/workspace')
+CLOBBER.include('Gemfile.lock')
+
+RSpec::Core::RakeTask.new('spec')
+
+desc "Run example baha build"
+task :example do
+  require 'pathname'
+  exampleyml = (Pathname.pwd + 'example' + 'example.yml')
+  puts "Running: baha build -d #{exampleyml}"
+  sh('baha', 'build', '-d', exampleyml.to_s)
+end
+
+task :default do
+  sh %{rake -T}
+end

data/baha.gemspec

@@ -0,0 +1,31 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'baha/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = "baha"
+  spec.version       = Baha::VERSION
+  spec.authors       = ["Justen Walker"]
+  spec.email         = ["justen.walker+github@gmail.com"]
+  spec.summary       = %q{Baha - A tool for describing image builds for Docker}
+  spec.description   = %q{Baha - A tool for describing image builds for Docker}
+  spec.homepage      = ""
+  spec.license       = "MIT"
+
+  spec.files         = `git ls-files -z`.split("\x0")
+  spec.executables   = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
+  spec.test_files    = spec.files.grep(%r{^(test|spec|features)/})
+  spec.require_paths = ["lib"]
+
+  spec.add_dependency 'thor', '~> 0.19.1'
+  spec.add_dependency 'docker-api', '~> 1.14.0'
+  spec.add_dependency 'json', '~> 1.8.1'
+
+  spec.add_development_dependency 'bundler', '~> 1.7'
+  spec.add_development_dependency 'rake', '~> 10.0'
+  spec.add_development_dependency 'rspec', '~> 3.1.0'
+  spec.add_development_dependency 'rspec-mocks', '~> 3.1.3'
+  spec.add_development_dependency 'rspec-its', '~> 1.1.0'
+  spec.add_development_dependency 'simplecov', '~> 0.9.1'
+end

data/bin/baha

@@ -0,0 +1,5 @@
+#!/usr/bin/env ruby
+
+require 'baha/cli'
+
+Baha::CLI.start(ARGV)

data/example/.gitignore

@@ -0,0 +1 @@
+workspace/

data/example/base/init.sh.erb

@@ -0,0 +1,11 @@
+#! /bin/bash
+
+## Install required packages
+apt-get update
+apt-get upgrade -y
+apt-get install -y curl gnupg
+
+## Cleanup after ourselves
+apt-get autoremove -y
+apt-get clean
+rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

data/example/base/test-template.erb

@@ -0,0 +1,22 @@
+### This is the parent image config
+parent: <%= parent %>
+
+### This is the maintainer of the image
+maintainer: <%= maintainer %>
+
+### This is the bind location for the workspace
+bind: <%= bind %>
+
+### This is the location of the workspace on the host machine
+workspace: <%= workspace %>
+
+### This is the image name
+name: <%= name %>
+
+### This is the target tag/version
+tag: <%= tag %>
+
+### The render function will render other files
+### START RENDER ###
+include: <%= render "init.sh.erb"  %>
+### END RENDER  ###

data/example/example.yml

@@ -0,0 +1,54 @@
+---
+### Docker URL. Default is unix:///var/run/docker.sock
+### You can also run Baha with the env variable:
+### DOCKER_HOST=tcp://192.168.59.103:2376 
+# docker_url: tcp://192.168.59.103:2376
+#
+defaults:
+  ### If an image doesn't have a parent, it will default to this one.
+  parent: ubuntu:14.04.1
+  ### When tagging an image, the name will be tagged with: {repository}/{name}
+  repository: docker.example.com/baha
+  ### Put your name and email here to claim authorship
+  maintainer: Ishmael <ishmael@example.com>
+  ### By default, the image's workspace will be mounted here inside the container
+  bind: '/.data'
+### If your docker is configured for SSL connections
+### You should supply the options here.
+###
+### ENV variables take precedence over the ssl config
+### - DOCKER_CERT_PATH and DOCKER_TLS_VERIFY=1
+###
+### Leave this out if your docker connection is insecure (http or unix socket)
+### Having an ssl config assumes the connection is https://
+### having a unix socket assumes the connection is insecure, this section would be ignored
+ssl:
+  ### Path to certificates and key
+  ### Assumes the files are: ca.pem, cert.pem, and key.pem
+  ### Same as specifying DOCKER_CERT_PATH=~/.boot2docker/certs/boot2docker-vm
+  # cert_path: ~/.boot2docker/certs/boot2docker-vm
+  #
+  ### You can also specify them individually:
+  # cert: ~/.boot2docker/certs/boot2docker-vm/cert.pem
+  # key: ~/.boot2docker/certs/boot2docker-vm/key.pem
+  # ca: ~/.boot2docker/certs/boot2docker-vm/ca.pem
+  #
+  ### Verify docker host's certificate chain
+  verify: true
+### Each image is an element of the images array
+### The images will be built in the order they appear
+images:
+  - name: base
+    tag: 1.0.0 # Tag your image with a version. If the version changes, the image will be rebuilt
+    ### This list of tasks will be executed in order *before* the image container is built.
+    ### It is meant to prepare the workspace by downloading and creating scripts to be run in the context of the container.
+    pre_build:
+      ### The template module copies/processes an ERB into the image's workspace folder
+      - { template: 'base/init.sh.erb', dest: 'init.sh' }
+      ### This template shows the available bindings
+      - { template: 'base/test-template.erb', dest: 'test.txt' }
+    command: ['/bin/bash','/.data/init.sh']
+  ### Instead of listing all of the image configs in the same file, you can
+  ### include them from another file relative to this config. 
+  ### Keep things modular and maintainable.
+  - include: rvm/image.yml

data/example/rvm/image.yml

@@ -0,0 +1,33 @@
+### Base on another image created before me
+parent: base:1.0.0
+name: rvm-test
+tag: 1.9.3
+### Override default maintainer
+maintainer: '"Captain Ahab" <ahab@example.com>'
+### Override default bind mount
+bind: '/.rvm'
+pre_build:
+  - { download: 'https://get.rvm.io', file: 'rvm.sh' }
+  - { template: 'rvm/init.sh.erb', dest: 'init.sh' }
+command: ['/bin/bash','/.rvm/init.sh']
+### Specify container's run configuration here
+config:
+  ### Takes an array of port numbers (assumes tcp)
+  ### Can also be in the form 9000/udp to expose UDP ports
+  exposedports: 
+    - 8080
+  ### Register volumes. Each array item should be a path
+  volumes:
+    - '/logs'
+  ### Environment variables. Hash of KEY=VALUE pairs
+  env:
+    HOME: '/root'
+  ### Entrypoint command. Should just be 1 string
+  entrypoint: '/bin/bash'
+  ### Override the docker containers hostname
+  hostname: 'my.hostname.docker'
+  ### Override the internal username executing the command
+  user: 'root'
+  ### Options coming after the entrypoint
+  cmd:
+    - '-l'

data/example/rvm/init.sh.erb

@@ -0,0 +1,12 @@
+#! /bin/bash
+
+## Install RVM
+gpg --keyserver hkp://keys.gnupg.net --recv-keys D39DC0E3
+/bin/bash /.rvm/rvm.sh stable || exit 2
+source /etc/profile.d/rvm.sh
+rvm install 1.9.3
+
+## Clean up
+apt-get autoremove -y
+apt-get clean
+rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

data/lib/baha/builder.rb

@@ -0,0 +1,130 @@
+require 'baha/config'
+require 'baha/image'
+require 'baha/log'
+require 'baha/pre_build'
+require 'fileutils'
+
+class Baha::Builder
+  LOG = Baha::Log.for_name("Builder")
+
+  class BuildError < RuntimeError
+    attr_reader :image, :status
+    def initialize(status,image)
+      super("Unable to build image : #{image.name} -- Status: #{status.inspect}")
+      @image = image
+      @status = status
+    end
+  end
+
+  attr_reader :config
+
+  def initialize(config)
+    @config = case config
+    when Baha::Config
+      config
+    else
+      Baha::Config.load(config.to_s)
+    end
+  end
+
+  def build!
+    LOG.info("Building Images")
+    LOG.debug { "Config: #{@config.inspect}" }
+    LOG.debug { "Initializing Docker" }
+    @config.init_docker!
+
+    @config.each_image do |image|
+      build_log = Baha::Log.for_name("Builder [#{image.name}]")
+      unless image.needs_update?
+        build_log.info { "Skipped image #{image.name} - No update needed" }
+        next
+      end
+      
+      ## Prepare Workspace
+      workspace = Pathname.new(@config.workspace) + image.name
+      unless workspace.exist?
+        build_log.debug { "Creating Workspace: #{workspace}" }
+        FileUtils.mkdir_p workspace.to_s
+      end
+
+      ## Pre-Build tasks
+      build_log.info { "Building #{image.name}" }
+      if image.pre_build
+        build_log.debug { "Preparing workspace for #{image.name}" }
+        image.pre_build.each do |task|
+          build_log.debug { "execute task: #{task.inspect}" }
+          Baha::PreBuild::Module.execute(task.merge({ :config => @config, :image => image }))
+        end
+      end
+
+      ## Build Image
+      container_config = {
+        'Image'       => image.parent_id,
+        'Cmd'         => image.command,
+        'Workingdir'  => image.bind
+      }
+      build_log.debug { "Creating container for #{image.name} => #{container_config.inspect}" }
+      container = Docker::Container.create(container_config)
+      build_log.debug { "Created container #{container.id} "}
+
+      build_log.debug { "Running container for #{image.name}" }
+      container.start({
+        'Binds' => "#{image.workspace.expand_path}:#{image.bind}"
+      })
+
+      begin
+        ## Stream logs
+        container.streaming_logs({'stdout' => true, 'stderr' => true, 'follow' => true, 'timestamps' => false }) do |out,msg|
+          case out
+          when :stdout
+            build_log.info { "++ #{msg.chomp}" }
+          when :stderr
+            build_log.warn { "++ #{msg.chomp}" }        
+          end
+        end
+        ## Wait for finish
+        build_log.debug { "Waiting #{image.timeout} seconds for container #{container.id} to finish building" }
+        status = container.wait(image.timeout)
+      rescue Exception => e
+        build_log.error { "Error building image #{image.name}" }
+        build_log.error { e }
+        build_log.info { "Removing container #{container.id}" }
+        container.stop
+        container.remove
+        raise BuildError.new("Interrupted",image)
+      end
+
+      if status['StatusCode'] != 0
+        build_log.error { "Error building image #{image.name}" }
+        build_log.info { "Removing container #{container.id}" }
+        container.remove
+        raise BuildError.new(status,image)
+      end
+
+      ## Commit Image
+      build_log.debug { "Committing Container #{container.id}" }
+      build_log.debug { "Run Config: #{image.commit_config}" }
+      build_image = container.commit({'run'=>image.commit_config})
+
+      build_log.info { "New Image created: #{build_image.id}"}
+      build_image = Docker::Image.get(build_image.id)
+
+      image.tags.each do |tag|
+        build_log.debug { "Tagging as #{tag}"}
+        t = tag.split(/:/)
+        build_image.tag(:repo => t[0], :tag => t[1])
+      end
+
+      ## Cleanup container
+      container.remove
+    end
+  end
+
+  def inspect
+    <<-eos.gsub(/\n?\s{2,}/,'')
+    #{self.class.name}<
+      @config=#{@config.inspect}
+    >
+    eos
+  end
+end

data/lib/baha/cli.rb

@@ -0,0 +1,69 @@
+require 'thor'
+require 'baha/builder'
+require 'baha/log'
+
+class Baha::CLI < Thor
+  desc "build [options] CONFIG", "Builds all docker images based on the given config"
+  long_desc <<-LONGDESC
+  Reads the CONFIG file and builds all of the docker images in the order they appear.
+  LONGDESC
+  option :logfile, {
+    :required => false,
+    :type => :string,
+    :desc => "Log output to LOGFILE. Omit to log to stdout."
+  }
+  option :debug, {
+    :aliases => :d,
+    :required => false,
+    :type => :boolean,
+    :default => false,
+    :desc => 'Toggle debug logging. Includes verbose.'
+  }
+  option :verbose, {
+    :aliases => :v,
+    :required => false,
+    :type => :boolean,
+    :default => false,
+    :desc => 'Toggle verbose logging'
+  }
+  option :quiet, {
+    :aliases => :q,
+    :required => false,
+    :type => :boolean,
+    :default => false,
+    :desc => 'Surpress all logging'
+  }
+  def build(config)
+    quiet = options[:quiet]
+    unless quiet
+      if options[:logfile].nil? or options[:logfile] == 'STDOUT'
+        Baha::Log.logfile = STDOUT 
+      else
+        Baha::Log.logfile = STDOUT 
+      end
+      if options[:debug]
+        Baha::Log.level = :debug
+      elsif options[:verbose]
+        Baha::Log.level = :info
+      else
+        Baha::Log.level = :error
+      end
+    end
+    begin
+      builder = Baha::Builder.new(config)
+      builder.build!
+    rescue Exception => e
+      unless quiet
+        Baha::Log.for_name("CLI").fatal("Error encountered while building images")
+        Baha::Log.for_name("CLI").fatal(e)
+      end
+      exit 1
+    ensure
+      Baha::Log.close!
+    end
+  end
+  desc "version", "Print version and exit"
+  def version
+    puts "baha " + Baha::VERSION
+  end
+end