dockit 1.0.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: e774509e3e1c4afab5a44bce1fc3eee91d7d25cf
+  data.tar.gz: daeda4ab4b09f3f9bc7b706fefb77271930ab4fb
+SHA512:
+  metadata.gz: 1d80feb61549460b39bdaa99adc5617ba09bbd75aed6e9bbe63cd20b682215d178a792682a2d202638030251595dbaf5152405b11469523b1db6f0ab3df6e7e2
+  data.tar.gz: 0867cb20ca447aec30f45524d2af9962c8639021a1282e45b7ac94bbc5ed4f1f3fd915aded2498ae38849f96c330e7049fb95066124f98c96c9d3a2503d663cf

data/.gitignore

@@ -0,0 +1,9 @@
+/.bundle/
+/.yardoc
+/Gemfile.lock
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/tmp/

data/Dockerfile

@@ -0,0 +1,17 @@
+FROM cybercode/alpine-ruby:2.2
+ENV DOCKER_VERSION=1.8.2 BIN=/usr/local/bin/docker
+
+RUN apk add -u curl ruby-json && curl -sSL -o $BIN \
+    https://get.docker.com/builds/Linux/x86_64/docker-$DOCKER_VERSION \
+    && chmod +x $BIN && apk del curl
+
+WORKDIR /app
+COPY . .
+RUN apk --update add --virtual build_deps \
+    build-base ruby-dev libc-dev linux-headers git \
+    && gem build dockit.gemspec \
+    && gem install --no-rdoc --no-ri dockit*.gem \
+    && apk del build_deps && rm -rf /app
+
+WORKDIR /
+ENTRYPOINT ["dockit"]

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in dockit.gemspec
+gemspec

data/README.md

@@ -0,0 +1,124 @@
+[docker-api]: https://github.com/swipely/docker-api
+
+# Dockit
+
+`Dockit` is an alternative composer for docker projects. Its (IMHO) advantage is that it is scriptable, and rather than a single yaml configuration file, each service has it's own configuration file (`Dockit.yaml`), as well as an optional `Dockit.rb` which can provide scriptable configuration (as `Thor` subcommands) for any phase of the build and deploy process.
+
+`Dockit` is built on the [Thor](https://github.com/erikhuda/thor) cli and the [docker-api] libraries.
+
+## Installation
+
+```sh
+$ gem install dockit
+```
+
+## Usage
+
+1. Create a top level deployment directory
+2. Create a sub-directory for each service
+3. Create a `Dockit.yaml` for each service (and optionally a `Docit.rb`.)
+4. Optionally, create a top level `Dockit.rb` subcommand file to orchestrate the build and deployment of the services.
+5. Run `dockit` in the root directory for help.
+
+##  `Dockit.yaml`
+
+The sections of the config file map directly to the argument sent by the
+[docker-api] to the corresponding api endpoints (see  [docker api](http://docs.docker.com/reference/api/docker_remote_api_v1.9/).)
+
+The top level sections are:
+
+- `build`
+- `create`
+- `run`
+
+At least one of the sections `build` or `create` are required. If their is no `build` section, the `create` section must specify an `Image` value. Note that most (all?) of the values specified in the `run` section can be specified in the `create: HostConfig:` instead.
+
+### Examples
+
+#### Simple build
+
+```yaml
+build:
+  t: my-image
+```
+
+Executing `dockit build` in the directory containing the file above, will create an image from the `Dockerfile` in the same directory named my-image.
+
+Then executing `dockit start` will create and run a container named `my-image`
+
+[docker hub]: https://registry.hub.docker.com/search?q=library
+
+#### Pre-generated (or [docker hub]) image
+```yaml
+create:
+  Image: postgres
+  name: db
+```
+
+Executing `dockit build` will do nothing. Executing `dockit start` will start run a container named `db` from the local (or [docker hub] postgresql image.
+
+#### Using locals and environment variables
+
+The yaml file is first processed by the `ERB` template library. The "bindings" passed to the template processor can be specified on the command line with the `--locals` (alias `-l`) option. Also, the command line option `--env` (alias `-e`) is passed as `-<env>` For example, given:
+
+```yaml
+create:
+  Image: postgres
+  name: db<%= env %>
+  Env:
+  - MYVAR=<%= myval %>
+```
+
+- `dockit start` will generate an error (myval not defined)
+- `dockit start -l myval:foo` will start a container named `db` with the environment variable `MVAR` set to `foo`.
+- `dockt start -l myval:foo -e test` will start a container named `db-test`
+
+## `Dockit.rb`
+
+The `dockit.rb` file can be used to add new subcommands to the cli on a project-wide or per-service basis. For per-service subcommands, the defined class name should be the "classified" directory name, for the project-wide, it should be named `All`. If the class inherits from `SubCommand` instead of `Thor`, it will inherit two useful methods:
+
+- `invoke_default(service)` will run the same-named (or specified) dockit command on the specified service.
+- `invoke_service(service)` will run the same-named (or specifed) subcommand from the `Dockit.rb` for the specified service.
+- `invoke_git(service, gem=false)` will run the same-name (or specifed) subcommand from the `Dockit.rb` for the specified service after checking out the git repository for the service as a gzip archive to `repos.tar.tz`. By default it used the `master` branch. If `gem` is true, it will checkout the gemfiles as a separate archive (`gemfiles.tar.gz`). Note that the `repos` key must be set in the services yaml file.
+
+For example:
+
+```ruby
+class All < SubCommand
+  desc 'build', 'build all images'
+  def build
+    invoke_service 'app'
+    invoke_default 'db'
+  end
+end
+```
+
+Would run the `build` method from the file `app/Dockit.rb` and then create a second image using the options from `db/Dockit.yaml`.
+
+## DigitalOcean integration
+
+If the `droplet_kit` gem is found at runtime, the `do` service be implemented. see `dockit help do` for available commands.
+
+## Running as a docker image
+
+If you don't want to install ruby, etc., the docker image is also available rom docker hub as `cybercode/dockit`. You can create your own version of the docker image from the source code with `docker build -t dockit .` (or `rake docker`). The `Dockerfile` has `dockit` as the entrypoint, so you can pass dockit arguments on the command-line. e.g.:
+
+```sh
+docker run --rm -v /var/run/docker.sock:/var/run/docker.sock -v $PWD:/app dockit help
+```
+
+## The Github boilerplate
+
+### Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `bin/console` for an interactive prompt that will allow you to experiment.
+
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release` to create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+
+### Contributing
+
+1. Fork it ( https://github.com/cybercode/dockit/fork )
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am [comment]`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create a new Pull Request

data/Rakefile

@@ -0,0 +1,28 @@
+require "bundler/gem_tasks"
+require 'rspec/core/rake_task'
+require 'rake/version_task'
+require 'rdoc/task'
+
+Rake::VersionTask.new
+
+desc 'Run all specs'
+RSpec::Core::RakeTask.new(:spec)
+
+task default: :spec
+task test: :spec
+
+Rake::RDocTask.new do |rdoc|
+  rdoc.rdoc_dir = 'doc'
+  rdoc.main = 'README'
+  rdoc.title = " dockit #{Version.current}"
+  rdoc.rdoc_files.include('README*')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+end
+
+desc "Generate Documentation"
+task doc: :rdoc
+
+desc "Build docker image"
+task :docker do
+  system 'docker build -t dockit .'
+end

data/VERSION

@@ -0,0 +1 @@
+1.0.0

data/bin/console

@@ -0,0 +1,10 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "dockit"
+
+require "pry"
+Pry.start
+
+#require "irb"
+#IRB.start

data/bin/dockit

@@ -0,0 +1,8 @@
+#!/usr/bin/env ruby
+# Allow running w/o deploying
+lib = File.expand_path('../../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+
+require 'dockit/cli'
+
+Default.start(ARGV)

data/bin/setup

@@ -0,0 +1,7 @@
+#!/bin/bash
+set -euo pipefail
+IFS=$'\n\t'
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/dockit.gemspec

@@ -0,0 +1,32 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+
+Gem::Specification.new do |s|
+  s.name          = "dockit"
+  s.version       = File.read("VERSION").chomp # file managed by version gem...
+  s.authors       = ["Rick Frankel"]
+  s.email         = ["dockitk@rickster.com"]
+
+  s.summary       = %q{A configuration manager and builder for docker projects.}
+  s.description   = %q{Dockit is a builder for complex docker projects. Think scriptable docker-composer.}
+  s.homepage      = "https://github.com/cybercode/dockit"
+  s.licenses      = ["MIT"]
+
+  s.files         = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
+  s.test_files    = `git ls-files -z -- spec/*`.split("\x0")
+  s.bindir        = "bin"
+  s.executables   = %w[dockit]
+  s.require_paths = ["lib"]
+
+  s.add_development_dependency "bundler"   , "~> 1.9"
+  s.add_development_dependency "rake"      , "~> 10.4"
+  s.add_development_dependency "rspec"     , "~> 3.2"
+  s.add_development_dependency "simplecov" , "~> 0.10"
+  s.add_development_dependency "pry"       , "~> 0.10"
+
+  s.add_dependency "thor", "~> 0.19"
+  s.add_dependency "dotenv", "~> 2.0"
+  s.add_dependency "docker-api", "~> 1.21"
+  s.add_dependency "version", "~> 1.0"
+end

data/lib/dockit.rb

@@ -0,0 +1,84 @@
+require 'docker'
+require 'yaml'
+require 'pathname'
+require 'version'
+
+require 'dockit/config'
+require 'dockit/service'
+require 'dockit/image'
+require 'dockit/container'
+
+module Dockit
+  is_versioned
+
+  class Log
+    def debug(msg)
+      $stderr.puts "DEBUG: " + msg.join(' ')
+    end
+  end
+
+  # This class encapsulates the environment used in the Dockit cli.
+  # The class has three main attributes:
+  #
+  # root ::
+  #  The (cached) root of the project.
+  # modules ::
+  #  The "modules", a map of +Dockit.rb+ files by directory name
+  # services ::
+  #  The "services",  a map of +Dockit.yaml+ files, by directory name
+  class Env
+    BASENAME = 'Dockit'.freeze
+    attr_reader :services
+    attr_reader :modules
+
+    ##
+    # Initialize services and modules in the project.
+    # depth [Integer] :: How deep to recurse looking for modules/services
+    # debug [Boolean] :: Log +docker-api+ calls.
+    def initialize(depth: 2, debug: false)
+      @root = nil
+      @modules  = find_subcommands(depth)
+      @services = find_services(depth)
+
+      Docker.logger = Dockit::Log.new if debug
+    end
+
+    ##
+    # The (cached) root of the project
+    # Returns [String] :: The absolute path of the project root.
+    def root
+      return @root if @root
+      @root = dir = Dir.pwd
+      begin
+        dir = File.dirname(dir)
+        return @root = dir if File.exist?(File.join(dir, "#{BASENAME}.rb"))
+      end while dir != '/'
+
+      @root
+    end
+
+    private
+    def find_services(depth)
+      find_relative(depth, 'yaml')
+    end
+
+    def find_subcommands(depth)
+      find_relative(depth, 'rb')
+    end
+
+    def find_relative(depth, ext)
+      result = {}
+      (0..depth).each do |i|
+        pat = File.join(root, ['*'] * i, "#{BASENAME}.#{ext}")
+        Pathname.glob(pat).inject(result) do |memo, path|
+          name       = path.dirname.relative_path_from(Pathname.new(root)).to_s
+          name       = 'all' if name == '.'
+          memo[name] = path.to_s
+
+          memo
+        end
+      end
+      result
+    end
+  end
+end

data/lib/dockit/cli.rb

@@ -0,0 +1,249 @@
+require 'io/console'
+require 'thor'
+require 'dockit'
+
+class SubCommand < Thor
+  no_commands do
+    # invoke command against the Dockit.yaml for the given service.
+    def invoke_default(service=nil, cmd: nil, opts: {})
+      service  ||= self.class.to_s.downcase
+      cmd ||= current_command_chain[-1]
+      cmd = "default:#{cmd}"
+      invoke cmd, [service], options.merge(opts)
+      instance_variable_get('@_invocations')[Default].slice!(-1)
+    end
+
+    # invoke the method in the Dockit.rb for the given service.
+    def invoke_service(service, cmd: nil, opts: {})
+      cmd ||= current_command_chain[-1]
+      cmd = "#{service}:#{cmd}"
+
+      say "Invoking #{cmd}"
+      invoke cmd, [], options.merge(opts)
+      instance_variable_get('@_invocations')[Default].slice!(-1)
+    end
+
+    # export git repository before running default command
+    def invoke_git(service, gem=false)
+      invoke_default service, cmd: 'git-build', opts: { gem: gem || options.gem }
+    end
+  end
+end
+
+class Default < Thor
+  DOCKIT_FILE = './Dockit.yaml'.freeze
+  @@root_echoed = false
+
+  class_option :host, type: :string, desc: 'override DOCKER_HOST env variable',
+               default: ENV['DOCKER_HOST'], aliases: ['H']
+  class_option :debug, type: :boolean, desc: "Log docker-api calls"
+  class_option :verbose, type: :boolean, aliases: ['v']
+  class_option :env, type: :string, desc: 'e.g., "test", "staging"', aliases: ['e']
+  class_option :locals, type: :hash, aliases: ['l'],
+               banner: "key:value [key:value ...]",
+               desc: "variables to pass to yaml file."
+
+  def initialize(*args)
+    super
+    ENV['DOCKER_HOST'] = options.host
+    unless @@root_echoed
+      puts "Project root: #{dockit.root}"
+      @@root_echoed=true
+    end
+  end
+
+  no_commands do
+    def _list(what)
+      puts what.to_s.capitalize, dockit.send(what).keys.collect {|s| "  #{s}"}
+    end
+  end
+
+  def help(*args)
+    super
+    if args.count < 1
+      say "Run 'dockit list' to see the complete list of SERVICEs."
+      say "Run 'dockit help COMMAND' to see command specific options."
+    end
+  end
+
+  desc "version", "Print version"
+  def version
+    say "Dockit version #{Dockit::VERSION}"
+  end
+  map %w[--version -V] => :version
+
+  desc "list", "List available services"
+  def list
+    _list :modules
+    _list :services
+  end
+
+  # "run" is a reserved word in thor...
+  desc 'start [SERVICE] [CMD]', 'run a service, optionally override "Cmd"'
+  option :transient, type: :boolean, desc: 'remove container after run'
+  def start(service=nil, *cmd)
+    opts = cmd.length > 0 ? options.merge(create: {Cmd: cmd}) : options
+    exec(service) do |s|
+      s.start(opts)
+    end
+  end
+
+  desc 'sh [SERVICE] [CMD]', 'run an interactive command (default "bash -l")'
+  def sh(service=nil, *cmd)
+    exec(service) do |s|
+      cmd = %w[bash -l] if cmd.length < 1
+
+      # in case image has an entrypoint, use the cmd as the entrypoint
+      (entrypoint, *cmd) = cmd
+      s.start(
+        transient: true,
+        verbose: options.verbose,
+        create: {
+          Entrypoint: [entrypoint],
+          Cmd: cmd,
+          name: 'sh',
+          Tty: true,
+          AttachStdin: true,
+          AttachStdout: true,
+          AttachStderr: true,
+          OpenStdin: true,
+          StdinOnce: true,
+        })
+    end
+  end
+
+  desc "cleanup", "Remove unused containers and images"
+  option :images     , type: :boolean, default: true, desc: "remove danging images"
+  option :containers , type: :boolean, default: true, desc: "remove exited containers"
+  option :force, type: :boolean, default: false, desc: "stop and remove all"
+  def cleanup
+    if options[:containers]
+      Dockit::Container.clean(force: options[:force]) if ask_force('containers')
+    end
+    if options[:images]
+      Dockit::Image.clean(force: options[:force]) if ask_force('images')
+    end
+  end
+
+  desc 'config [SERVICE]', 'show parsed Dockit.yaml config file'
+  def config(service=nil)
+    exec(service) do |s|
+      say s.config.instance_variable_get('@config').to_yaml
+    end
+  end
+  desc 'push REGISTRY [SERVICE]', 'push image for SERVICE to REGSITRY'
+  option :force, type: :boolean, desc: 'overwrite current lastest version'
+  option :tag,  desc: 'repos tag (defaults to "latest")', aliases: ['t']
+  def push(registry, service=nil)
+    exec(service) do |s|
+      s.push(registry, options[:tag], options[:force])
+    end
+  end
+
+  desc 'pull REGISTRY [SERVICE]', 'pull image for SERVICE from REGSITRY'
+  option :force, type: :boolean, desc: 'overwrite current tagged version'
+  option :tag,  desc: 'repos tag (defaults to "latest")', aliases: ['t']
+  def pull(registry, service=nil)
+    exec(service) do |s|
+      s.pull(registry, options[:tag], options[:force])
+    end
+  end
+
+  desc 'build [SERVICE]', "Build image from current directory or service name"
+  def build(service=nil)
+    exec(service) do |s|
+      s.build()
+    end
+  end
+
+  desc 'git-build', 'build from git (gem) repository'
+  option :branch, desc: '<tree-ish> git reference', default: 'master'
+  option :gem, type: :boolean, desc: "update Gemfiles export"
+  def git_build(service=nil)
+    exec(service) do |s|
+      unless repos = s.config.get(:repos)
+        say "'repos' not defined in config file. Exiting",:red
+        exit 1
+      end
+
+      say "Exporting in #{Dir.pwd}", :green
+      say "<- #{repos} #{options.branch}", :green
+      if options.gem
+        # grab the Gemfiles separately for the bundler Dockerfile hack
+        say "-> Gemfiles", :green
+        export(repos, 'gemfile.tar.gz', 'Gemfile*')
+      end
+
+      say "-> repos.tar.gz", :green
+      export(repos, 'repos.tar.gz')
+
+      s.build
+    end
+  end
+
+  private
+  GIT_CMD   = 'git archive -o %s --format tar.gz --remote %s %s %s'.freeze
+  def export(repos, archive, args='')
+    cmd = GIT_CMD % [archive, repos, options.branch, args]
+    say "#{cmd}\n", :blue if options.debug
+
+    unless system(cmd)
+      say "Export error", :red
+      exit 1
+    end
+  end
+
+  def ask_force(type)
+    force = options[:force]
+    say "Removing #{force ? 'ALL' : ''} #{type}...", force ? :red : nil
+
+    return true ### DISABLED# unless force
+    yes? '... Are you sure?'
+  end
+
+  def dockit
+    @@dockit ||= Dockit::Env.new(debug: options[:debug])
+  end
+
+  def exec(service)
+    file = _file(service)
+    if file != DOCKIT_FILE
+      say "Processing #{service}"
+      # problem w/ path length for docker build, so change to local directory
+      Dir.chdir(File.dirname(file))
+    end
+
+    locals = options[:locals]||{}
+    env = options[:env]
+    locals[:env] = env ? "-#{env}" : ""
+    yield Dockit::Service.new(locals: locals)
+  end
+
+  def _file(service)
+    return dockit.services[service] if service && dockit.services[service]
+    return DOCKIT_FILE if File.exist?(DOCKIT_FILE)
+    say "No config file in current directory", :red
+    help
+    exit
+  end
+end
+
+# Add digital ocean support if droplet_kit is installed
+begin
+  require 'dockit/digitalocean'
+  Default.desc 'do', 'manage docker server'
+  Default.subcommand 'do', DO
+rescue LoadError
+end
+
+# it would be nice to do this in the initialization method of the Default
+# class above, but it hasn't been instantiated yet when invoking a subcommand.
+Dockit::Env.new.modules.each do |k, v|
+  begin
+    require v
+    Default.desc k, "#{k} submodule, see 'help #{k}'"
+    Default.subcommand k, Module.const_get(k.capitalize)
+  rescue NameError => e
+    STDERR.puts "Can't load #{v}: #{e}"
+  end
+end