athlete 0.0.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 6d3860c8ddee3f9c32fe243c211833b00ae37488
+  data.tar.gz: 47a5c48f169f581acb8c100b9428e69dec334778
+SHA512:
+  metadata.gz: a0ee2dfa941b4e3c504b6530bfe344170da34a8cb6500b05cf062c49832d55c07a05c784472034f8258990e2b9169a8e09df95072d050f3e183431863e1b2a2b
+  data.tar.gz: 17ca485cf67f9ff3afe517b049b3c104feda65f9b30d63ab3e0e3fa8180e025cf1f3f575478f1ba63497ed64f01e59e120541e6d7265ef1448802ddf405fe32d

data/.gitignore

@@ -0,0 +1,22 @@
+*.gem
+*.rbc
+.bundle
+.config
+.yardoc
+Gemfile.lock
+InstalledFiles
+_yardoc
+coverage
+doc/
+lib/bundler/man
+pkg
+rdoc
+spec/reports
+test/tmp
+test/version_tmp
+tmp
+*.bundle
+*.so
+*.o
+*.a
+mkmf.log

data/.ruby-gemset

@@ -0,0 +1 @@
+athlete

data/.ruby-version

@@ -0,0 +1 @@
+2.1.2

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in athlete.gemspec
+gemspec

data/LICENSE.txt

@@ -0,0 +1,22 @@
+Copyright (c) 2015 Andy Sykes
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,297 @@
+# Athlete
+
+Athlete is a Capistrano-like deployment tool for deploying Docker
+containers into [Marathon](https://mesosphere.github.io/marathon/).
+
+Warning: Athlete is at best a beta. It may not understand all Marathon responses, it
+lacks some features, and it doesn't do some basic error checking. Use at your own risk.
+Pull requests welcomed. YMMV.
+
+## Why Athlete exists
+
+If you're a Ruby developer, Capistrano makes things so fantastically easy that you miss it when
+you start to package apps as Docker containers.
+
+Athlete was written to become, internally, a simple Capistrano-like tool that allows developers
+to quickly and easily deploy to our Marathon cluster.
+
+## Features
+
+- Simple DSL for defining Docker builds and deployments
+- Detects deployment failures
+- Simple (~750 LoC)
+- Allows Marathon properties to be set outside deployments or forced
+
+## Installation
+
+Add these lines to your application's Gemfile, in the `:development` group:
+
+    group :development do
+      ... some other gems ...
+      gem 'athlete'
+    end
+
+And then execute:
+
+    bundle install --binstubs
+
+Athlete isn't required at runtime.
+
+## Usage
+
+Athlete performs two actions: building, and deploying.
+
+An Athlete build runs `docker build`, tags the image appropriately
+(including the registry name if you're using a private registry),
+and the pushes it to the registry. This push step can be skipped.
+
+An Athlete deploy uses Marathon's REST API to deploy one of the Docker
+containers built in the build step or any arbitrary Docker container -
+you are not limited to only deploying containers you built.
+
+### Command line usage
+
+(You may want to read the configuration section first, since you
+need a configuration before you can do anything.)
+
+To get help with the CLI:
+
+    bin/athlete help
+
+To build all builds in the configuration file:
+
+    bin/athlete build
+
+To build all builds put not push them:
+
+    bin/athlete build --no-push
+
+To deploy all deployments in the configuration file:
+
+    bin/athlete deploy
+
+### Configuration
+
+Athlete is configured using a simple DSL.
+
+By default, it expects the configuration file to be called `athlete.rb` and
+to be placed in the `config` directory relative to the base of your app.
+
+__Todo: write a command to produce a template athlete.rb file__
+
+A build takes some information about the name you want to give your
+container, how it should be versioned, and what registry it should be pushed to.
+
+A deployment takes some information about where to deploy to, how many
+instances to run, how much CPU and memory resource to allocate, and so on.
+
+Here's an example of a simple image build and deploy:
+
+```ruby
+Athlete::Build.define('my-image') do
+  registry 'my-registry:5000'
+  version '1'
+end
+
+Athlete::Deployment.define('my-app') do
+  marathon_url 'http://marathon:8080'
+  # image_name 'ubuntu:12.04'
+  # command ['/bin/sleep', '600']
+  # arguments ['something']
+  # environment_variables {'RACK_ENV' => 'production'}
+  build_name 'my-image'
+  cpus 1, :override
+  memory 128, :inherit
+  instances 1, :override
+  minimum_health_capacity 0.5, :inherit
+end
+```
+
+This will build an image tagged `my-registry:5000/my-image:1`, remembering
+that the Docker image conventions are: `registry_url:registry_port/image_name:image_version`.
+
+Once built, it will deploy a container based on that image to the Marathon host
+at `http://marathon:8080`. It will request 1 CPU and 128MB of RAM from Marathon,
+and run one instance of the container. See below for details of what the property
+`minimum_health_capacity` does.
+
+#### Build DSL reference
+
+| Property | Required | Description |
+| -------- | -------- | ----------- |
+| registry | no       | The Docker registry to push to - if unspecified, we use the Docker Hub. |
+| version  | yes      | How to version the image; this can be any stringifiable object, or the symbol `:git`, which will version using the output of `git rev-parse HEAD` |
+
+#### Deployment DSL reference
+
+Some properties are defined with an extra parameter which is either
+`:override` or `:inherit` - in the above example, `instances 1, :override`
+is specified with `:override`, and `memory 128, :inherit` is specified with
+`:inherit`.
+
+These properties are ones that can be varied in Marathon through other means.
+For example, you may have a separate 'scaling' system that changes the number
+of instances of a container in response to some parameter. Let's say that it
+has acted to increase the number of instances to 5, and our `athlete.rb` has
+this line in the deployment section:
+
+    instances 1, :override
+
+When you deploy, Athlete will see that Marathon's currently running value for
+instances of the app is 5, and that you set it to 1 in your deployment configuration
+with `:override`, and it will _force_ there to be only 1 instance after the deployment.
+
+This would be non-ideal, since your scaling system is 'authoritative' for this
+property - it decides how many instances should run. Resetting it when you deploy
+could break production!
+
+If you had set the instances property to be `:inherit`, like so:
+
+    instances 1, :inherit
+
+Then when you run a deploy, Athlete will completely ignore the instance value set in
+the configuration file, and just trust whatever is currently set in Marathon.
+
+The only time _all parameters_ will be sent to Marathon is when an app
+does not already exist. In that case, we have to supply some initial values to get the app going.
+
+##### `marathon_url`
+
+__required__: yes
+__override/inherit__: no
+
+The URL to the Marathon REST API endpoint you're using.
+
+##### `build_name`
+
+__required__: yes (if not supplying `image_name`)
+__override/inherit__: no
+
+The build name to get Docker image information from. You must
+reference a build defined earlier in the `athlete.rb` file. The name of a build
+is the string supplied to the `define` call. E.g.
+
+    Athlete::Build.define('my-image') do
+      registry 'my-registry:5000'
+      version '1'
+    end
+
+You would reference this by setting:
+
+    build_name 'my-image'
+
+in your deployment definition.
+
+It is required if you are not specifying `image_name`.
+
+##### `image_name`
+
+__required__: yes (if not supplying `build_name`)
+__override/inherit__: no
+
+The Docker image name to deploy. You must specify the entire image name,
+including whatever tagged version you want. E.g. `ubuntu:12.04`.
+
+It is required if you are not specifying `build_name`.
+
+##### `command`
+
+__required__: no
+__override/inherit__: no
+
+The command to run inside the Docker container. This will override
+the `CMD` section of the Dockerfile.
+
+##### `arguments`
+
+__required__: no
+__override/inherit__: no
+
+Arguments to supply to the container's ENTRYPOINT. This should be
+an array of strings.
+
+##### `cpus`
+
+__required__: yes (on a cold deploy)
+__override/inherit__: yes
+
+CPU resource to request for this app. This can be a fractional value (e.g. 0.1).
+
+##### `memory`
+
+__required__: yes (on a cold deploy)
+__override/inherit__: yes
+
+Memory to request for this app in MB.
+
+##### `environment_variables`
+
+__required__: no
+__override/inherit__: no
+
+Environment variables to pass into the container at startup - must be specified
+as a hash of `ENV_VAR_NAME => ENV_VAR_VALUE`.
+
+##### `instances`
+
+__required__: no
+__override/inherit__: yes
+
+Number of instances of the container to run.
+
+##### `minimum_health_capacity`
+
+__required__: no
+__override/inherit__: yes
+
+This description is taken from the
+[Marathon documentation](https://mesosphere.github.io/marathon/docs/rest-api.html#post-/v2/apps).
+
+> During an upgrade all instances of an application get replaced by a new version.
+> The minimumHealthCapacity defines the minimum number of healthy nodes, that do not sacrifice
+> overall application purpose. It is a number between 0 and 1 which is multiplied with the
+> instance count. The default minimumHealthCapacity is 1, which means no old instance can be stopped,
+> before all new instances are deployed. A value of 0.5 means that an upgrade can be deployed side by side,
+> by taking half of the instances down in the first step, deploy half of the new version and
+> then take the other half down and deploy the rest. A value of 0 means take all instances down
+> immediately and replace with the new application.
+
+## Contributing
+
+1. Fork it ( https://github.com/forward3d/athlete/fork )
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create a new pull request
+
+## Acknowledgements
+
+This repository includes code from the [marathon_client gem](https://github.com/mesosphere/marathon_client).
+It is included with Athlete as Athlete requires some unreleased functionality, and some additional
+functionality, and it is not possible (rightly!) to make gems dependent on git repositories.
+
+In accordance with the MIT license, the license information from marathon_client is included here, 
+and in the `lib/marathon` directory, which are files covered by the below license:
+
+    Copyright (c) 2013 Tobi Knaup
+    
+    MIT License
+    
+    Permission is hereby granted, free of charge, to any person obtaining
+    a copy of this software and associated documentation files (the
+    "Software"), to deal in the Software without restriction, including
+    without limitation the rights to use, copy, modify, merge, publish,
+    distribute, sublicense, and/or sell copies of the Software, and to
+    permit persons to whom the Software is furnished to do so, subject to
+    the following conditions:
+    
+    The above copyright notice and this permission notice shall be
+    included in all copies or substantial portions of the Software.
+    
+    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+    EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+    MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+    NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+    LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+    OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+    WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/Rakefile

@@ -0,0 +1,2 @@
+require "bundler/gem_tasks"
+

data/athlete.gemspec

@@ -0,0 +1,27 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'athlete/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = "athlete"
+  spec.version       = Athlete::VERSION
+  spec.authors       = ["Andy Sykes"]
+  spec.email         = ["github@tinycat.co.uk"]
+  spec.summary       = %q{A deployment tool for Marathon and Mesos}
+  spec.description   = %q{A deployment tool for building Docker containers for Marathon and Mesos}
+  spec.homepage      = "https://github.com/forward3d/athlete"
+  spec.license       = "MIT"
+
+  spec.files         = `git ls-files -z`.split("\x0")
+  spec.executables   = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
+  spec.test_files    = spec.files.grep(%r{^(test|spec|features)/})
+  spec.require_paths = ["lib"]
+
+  spec.add_development_dependency "bundler", "~> 1.6"
+  spec.add_development_dependency "rake"
+  
+  spec.add_dependency "thor", "~> 0.19"
+  spec.add_dependency "httparty", "~> 0.13"
+  spec.add_dependency "multi_json", "~> 1.10"
+end

data/bin/athlete

@@ -0,0 +1,5 @@
+#!/usr/bin/env ruby
+
+require_relative '../lib/athlete'
+
+Athlete::CLI.start()

data/lib/athlete.rb

@@ -0,0 +1,20 @@
+require 'logger'
+require 'open3'
+require 'httparty'
+require 'multi_json'
+require_relative 'marathon/client'
+require_relative 'marathon/response'
+
+require_relative "athlete/version"
+require_relative "athlete/logging"
+require_relative "athlete/utils"
+require_relative "athlete/build"
+require_relative "athlete/cli"
+require_relative "athlete/deployment"
+
+module Athlete
+  class BuildConfigurationInvalid < Exception; end
+  class BuildFailedException < Exception; end
+  class CommandExecutionFailed < Exception; end
+  class ConfigurationInvalidException < Exception; end
+end

data/lib/athlete/build.rb

@@ -0,0 +1,130 @@
+module Athlete
+  class Build
+    include Logging
+    
+    @builds = {}
+    
+    class << self
+      attr_accessor :builds
+    end
+    
+    # Define valid properties
+    @@valid_properties =  %w{
+      name
+      registry
+      version
+    }
+    
+    def initialize
+      @@valid_properties.each do |property|
+        self.class.class_eval {
+          define_method(property) do |arg|
+            instance_variable_set("@#{property}", arg)
+            self.class.class_eval{attr_reader property.to_sym}
+          end
+        }
+      end
+    end
+    
+    def setup_dsl_methods
+      @@valid_properties.each do |property|
+        self.class.class_eval {
+          define_method(property) do |arg|
+            instance_variable_set("@#{property}", arg)
+            self.class.class_eval{attr_reader property.to_sym}
+          end
+        }
+      end
+    end
+    
+    def self.define(name, &block)
+      build = Athlete::Build.new
+      build.name name
+      build.instance_eval(&block)
+      build.fill_default_values
+      @builds[build.name] = build
+    end
+    
+    def fill_default_values
+      @version_method ||= :git_head
+    end
+    
+    def final_image_name
+      @final_image_name ||= @registry.nil? ? "#{@name}:#{determined_version}" : "#{@registry}/#{@name}:#{determined_version}"
+    end
+    
+    def determined_version
+      case @version
+      when :git_head
+        return git_tag
+      else
+        return @version.to_s
+      end
+    end
+    
+    # Figure out the short hash of the current HEAD
+    def git_tag
+      return @git_tag if @git_tag
+      @git_tag = `git rev-parse --short HEAD 2>&1`.chomp
+      if $? != 0
+        raise Athlete::BuildFailedException, "Could not determine git hash of HEAD, output was: #{@git_tag}"
+      end
+      @git_tag
+    end
+    
+    # Create the image name with a specified git tag
+    def image_name_with_specified_version(version)
+      @registry.nil? ? "#{@name}:#{version}" : "#{@registry}/#{@name}:#{version}"
+    end
+    
+    def perform(should_push)
+      build
+      if should_push
+        push
+      else
+        info "Skipping push of image as --no-push was specified"
+      end
+    end
+    
+    # Build the Docker image
+    def build
+      info "Building with image name as: '#{final_image_name}', tagged #{determined_version}"
+      command = "docker build -t #{final_image_name} ."
+      logged_command = get_loglevel == Logger::INFO  ? 'docker build' : command
+      retval = Utils::Subprocess.run command do |stdout, stderr, thread|
+        info "[#{logged_command}] [stdout] #{stdout}"
+        info "[#{logged_command}] [stderr] #{stderr}" if stderr != nil
+      end
+      if retval.exitstatus != 0
+        raise Athlete::CommandExecutionFailed, "The command #{command} exited with non-zero status #{retval.exitstatus}"
+      end
+    end
+    
+    # Push image to remote registry (Docker Hub or private registry)
+    def push
+      if @registry.nil?
+        info "Preparing to push image to the Docker Hub"
+      else
+        info "Preparing to push image to '#{@registry}'"
+      end
+      
+      command = "docker push #{final_image_name}"
+      retval = Utils::Subprocess.run "docker push #{final_image_name}" do |stdout, stderr, thread|
+        info "[#{logged_command}] [stdout] #{stdout}"
+        info "[#{logged_command}] [stderr] #{stderr}" if stderr != nil
+      end
+      
+    end
+    
+    def readable_output
+      lines = []
+      lines << "  Build name: #{@name}"
+      @@valid_properties.sort.each do |property|
+        next if property == 'name'
+        lines << sprintf("    %-10s: %s", property, instance_variable_get("@#{property}")) if instance_variable_get("@#{property}")
+      end
+      puts lines.join("\n")
+    end
+    
+  end
+end