centurion 1.6.0 → 1.8.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 37e2e34bed03a54303fbb97f00a7ea41ca703c92
+  data.tar.gz: 9ce533114ecddb1fcf67108b3d3b6e5cb722aa5b
+SHA512:
+  metadata.gz: a26cced05248fc737b51aa7b550086d7284edc6635afecc418e5835acbbffe47a5b164a405775044008d56eda57afea3470345cf2e685b4a31aac46c876e81de
+  data.tar.gz: 67d3855249e3ed55a2a0c7431fbf1bc01c0cdc22e9d731f572f75d4e133bfe736c419f24c8f9fd70a52c076786045f859ec95a5db3d841c9f434e26eaa8c7403

data/CONTRIBUTORS.md

@@ -11,6 +11,7 @@ Your name could be here!
  * [Matt Sanford][mzsanford]
  * [Suren Karapetyan][skarap]
  * [Jon Wood][jellybob]
+ * [Mark Borcherding][markborcherding]
 
 Pre-release
 -----------
@@ -71,3 +72,4 @@ Contributor                                   | Commits    | Additions | Deletio
 [mzsanford]: https://github.com/mzsanford
 [skarap]: https://github.com/skarap
 [jellybob]: https://github.com/jellybob
+[markborcherding]: https://github.com/markborcherding

data/README.md

@@ -15,7 +15,7 @@ tools directly so you can use anything they currently support via the normal
 registry mechanism.
 
 If you haven't been using a registry, you should read up on how to do that
-before trying to deploy anything with Centurion.  
+before trying to deploy anything with Centurion.
 
 Commercial Docker Registry Providers:
 - Docker, Inc. [provides repositories](https://index.docker.io/), and hosts the
@@ -37,6 +37,8 @@ one roll-up commit of all our internal code. But all internal development will
 now be on public GitHub. See the CONTRIBUTORS file for the contributors to the
 original internal project.
 
+The **current stable release** is 1.6.0.
+
 Installation
 ------------
 
@@ -59,8 +61,8 @@ the commands should just work now.
 Configuration
 -------------
 
-Centurion expects to find configuration tasks in the current working directory.
-Soon it will also support reading configuration from etcd.
+Centurion expects to find configuration tasks in the current working directory
+tree.
 
 We recommend putting all your configuration for multiple applications into a
 single repo rather than spreading it around by project. This allows a central
@@ -148,7 +150,7 @@ You can cause your container to be started with a specific DNS server
 IP address (the equivalent of `docker run --dns 172.17.42.1 ...`) like this:
 ```ruby
   task :production => :common do
-    set :custom_dns, '172.17.42.1'
+    set :dns, '172.17.42.1'
     # ...
   end
 ```
@@ -176,17 +178,47 @@ With this, the container will be named something like `backend-4f692997`.
 ### Container Hostnames
 
 If you don't specify a hostname to use inside your container, the container
-will be given the hostname of the Docker server. This probably is good for
+will be given a hostname matching the container ID. This probably is good for
 a lot of situations, but it might not be good for yours. If you need to have
-a specific hostname, you can cause Centurion to do that:
+a specific hostname, you can ask Centurion to do that:
 
 ```ruby
 set :container_hostname, 'yourhostname'
 ```
 
-This is currently pretty inflexible in that the hostname will now be the same
-on all of your hosts. A possible future addition is the ability to pass
-a block to be evaluated on each host.
+That will make *all* of your containers named 'yourhostname'. If you want to do
+something more dynamic, you can pass a `Proc` or a lambda like this:
+
+```ruby
+set :container_hostname, ->(hostname) { "#{hostname}.example.com" }
+```
+
+The lambda will be passed the current server's hostname. So, this example will
+cause ".example.com" to be appended to the hostname of each Docker host during
+deployment.
+
+*If you want to restore the old behavior from Centurion 1.6.0* and earlier, you
+can do the following:
+
+```ruby
+set :container_hostname, ->(hostname) { hostname }
+```
+
+That will cause the container hostname to match the server's hostname.
+
+### Network modes
+
+You may specify the network mode you would like a container to use via:
+
+```ruby
+set :network_mode, 'networkmode'
+```
+
+Docker (and therefore Centurion) supports one of `bridge` (the default), `host`,
+and `container:<container-id>` for this argument.
+
+*Note:* While `host_port` remains required, the mappings specified in it are
+*ignored* when using `host` and `container...` network modes.
 
 ### CGroup Resource Constraints
 
@@ -205,6 +237,27 @@ cpu_shares 512
 For more information on Docker's CGroup limits see [the Docker
 docs](https://docs.docker.com/reference/run/#runtime-constraints-on-cpu-and-memory).
 
+### Adding Extended Capabilities
+
+Additional kernel capabilities may be granted to containers, permitting them
+device access they do not normally have. You may specify these as follows:
+
+```ruby
+add_capability 'SOME_CAPABILITY'
+add_capability 'ANOTHER_CAPABILITY'
+drop_capability 'SOMEOTHER_CAPABILITY'
+```
+
+You may also ask for all but a few capabilities as follows:
+
+```ruby
+add_capability 'ALL'
+drop_capability 'SOME_CAPABILITY'
+```
+
+For more information on which kernel capabilities may be specified, see the
+[Docker docs](https://docs.docker.com/reference/run/#runtime-privilege-linux-capabilities-and-lxc-configuration).
+
 ### Interpolation
 
 Currently there a couple of special strings for interpolation that can be added
@@ -229,7 +282,8 @@ You just need to enable the tls mode as the following:
   end
 ```
 
-Centurion will only set the `--tlsverify` to true and Docker will read your certificate from the `~/.docker/` path.
+Centurion will only set the `--tlsverify` to true and Docker will read your
+certificate from the `~/.docker/` path.
 
 #### Your certificate files are not in `~/.docker/`
 
@@ -315,9 +369,9 @@ are the same everywhere. Settings are per-project.
    an individual container to come up before giving up as a failure. Defaults
    to 24 attempts.
  * `rolling_deploy_skip_ports` => Either a single port, or an array of ports
-   that should be skipped for status checks. By default status checking assumes 
-   an HTTP server is on the other end and if you are deploying a container where some 
-   ports are not HTTP services, this allows you to only health check the ports 
+   that should be skipped for status checks. By default status checking assumes
+   an HTTP server is on the other end and if you are deploying a container where some
+   ports are not HTTP services, this allows you to only health check the ports
    that are. The default is an empty array. If you have non-HTTP services that you
    want to check, see Custom Health Checks in the previous section.
 
@@ -375,14 +429,14 @@ Centurion needs to have access to some registry in order to pull images to
 remote Docker servers. This needs to be either a hosted registry (public or
 private), or [Dogestry](https://github.com/dogestry/dogestry).
 
-#### Access to the registry 
+#### Access to the registry
 
 If you are not using either Dogestry, or the public registry, you may need to
 provide authentication credentials.  Centurion needs to access the Docker
 registry hosting your images directly to retrive image ids and tags. This is
 supported in both the config file and also as command line arguments.
 
-The command line arguments are: 
+The command line arguments are:
  * `--registry-user` => The username to pass to the registry
  * `--registry-password` => The password
 
@@ -423,9 +477,22 @@ namespace :environment do
 end
 ```
 
+**TLS with Dogestry**: Because this involves a little passing around of both
+settings and environment variables, there are a couple of things to verify to
+make sure everything is passed properly between Centurion and Dogestry. If your
+keys have the default names and are in located in the path represented by
+`DOCKER_CERT_PATH` in your environment, this should just work. Otherwise you'll
+need to be sure to `set :tlsverify, true` and *also* set the TLS cert names as
+decribed above. 
+
 Development
 -----------
 
+Centurion supports a few features to make development easier when
+building your deployment tooling or debugging your containers.
+
+#### Overriding Environment Variables
+
 Sometimes when you're doing development you want to try out some configuration
 settings in environment variables that aren't in the config yet. Or perhaps you
 want to override existing settings to test with. You can provide the
@@ -441,6 +508,24 @@ this functionality for production deployments. It will work, but it means that
 the config is not the whole source of truth for your container configuration.
 Caveat emptor.
 
+#### Exporting Environment Variables Locally
+
+Sometimes you need to test how your code works inside the container and you
+need to have all of your configuration exported. Centurion includes an action
+that will let you do that. It exports all of your environment settings for the
+environment you specify. It then partially sanitizes them to preserve things
+like `rbenv` settings. Then it executes `/bin/bash` locally.
+
+The action is named `dev:export_only` and you call it like this:
+
+```bash
+$ bundle exec centurion -e development -p testing_project -a dev:export_only
+$ bundle exec rake spec
+```
+
+It's important to note that the second line is actually being invoked with new
+environment exported.
+
 Future Additions
 ----------------
 

data/lib/centurion/deploy.rb

@@ -5,12 +5,9 @@ module Centurion; end
 
 module Centurion::Deploy
   FAILED_CONTAINER_VALIDATION = 100
-  INVALID_CGROUP_CPUSHARES_VALUE = 101
-  INVALID_CGROUP_MEMORY_VALUE = 102
 
-  def stop_containers(target_server, port_bindings, timeout = 30)
-    public_port    = public_port_for(port_bindings)
-    old_containers = target_server.find_containers_by_public_port(public_port)
+  def stop_containers(target_server, service, timeout = 30)
+    old_containers = target_server.find_containers_by_public_port(service.public_ports.first)
     info "Stopping container(s): #{old_containers.inspect}"
 
     old_containers.each do |old_container|
@@ -19,10 +16,10 @@ module Centurion::Deploy
     end
   end
 
-  def wait_for_health_check_ok(health_check_method, target_server, port, endpoint, image_id, tag, sleep_time=5, retries=12)
+  def wait_for_health_check_ok(health_check_method, target_server, container_id, port, endpoint, image_id, tag, sleep_time=5, retries=12)
     info 'Waiting for the port to come up'
     1.upto(retries) do
-      if container_up?(target_server, port) && health_check_method.call(target_server, port, endpoint)
+      if container_up?(target_server, container_id) && health_check_method.call(target_server, port, endpoint)
         info 'Container is up!'
         break
       end
@@ -37,20 +34,13 @@ module Centurion::Deploy
     end
   end
 
-  def container_up?(target_server, port)
+  def container_up?(target_server, container_id)
     # The API returns a record set like this:
     #[{"Command"=>"script/run ", "Created"=>1394470428, "Id"=>"41a68bda6eb0a5bb78bbde19363e543f9c4f0e845a3eb130a6253972051bffb0", "Image"=>"quay.io/newrelic/rubicon:5f23ac3fad7979cd1efdc9295e0d8c5707d1c806", "Names"=>["/happy_pike"], "Ports"=>[{"IP"=>"0.0.0.0", "PrivatePort"=>80, "PublicPort"=>8484, "Type"=>"tcp"}], "Status"=>"Up 13 seconds"}]
 
-    running_containers = target_server.find_containers_by_public_port(port)
-    container = running_containers.pop
+    container = target_server.find_container_by_id(container_id)
 
-    unless running_containers.empty?
-      # This _should_ never happen, but...
-      error "More than one container is bound to port #{port} on #{target_server}!"
-      return false
-    end
-
-    if container && container['Ports'].any? { |bind| bind['PublicPort'].to_i == port.to_i }
+    if container
       info "Found container up for #{Time.now.to_i - container['Created'].to_i} seconds"
       return true
     end
@@ -74,149 +64,61 @@ module Centurion::Deploy
     false
   end
 
-  def is_a_uint64?(value)
-    result = false
-    if !value.is_a? Integer
-      return result
-    end
-    if value < 0 || value > 0xFFFFFFFFFFFFFFFF
-      return result
-    end
-    return true
-  end
-
   def wait_for_load_balancer_check_interval
     sleep(fetch(:rolling_deploy_check_interval, 5))
   end
 
-  def cleanup_containers(target_server, port_bindings)
-    public_port    = public_port_for(port_bindings)
-    old_containers = target_server.old_containers_for_port(public_port)
+  def cleanup_containers(target_server, service)
+    old_containers = target_server.old_containers_for_name(service.name)
     old_containers.shift(2)
 
-    info "Public port #{public_port}"
+    info "Service name #{service.name}"
     old_containers.each do |old_container|
       info "Removing old container #{old_container['Id'][0..7]} (#{old_container['Names'].join(',')})"
       target_server.remove_container(old_container['Id'])
     end
   end
 
-  def container_config_for(target_server, image_id, port_bindings=nil, env_vars=nil, volumes=nil, command=nil, memory=nil, cpu_shares=nil)
-
-    if memory && ! is_a_uint64?(memory)
-      error "Invalid value for CGroup memory constraint: #{memory}, value must be a between 0 and 18446744073709551615"
-      exit(INVALID_CGROUP_MEMORY_VALUE)
-    end
-
-    if cpu_shares && ! is_a_uint64?(cpu_shares)
-      error "Invalid value for CGroup CPU constraint: #{cpu_shares}, value must be between 0 and 18446744073709551615"
-      exit(INVALID_CGROUP_CPUSHARES_VALUE)
-    end
+  def hostname_proc
+    hostname = fetch(:container_hostname)
+    return nil if hostname.nil?
 
-    container_config = {
-      'Image'        => image_id,
-      'Hostname'     => fetch(:container_hostname, target_server.hostname),
-    }
-
-    container_config.merge!('Cmd' => command) if command
-    container_config.merge!('Memory' => memory) if memory
-    container_config.merge!('CpuShares' => cpu_shares) if cpu_shares
-
-    if port_bindings
-      container_config['ExposedPorts'] ||= {}
-      port_bindings.keys.each do |port|
-        container_config['ExposedPorts'][port] = {}
-      end
-    end
-
-    if env_vars
-      container_config['Env'] = env_vars.map do |k,v|
-        "#{k}=#{interpolate_var(v, target_server)}"
-      end
-    end
-
-    if volumes
-      container_config['Volumes'] = volumes.inject({}) do |memo, v|
-        memo[v.split(/:/).last] = {}
-        memo
-      end
-      container_config['VolumesFrom'] = 'parent'
+    if hostname.respond_to?(:call)
+      hostname
+    else
+      ->(h) { hostname }
     end
-
-    container_config
-  end
-
-  def start_new_container(target_server, image_id, port_bindings, volumes, env_vars=nil, command=nil, memory=nil, cpu_shares=nil)
-    container_config = container_config_for(target_server, image_id, port_bindings, env_vars, volumes, command, memory, cpu_shares)
-    start_container_with_config(target_server, volumes, port_bindings, container_config)
-  end
-
-  def launch_console(target_server, image_id, port_bindings, volumes, env_vars=nil)
-    container_config = container_config_for(target_server, image_id, port_bindings, env_vars, volumes, ['/bin/bash']).merge(
-      'AttachStdin' => true,
-      'Tty'         => true,
-      'OpenStdin'   => true,
-    )
-
-    container = start_container_with_config(target_server, volumes, port_bindings, container_config)
-
-    target_server.attach(container['Id'])
   end
 
-  private
-
-  # By default we always use on-failure policy.
-  def build_host_config_restart_policy(host_config={})
-    host_config['RestartPolicy'] = {}
+  def start_new_container(server, service, restart_policy)
+    container_config = service.build_config(server.hostname, &hostname_proc)
+    info "Creating new container for #{container_config['Image'][0..7]}"
+    container = server.create_container(container_config, service.name)
 
-    restart_policy_name = fetch(:restart_policy_name) || 'on-failure'
-    restart_policy_name = 'on-failure' unless ["always", "on-failure", "no"].include?(restart_policy_name)
+    host_config = service.build_host_config(restart_policy)
 
-    restart_policy_max_retry_count = fetch(:restart_policy_max_retry_count) || 10
+    info "Starting new container #{container['Id'][0..7]}"
+    server.start_container(container['Id'], host_config)
 
-    host_config['RestartPolicy']['Name'] = restart_policy_name
-    host_config['RestartPolicy']['MaximumRetryCount'] = restart_policy_max_retry_count if restart_policy_name == 'on-failure'
+    info "Inspecting new container #{container['Id'][0..7]}:"
+    (server.inspect_container(container['Id']) || {}).each_pair do |key,value|
+      info "\t#{key} => #{value.inspect}"
+    end
 
-    host_config
+    container
   end
 
-  def start_container_with_config(target_server, volumes, port_bindings, container_config)
+  def launch_console(server, service)
+    container_config = service.build_console_config(server.hostname)
     info "Creating new container for #{container_config['Image'][0..7]}"
-    new_container = target_server.create_container(container_config, fetch(:name))
 
-    host_config = {}
+    container = server.create_container(container_config, service.name)
 
-    # Map some host volumes if needed
-    host_config['Binds'] = volumes if volumes && !volumes.empty?
+    host_config = service.build_host_config
 
-    # Bind the ports
-    host_config['PortBindings'] = port_bindings
+    info "Starting new container #{container['Id'][0..7]}"
+    server.start_container(container['Id'], host_config)
 
-    # DNS if specified
-    dns = fetch(:custom_dns)
-    host_config['Dns'] = dns if dns
-
-    # Restart Policy
-    host_config = build_host_config_restart_policy(host_config)
-
-    info "Starting new container #{new_container['Id'][0..7]}"
-    target_server.start_container(new_container['Id'], host_config)
-
-    info "Inspecting new container #{new_container['Id'][0..7]}:"
-    info target_server.inspect_container(new_container['Id'])
-
-    new_container
+    server.attach(container['Id'])
   end
-
-  def interpolate_var(val, target_server)
-    val.gsub('%DOCKER_HOSTNAME%', target_server.hostname)
-      .gsub('%DOCKER_HOST_IP%', host_ip(target_server.hostname))
-  end
-
-  def host_ip(hostname)
-    @host_ip ||= {}
-    return @host_ip[hostname] if @host_ip.has_key?(hostname)
-    @host_ip[hostname] = Socket.getaddrinfo(hostname, nil).first[2]
-  end
-
 end