centurion 1.3.1 → 1.4.0
Sign up to get free protection for your applications and to get access to all the features.
- data/README.md +64 -5
- data/bin/centurion +2 -2
- data/lib/centurion/deploy.rb +3 -3
- data/lib/centurion/deploy_dsl.rb +21 -5
- data/lib/centurion/docker_server.rb +6 -3
- data/lib/centurion/docker_server_group.rb +5 -3
- data/lib/centurion/docker_via_api.rb +33 -9
- data/lib/centurion/docker_via_cli.rb +45 -5
- data/lib/centurion/dogestry.rb +6 -19
- data/lib/centurion/version.rb +1 -1
- data/lib/tasks/deploy.rake +12 -17
- data/spec/capistrano_dsl_spec.rb +1 -1
- data/spec/deploy_dsl_spec.rb +1 -1
- data/spec/deploy_spec.rb +66 -35
- data/spec/docker_server_group_spec.rb +1 -1
- data/spec/docker_server_spec.rb +2 -2
- data/spec/docker_via_api_spec.rb +211 -76
- data/spec/docker_via_cli_spec.rb +74 -25
- data/spec/dogestry_spec.rb +14 -7
- data/spec/support/matchers/capistrano_dsl_matchers.rb +2 -2
- metadata +2 -2
data/README.md
CHANGED
@@ -153,12 +153,74 @@ IP address (the equivalent of `docker run --dns 172.17.42.1 ...`) like this:
|
|
153
153
|
end
|
154
154
|
```
|
155
155
|
|
156
|
+
### Container Names
|
157
|
+
|
158
|
+
This is the name that shows up in the `docker ps` output. It's the name
|
159
|
+
of the container, not the hostname inside the container.
|
160
|
+
|
161
|
+
If you want to name your container, use the `name` setting. The
|
162
|
+
actual name for the created container will have a random hex string
|
163
|
+
appended, to avoid name conflicts when you repeatedly deploy an image:
|
164
|
+
```ruby
|
165
|
+
task :common do
|
166
|
+
set :name, 'backend'
|
167
|
+
# ...
|
168
|
+
end
|
169
|
+
```
|
170
|
+
With this, the container will be named something like `backend-4f692997`.
|
171
|
+
|
172
|
+
### Container Hostnames
|
173
|
+
|
174
|
+
If you don't specify a hostname to use inside your container, the container
|
175
|
+
will be given the hostname of the Docker server. This probably is good for
|
176
|
+
a lot of situations, but it might not be good for yours. If you need to have
|
177
|
+
a specific hostname, you can cause Centurion to do that:
|
178
|
+
|
179
|
+
```ruby
|
180
|
+
set :container_hostname, 'yourhostname'
|
181
|
+
```
|
182
|
+
|
183
|
+
This is currently pretty inflexible in that the hostname will now be the same
|
184
|
+
on all of your hosts. A possible future addition is the ability to pass
|
185
|
+
a block to be evaluated on each host.
|
186
|
+
|
156
187
|
### Interpolation
|
157
188
|
|
158
189
|
Currently there is one special string for interpolation that can be added to
|
159
190
|
any `env_var` value in the DSL. `%DOCKER_HOSTNAME%` will be replaced with the
|
160
191
|
current server's hostname in the environment variable at deployment time.
|
161
192
|
|
193
|
+
### Use TLS certificate
|
194
|
+
|
195
|
+
Centurion can use your certificate in order to execute the Docker commands.
|
196
|
+
In order to do so you have 2 choices.
|
197
|
+
|
198
|
+
#### Your certificate files are in ~/.docker/
|
199
|
+
|
200
|
+
You just need to enable the tls mode as the following:
|
201
|
+
|
202
|
+
```ruby
|
203
|
+
task :production => :common do
|
204
|
+
set :tls, true
|
205
|
+
# ...
|
206
|
+
end
|
207
|
+
```
|
208
|
+
|
209
|
+
Centurion will only set the `--tlsverify` to true and Docker will read your certificate from the `~/.docker/` path.
|
210
|
+
|
211
|
+
#### Your certificate files are not in ~/.docker/
|
212
|
+
|
213
|
+
Given your files are in `/usr/local/certs/`
|
214
|
+
You have to set the following keys:
|
215
|
+
|
216
|
+
```ruby
|
217
|
+
task :production => :common do
|
218
|
+
set :tlscacert, '/usr/local/certs/ca.pem'
|
219
|
+
set :tlscert, '/usr/local/certs/ssl.crt'
|
220
|
+
set :tlskey, '/usr/local/certs/ssl.key'
|
221
|
+
# ...
|
222
|
+
end
|
223
|
+
```
|
162
224
|
|
163
225
|
Deploying
|
164
226
|
---------
|
@@ -276,7 +338,7 @@ These correspond to the following settings:
|
|
276
338
|
#### Alternative Docker Registry
|
277
339
|
|
278
340
|
Centurion normally uses the built-in registry support in the Docker daemon to
|
279
|
-
handle pushing and pulling images.But Centurion also has the ability to use
|
341
|
+
handle pushing and pulling images. But Centurion also has the ability to use
|
280
342
|
external tooling to support hosting your registry on Amazon S3. That tooling is
|
281
343
|
from a project called [Dogestry](https://github.com/newrelic-forks/dogestry).
|
282
344
|
We have recently improved that tooling substantially in coordination with the
|
@@ -287,12 +349,10 @@ with Amazon S3 to provide reliable hosting of images. Setting Centurion up to
|
|
287
349
|
use Dogestry is pretty trivial:
|
288
350
|
|
289
351
|
1. Install Dogestry binaries on the client from which Dogestry is run.
|
290
|
-
Binaries are provided in the GitHub release.
|
352
|
+
Binaries are provided in the [GitHub release](https://github.com/newrelic-forks/dogestry).
|
291
353
|
1. Add the settings necessary to get Centurion to pull from Dogestry. A config
|
292
354
|
example is provided below:
|
293
355
|
|
294
|
-
See example below to use `dogestry`:
|
295
|
-
|
296
356
|
```ruby
|
297
357
|
namespace :environment do
|
298
358
|
task :common do
|
@@ -312,7 +372,6 @@ We're currently looking at the following feature additions:
|
|
312
372
|
|
313
373
|
* [etcd](https://github.com/coreos/etcd) integration for configs and discovery
|
314
374
|
* Add the ability to show all the available tasks on the command line
|
315
|
-
* Certificate authentication
|
316
375
|
* Customized tasks
|
317
376
|
* Dynamic host allocation to a pool of servers
|
318
377
|
|
data/bin/centurion
CHANGED
@@ -29,7 +29,7 @@ opts = Trollop::options do
|
|
29
29
|
opt :tag, 'tag (latest...)', type: String, required: false, short: '-t'
|
30
30
|
opt :hosts, 'hosts, comma separated', type: String, required: false, short: '-h'
|
31
31
|
opt :docker_path, 'path to docker executable (default: docker)', type: String, default: 'docker', short: '-d'
|
32
|
-
opt :
|
32
|
+
opt :no_pull, 'Skip the pull_image step', type: :flag, default: false, short: '-n'
|
33
33
|
opt :registry_user, 'user for registry auth (default: nil)', type: String, default: nil, short: :none
|
34
34
|
opt :registry_password,'password for registry auth (default: nil)', type: String, default: nil, short: :none
|
35
35
|
end
|
@@ -62,7 +62,7 @@ set :docker_registry, Centurion::DockerRegistry::OFFICIAL_URL unless any?(:docke
|
|
62
62
|
# Specify a path to docker executable
|
63
63
|
set :docker_path, opts[:docker_path]
|
64
64
|
|
65
|
-
set :no_pull, opts[:
|
65
|
+
set :no_pull, opts[:no_pull_given]
|
66
66
|
set :registry_user, opts[:registry_user] if opts[:registry_user]
|
67
67
|
set :registry_password, opts[:registry_password] if opts[:registry_password]
|
68
68
|
|
data/lib/centurion/deploy.rb
CHANGED
@@ -87,10 +87,10 @@ module Centurion::Deploy
|
|
87
87
|
end
|
88
88
|
end
|
89
89
|
|
90
|
-
def container_config_for(target_server, image_id, port_bindings=nil, env_vars=nil, volumes=nil, command=nil)
|
90
|
+
def container_config_for(target_server, image_id, port_bindings=nil, env_vars=nil, volumes=nil, command=nil, hostname=nil)
|
91
91
|
container_config = {
|
92
92
|
'Image' => image_id,
|
93
|
-
'Hostname' => target_server.hostname,
|
93
|
+
'Hostname' => fetch(:container_hostname, target_server.hostname),
|
94
94
|
}
|
95
95
|
|
96
96
|
container_config.merge!('Cmd' => command) if command
|
@@ -140,7 +140,7 @@ module Centurion::Deploy
|
|
140
140
|
|
141
141
|
def start_container_with_config(target_server, volumes, port_bindings, container_config)
|
142
142
|
info "Creating new container for #{container_config['Image'][0..7]}"
|
143
|
-
new_container = target_server.create_container(container_config)
|
143
|
+
new_container = target_server.create_container(container_config, fetch(:name))
|
144
144
|
|
145
145
|
host_config = {}
|
146
146
|
# Map some host volumes if needed
|
data/lib/centurion/deploy_dsl.rb
CHANGED
@@ -3,9 +3,7 @@ require 'uri'
|
|
3
3
|
|
4
4
|
module Centurion::DeployDSL
|
5
5
|
def on_each_docker_host(&block)
|
6
|
-
|
7
|
-
hosts.each { |host| block.call(host) }
|
8
|
-
end
|
6
|
+
build_server_group.tap { |hosts| hosts.each { |host| block.call(host) } }
|
9
7
|
end
|
10
8
|
|
11
9
|
def env_vars(new_vars)
|
@@ -63,8 +61,7 @@ module Centurion::DeployDSL
|
|
63
61
|
end
|
64
62
|
|
65
63
|
def get_current_tags_for(image)
|
66
|
-
|
67
|
-
hosts.inject([]) do |memo, target_server|
|
64
|
+
build_server_group.inject([]) do |memo, target_server|
|
68
65
|
tags = target_server.current_tags_for(image)
|
69
66
|
memo += [{ server: target_server.hostname, tags: tags }] if tags
|
70
67
|
memo
|
@@ -77,6 +74,11 @@ module Centurion::DeployDSL
|
|
77
74
|
|
78
75
|
private
|
79
76
|
|
77
|
+
def build_server_group
|
78
|
+
hosts, docker_path = fetch(:hosts, []), fetch(:docker_path)
|
79
|
+
Centurion::DockerServerGroup.new(hosts, docker_path, build_tls_params)
|
80
|
+
end
|
81
|
+
|
80
82
|
def add_to_bindings(host_ip, container_port, port, type='tcp')
|
81
83
|
set(:port_bindings, fetch(:port_bindings, {}).tap do |bindings|
|
82
84
|
binding = { 'HostPort' => port.to_s }.tap do |b|
|
@@ -100,4 +102,18 @@ module Centurion::DeployDSL
|
|
100
102
|
raise ArgumentError.new("Options must contain #{missing.inspect}")
|
101
103
|
end
|
102
104
|
end
|
105
|
+
|
106
|
+
def tls_paths_available?
|
107
|
+
Centurion::DockerViaCli.tls_keys.all? { |key| fetch(key).present? }
|
108
|
+
end
|
109
|
+
|
110
|
+
def build_tls_params
|
111
|
+
return {} unless fetch(:tlsverify)
|
112
|
+
{
|
113
|
+
tls: fetch(:tlsverify || tls_paths_available?),
|
114
|
+
tlscacert: fetch(:tlscacert),
|
115
|
+
tlscert: fetch(:tlscert),
|
116
|
+
tlskey: fetch(:tlskey)
|
117
|
+
}
|
118
|
+
end
|
103
119
|
end
|
@@ -18,10 +18,11 @@ class Centurion::DockerServer
|
|
18
18
|
:old_containers_for_port, :remove_container
|
19
19
|
def_delegators :docker_via_cli, :pull, :tail, :attach
|
20
20
|
|
21
|
-
def initialize(host, docker_path)
|
21
|
+
def initialize(host, docker_path, tls_params = {})
|
22
22
|
@docker_path = docker_path
|
23
23
|
@hostname, @port = host.split(':')
|
24
24
|
@port ||= '2375'
|
25
|
+
@tls_params = tls_params
|
25
26
|
end
|
26
27
|
|
27
28
|
def current_tags_for(image)
|
@@ -44,11 +45,13 @@ class Centurion::DockerServer
|
|
44
45
|
private
|
45
46
|
|
46
47
|
def docker_via_api
|
47
|
-
@docker_via_api ||= Centurion::DockerViaApi.new(@hostname, @port
|
48
|
+
@docker_via_api ||= Centurion::DockerViaApi.new(@hostname, @port,
|
49
|
+
@tls_params)
|
48
50
|
end
|
49
51
|
|
50
52
|
def docker_via_cli
|
51
|
-
@docker_via_cli ||= Centurion::DockerViaCli.new(@hostname, @port,
|
53
|
+
@docker_via_cli ||= Centurion::DockerViaCli.new(@hostname, @port,
|
54
|
+
@docker_path, @tls_params)
|
52
55
|
end
|
53
56
|
|
54
57
|
def parse_image_tags_for(running_containers)
|
@@ -8,10 +8,12 @@ class Centurion::DockerServerGroup
|
|
8
8
|
include Centurion::Logging
|
9
9
|
|
10
10
|
attr_reader :hosts
|
11
|
-
|
12
|
-
def initialize(hosts, docker_path)
|
11
|
+
|
12
|
+
def initialize(hosts, docker_path, tls_params = {})
|
13
13
|
raise ArgumentError.new('Bad Host list!') if hosts.nil? || hosts.empty?
|
14
|
-
@hosts = hosts.map
|
14
|
+
@hosts = hosts.map do |hostname|
|
15
|
+
Centurion::DockerServer.new(hostname, docker_path, tls_params)
|
16
|
+
end
|
15
17
|
end
|
16
18
|
|
17
19
|
def each(&block)
|
@@ -1,12 +1,14 @@
|
|
1
1
|
require 'excon'
|
2
2
|
require 'json'
|
3
3
|
require 'uri'
|
4
|
+
require 'securerandom'
|
4
5
|
|
5
6
|
module Centurion; end
|
6
7
|
|
7
8
|
class Centurion::DockerViaApi
|
8
|
-
def initialize(hostname, port)
|
9
|
-
@
|
9
|
+
def initialize(hostname, port, tls_args = {})
|
10
|
+
@tls_args = tls_args # Required by tls_enable?
|
11
|
+
@base_uri = "http#{'s' if tls_enable?}://#{hostname}:#{port}"
|
10
12
|
|
11
13
|
configure_excon_globally
|
12
14
|
end
|
@@ -14,7 +16,7 @@ class Centurion::DockerViaApi
|
|
14
16
|
def ps(options={})
|
15
17
|
path = "/v1.7/containers/json"
|
16
18
|
path += "?all=1" if options[:all]
|
17
|
-
response = Excon.get(@base_uri + path)
|
19
|
+
response = Excon.get(@base_uri + path, tls_excon_arguments)
|
18
20
|
|
19
21
|
raise unless response.status == 200
|
20
22
|
JSON.load(response.body)
|
@@ -26,7 +28,7 @@ class Centurion::DockerViaApi
|
|
26
28
|
|
27
29
|
response = Excon.get(
|
28
30
|
@base_uri + path,
|
29
|
-
:headers => {'Accept' => 'application/json'}
|
31
|
+
tls_excon_arguments.merge(:headers => {'Accept' => 'application/json'})
|
30
32
|
)
|
31
33
|
raise response.inspect unless response.status == 200
|
32
34
|
JSON.load(response.body)
|
@@ -46,6 +48,7 @@ class Centurion::DockerViaApi
|
|
46
48
|
path = "/v1.7/containers/#{container_id}"
|
47
49
|
response = Excon.delete(
|
48
50
|
@base_uri + path,
|
51
|
+
tls_excon_arguments
|
49
52
|
)
|
50
53
|
raise response.inspect unless response.status == 204
|
51
54
|
true
|
@@ -55,17 +58,21 @@ class Centurion::DockerViaApi
|
|
55
58
|
path = "/v1.7/containers/#{container_id}/stop?t=#{timeout}"
|
56
59
|
response = Excon.post(
|
57
60
|
@base_uri + path,
|
61
|
+
tls_excon_arguments
|
58
62
|
)
|
59
63
|
raise response.inspect unless response.status == 204
|
60
64
|
true
|
61
65
|
end
|
62
66
|
|
63
|
-
def create_container(configuration)
|
67
|
+
def create_container(configuration, name = nil)
|
64
68
|
path = "/v1.10/containers/create"
|
65
69
|
response = Excon.post(
|
66
70
|
@base_uri + path,
|
67
|
-
|
68
|
-
|
71
|
+
tls_excon_arguments.merge(
|
72
|
+
:query => name ? {:name => "#{name}-#{SecureRandom.hex(7)}"} : nil,
|
73
|
+
:body => configuration.to_json,
|
74
|
+
:headers => { "Content-Type" => "application/json" }
|
75
|
+
)
|
69
76
|
)
|
70
77
|
raise response.inspect unless response.status == 201
|
71
78
|
JSON.load(response.body)
|
@@ -75,8 +82,10 @@ class Centurion::DockerViaApi
|
|
75
82
|
path = "/v1.10/containers/#{container_id}/start"
|
76
83
|
response = Excon.post(
|
77
84
|
@base_uri + path,
|
78
|
-
|
79
|
-
|
85
|
+
tls_excon_arguments.merge(
|
86
|
+
:body => configuration.to_json,
|
87
|
+
:headers => { "Content-Type" => "application/json" }
|
88
|
+
)
|
80
89
|
)
|
81
90
|
case response.status
|
82
91
|
when 204
|
@@ -92,6 +101,7 @@ class Centurion::DockerViaApi
|
|
92
101
|
path = "/v1.7/containers/#{container_id}/json"
|
93
102
|
response = Excon.get(
|
94
103
|
@base_uri + path,
|
104
|
+
tls_excon_arguments
|
95
105
|
)
|
96
106
|
raise response.inspect unless response.status == 200
|
97
107
|
JSON.load(response.body)
|
@@ -109,6 +119,19 @@ class Centurion::DockerViaApi
|
|
109
119
|
end
|
110
120
|
end
|
111
121
|
|
122
|
+
def tls_enable?
|
123
|
+
@tls_args.is_a?(Hash) && @tls_args.size > 0
|
124
|
+
end
|
125
|
+
|
126
|
+
def tls_excon_arguments
|
127
|
+
return {} unless [:tlscert, :tlskey].all? { |key| @tls_args.key?(key) }
|
128
|
+
|
129
|
+
{
|
130
|
+
client_cert: @tls_args[:tlscert],
|
131
|
+
client_key: @tls_args[:tlskey]
|
132
|
+
}
|
133
|
+
end
|
134
|
+
|
112
135
|
def configure_excon_globally
|
113
136
|
Excon.defaults[:connect_timeout] = 120
|
114
137
|
Excon.defaults[:read_timeout] = 120
|
@@ -117,5 +140,6 @@ class Centurion::DockerViaApi
|
|
117
140
|
Excon.defaults[:debug_response] = true
|
118
141
|
Excon.defaults[:nonblock] = false
|
119
142
|
Excon.defaults[:tcp_nodelay] = true
|
143
|
+
Excon.defaults[:ssl_ca_file] = @tls_args[:tlscacert]
|
120
144
|
end
|
121
145
|
end
|
@@ -6,22 +6,62 @@ module Centurion; end
|
|
6
6
|
class Centurion::DockerViaCli
|
7
7
|
include Centurion::Logging
|
8
8
|
|
9
|
-
def initialize(hostname, port, docker_path)
|
9
|
+
def initialize(hostname, port, docker_path, tls_args = {})
|
10
10
|
@docker_host = "tcp://#{hostname}:#{port}"
|
11
11
|
@docker_path = docker_path
|
12
|
+
@tls_args = tls_args
|
12
13
|
end
|
13
14
|
|
14
15
|
def pull(image, tag='latest')
|
15
|
-
info
|
16
|
-
echo(
|
16
|
+
info 'Using CLI to pull'
|
17
|
+
echo(build_command(:pull, "#{image}:#{tag}"))
|
17
18
|
end
|
18
19
|
|
19
20
|
def tail(container_id)
|
20
21
|
info "Tailing the logs on #{container_id}"
|
21
|
-
echo(
|
22
|
+
echo(build_command(:logs, container_id))
|
22
23
|
end
|
23
24
|
|
24
25
|
def attach(container_id)
|
25
|
-
|
26
|
+
echo(build_command(:attach, container_id))
|
27
|
+
end
|
28
|
+
|
29
|
+
private
|
30
|
+
|
31
|
+
def self.tls_keys
|
32
|
+
[:tlscacert, :tlscert, :tlskey]
|
33
|
+
end
|
34
|
+
|
35
|
+
def all_tls_path_available?
|
36
|
+
self.class.tls_keys.all? { |key| @tls_args.key?(key) }
|
37
|
+
end
|
38
|
+
|
39
|
+
def tls_parameters
|
40
|
+
return '' if @tls_args.nil? || @tls_args == {}
|
41
|
+
|
42
|
+
tls_flags = ''
|
43
|
+
|
44
|
+
# --tlsverify can be set without passing the cacert, cert and key flags
|
45
|
+
if @tls_args[:tls] == true || all_tls_path_available?
|
46
|
+
tls_flags << ' --tlsverify'
|
47
|
+
end
|
48
|
+
|
49
|
+
self.class.tls_keys.each do |key|
|
50
|
+
tls_flags << " --#{key}=#{@tls_args[key]}" if @tls_args[key]
|
51
|
+
end
|
52
|
+
|
53
|
+
tls_flags
|
54
|
+
end
|
55
|
+
|
56
|
+
def build_command(action, destination)
|
57
|
+
command = "#{@docker_path} -H=#{@docker_host}"
|
58
|
+
command << tls_parameters
|
59
|
+
command << case action
|
60
|
+
when :pull then ' pull '
|
61
|
+
when :logs then ' logs -f '
|
62
|
+
when :attach then ' attach '
|
63
|
+
end
|
64
|
+
command << destination
|
65
|
+
command
|
26
66
|
end
|
27
67
|
end
|
data/lib/centurion/dogestry.rb
CHANGED
@@ -52,12 +52,7 @@ class Centurion::Dogestry
|
|
52
52
|
"s3://#{s3_bucket}/?region=#{s3_region}"
|
53
53
|
end
|
54
54
|
|
55
|
-
def
|
56
|
-
@options[:docker_host] || 'tcp://localhost:2375'
|
57
|
-
end
|
58
|
-
|
59
|
-
def set_envs(docker_host)
|
60
|
-
ENV['DOCKER_HOST'] = docker_host
|
55
|
+
def set_envs()
|
61
56
|
ENV['AWS_ACCESS_KEY'] = aws_access_key_id
|
62
57
|
ENV['AWS_SECRET_KEY'] = aws_secret_key
|
63
58
|
|
@@ -70,22 +65,14 @@ class Centurion::Dogestry
|
|
70
65
|
command
|
71
66
|
end
|
72
67
|
|
73
|
-
def
|
68
|
+
def pull(repo, pull_hosts)
|
74
69
|
validate_before_exec
|
75
|
-
set_envs(
|
70
|
+
set_envs()
|
76
71
|
|
77
|
-
|
72
|
+
hosts = pull_hosts.join(",")
|
73
|
+
flags = "-pullhosts #{hosts}"
|
78
74
|
|
79
|
-
echo(exec_command('
|
75
|
+
echo(exec_command('pull', repo, flags))
|
80
76
|
end
|
81
77
|
|
82
|
-
def upload_temp_dir_image_to_docker(repo, local_dir, docker_host)
|
83
|
-
validate_before_exec
|
84
|
-
set_envs(docker_host)
|
85
|
-
|
86
|
-
command = "dogestry upload #{local_dir} #{repo}"
|
87
|
-
info "Executing: #{command}"
|
88
|
-
|
89
|
-
echo(command)
|
90
|
-
end
|
91
78
|
end
|