centurion 1.6.0 → 1.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CONTRIBUTORS.md +2 -0
- data/README.md +100 -15
- data/lib/centurion/deploy.rb +36 -134
- data/lib/centurion/deploy_dsl.rb +62 -21
- data/lib/centurion/docker_server.rb +12 -2
- data/lib/centurion/docker_via_api.rb +18 -10
- data/lib/centurion/dogestry.rb +16 -2
- data/lib/centurion/service.rb +218 -0
- data/lib/centurion/version.rb +1 -1
- data/lib/core_ext/numeric_bytes.rb +59 -57
- data/lib/tasks/centurion.rake +3 -0
- data/lib/tasks/deploy.rake +29 -42
- data/spec/deploy_dsl_spec.rb +78 -17
- data/spec/deploy_spec.rb +45 -343
- data/spec/docker_server_spec.rb +16 -1
- data/spec/docker_via_api_spec.rb +32 -55
- data/spec/service_spec.rb +288 -0
- metadata +27 -42
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 37e2e34bed03a54303fbb97f00a7ea41ca703c92
|
4
|
+
data.tar.gz: 9ce533114ecddb1fcf67108b3d3b6e5cb722aa5b
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: a26cced05248fc737b51aa7b550086d7284edc6635afecc418e5835acbbffe47a5b164a405775044008d56eda57afea3470345cf2e685b4a31aac46c876e81de
|
7
|
+
data.tar.gz: 67d3855249e3ed55a2a0c7431fbf1bc01c0cdc22e9d731f572f75d4e133bfe736c419f24c8f9fd70a52c076786045f859ec95a5db3d841c9f434e26eaa8c7403
|
data/CONTRIBUTORS.md
CHANGED
@@ -11,6 +11,7 @@ Your name could be here!
|
|
11
11
|
* [Matt Sanford][mzsanford]
|
12
12
|
* [Suren Karapetyan][skarap]
|
13
13
|
* [Jon Wood][jellybob]
|
14
|
+
* [Mark Borcherding][markborcherding]
|
14
15
|
|
15
16
|
Pre-release
|
16
17
|
-----------
|
@@ -71,3 +72,4 @@ Contributor | Commits | Additions | Deletio
|
|
71
72
|
[mzsanford]: https://github.com/mzsanford
|
72
73
|
[skarap]: https://github.com/skarap
|
73
74
|
[jellybob]: https://github.com/jellybob
|
75
|
+
[markborcherding]: https://github.com/markborcherding
|
data/README.md
CHANGED
@@ -15,7 +15,7 @@ tools directly so you can use anything they currently support via the normal
|
|
15
15
|
registry mechanism.
|
16
16
|
|
17
17
|
If you haven't been using a registry, you should read up on how to do that
|
18
|
-
before trying to deploy anything with Centurion.
|
18
|
+
before trying to deploy anything with Centurion.
|
19
19
|
|
20
20
|
Commercial Docker Registry Providers:
|
21
21
|
- Docker, Inc. [provides repositories](https://index.docker.io/), and hosts the
|
@@ -37,6 +37,8 @@ one roll-up commit of all our internal code. But all internal development will
|
|
37
37
|
now be on public GitHub. See the CONTRIBUTORS file for the contributors to the
|
38
38
|
original internal project.
|
39
39
|
|
40
|
+
The **current stable release** is 1.6.0.
|
41
|
+
|
40
42
|
Installation
|
41
43
|
------------
|
42
44
|
|
@@ -59,8 +61,8 @@ the commands should just work now.
|
|
59
61
|
Configuration
|
60
62
|
-------------
|
61
63
|
|
62
|
-
Centurion expects to find configuration tasks in the current working directory
|
63
|
-
|
64
|
+
Centurion expects to find configuration tasks in the current working directory
|
65
|
+
tree.
|
64
66
|
|
65
67
|
We recommend putting all your configuration for multiple applications into a
|
66
68
|
single repo rather than spreading it around by project. This allows a central
|
@@ -148,7 +150,7 @@ You can cause your container to be started with a specific DNS server
|
|
148
150
|
IP address (the equivalent of `docker run --dns 172.17.42.1 ...`) like this:
|
149
151
|
```ruby
|
150
152
|
task :production => :common do
|
151
|
-
set :
|
153
|
+
set :dns, '172.17.42.1'
|
152
154
|
# ...
|
153
155
|
end
|
154
156
|
```
|
@@ -176,17 +178,47 @@ With this, the container will be named something like `backend-4f692997`.
|
|
176
178
|
### Container Hostnames
|
177
179
|
|
178
180
|
If you don't specify a hostname to use inside your container, the container
|
179
|
-
will be given
|
181
|
+
will be given a hostname matching the container ID. This probably is good for
|
180
182
|
a lot of situations, but it might not be good for yours. If you need to have
|
181
|
-
a specific hostname, you can
|
183
|
+
a specific hostname, you can ask Centurion to do that:
|
182
184
|
|
183
185
|
```ruby
|
184
186
|
set :container_hostname, 'yourhostname'
|
185
187
|
```
|
186
188
|
|
187
|
-
|
188
|
-
|
189
|
-
|
189
|
+
That will make *all* of your containers named 'yourhostname'. If you want to do
|
190
|
+
something more dynamic, you can pass a `Proc` or a lambda like this:
|
191
|
+
|
192
|
+
```ruby
|
193
|
+
set :container_hostname, ->(hostname) { "#{hostname}.example.com" }
|
194
|
+
```
|
195
|
+
|
196
|
+
The lambda will be passed the current server's hostname. So, this example will
|
197
|
+
cause ".example.com" to be appended to the hostname of each Docker host during
|
198
|
+
deployment.
|
199
|
+
|
200
|
+
*If you want to restore the old behavior from Centurion 1.6.0* and earlier, you
|
201
|
+
can do the following:
|
202
|
+
|
203
|
+
```ruby
|
204
|
+
set :container_hostname, ->(hostname) { hostname }
|
205
|
+
```
|
206
|
+
|
207
|
+
That will cause the container hostname to match the server's hostname.
|
208
|
+
|
209
|
+
### Network modes
|
210
|
+
|
211
|
+
You may specify the network mode you would like a container to use via:
|
212
|
+
|
213
|
+
```ruby
|
214
|
+
set :network_mode, 'networkmode'
|
215
|
+
```
|
216
|
+
|
217
|
+
Docker (and therefore Centurion) supports one of `bridge` (the default), `host`,
|
218
|
+
and `container:<container-id>` for this argument.
|
219
|
+
|
220
|
+
*Note:* While `host_port` remains required, the mappings specified in it are
|
221
|
+
*ignored* when using `host` and `container...` network modes.
|
190
222
|
|
191
223
|
### CGroup Resource Constraints
|
192
224
|
|
@@ -205,6 +237,27 @@ cpu_shares 512
|
|
205
237
|
For more information on Docker's CGroup limits see [the Docker
|
206
238
|
docs](https://docs.docker.com/reference/run/#runtime-constraints-on-cpu-and-memory).
|
207
239
|
|
240
|
+
### Adding Extended Capabilities
|
241
|
+
|
242
|
+
Additional kernel capabilities may be granted to containers, permitting them
|
243
|
+
device access they do not normally have. You may specify these as follows:
|
244
|
+
|
245
|
+
```ruby
|
246
|
+
add_capability 'SOME_CAPABILITY'
|
247
|
+
add_capability 'ANOTHER_CAPABILITY'
|
248
|
+
drop_capability 'SOMEOTHER_CAPABILITY'
|
249
|
+
```
|
250
|
+
|
251
|
+
You may also ask for all but a few capabilities as follows:
|
252
|
+
|
253
|
+
```ruby
|
254
|
+
add_capability 'ALL'
|
255
|
+
drop_capability 'SOME_CAPABILITY'
|
256
|
+
```
|
257
|
+
|
258
|
+
For more information on which kernel capabilities may be specified, see the
|
259
|
+
[Docker docs](https://docs.docker.com/reference/run/#runtime-privilege-linux-capabilities-and-lxc-configuration).
|
260
|
+
|
208
261
|
### Interpolation
|
209
262
|
|
210
263
|
Currently there a couple of special strings for interpolation that can be added
|
@@ -229,7 +282,8 @@ You just need to enable the tls mode as the following:
|
|
229
282
|
end
|
230
283
|
```
|
231
284
|
|
232
|
-
Centurion will only set the `--tlsverify` to true and Docker will read your
|
285
|
+
Centurion will only set the `--tlsverify` to true and Docker will read your
|
286
|
+
certificate from the `~/.docker/` path.
|
233
287
|
|
234
288
|
#### Your certificate files are not in `~/.docker/`
|
235
289
|
|
@@ -315,9 +369,9 @@ are the same everywhere. Settings are per-project.
|
|
315
369
|
an individual container to come up before giving up as a failure. Defaults
|
316
370
|
to 24 attempts.
|
317
371
|
* `rolling_deploy_skip_ports` => Either a single port, or an array of ports
|
318
|
-
that should be skipped for status checks. By default status checking assumes
|
319
|
-
an HTTP server is on the other end and if you are deploying a container where some
|
320
|
-
ports are not HTTP services, this allows you to only health check the ports
|
372
|
+
that should be skipped for status checks. By default status checking assumes
|
373
|
+
an HTTP server is on the other end and if you are deploying a container where some
|
374
|
+
ports are not HTTP services, this allows you to only health check the ports
|
321
375
|
that are. The default is an empty array. If you have non-HTTP services that you
|
322
376
|
want to check, see Custom Health Checks in the previous section.
|
323
377
|
|
@@ -375,14 +429,14 @@ Centurion needs to have access to some registry in order to pull images to
|
|
375
429
|
remote Docker servers. This needs to be either a hosted registry (public or
|
376
430
|
private), or [Dogestry](https://github.com/dogestry/dogestry).
|
377
431
|
|
378
|
-
#### Access to the registry
|
432
|
+
#### Access to the registry
|
379
433
|
|
380
434
|
If you are not using either Dogestry, or the public registry, you may need to
|
381
435
|
provide authentication credentials. Centurion needs to access the Docker
|
382
436
|
registry hosting your images directly to retrive image ids and tags. This is
|
383
437
|
supported in both the config file and also as command line arguments.
|
384
438
|
|
385
|
-
The command line arguments are:
|
439
|
+
The command line arguments are:
|
386
440
|
* `--registry-user` => The username to pass to the registry
|
387
441
|
* `--registry-password` => The password
|
388
442
|
|
@@ -423,9 +477,22 @@ namespace :environment do
|
|
423
477
|
end
|
424
478
|
```
|
425
479
|
|
480
|
+
**TLS with Dogestry**: Because this involves a little passing around of both
|
481
|
+
settings and environment variables, there are a couple of things to verify to
|
482
|
+
make sure everything is passed properly between Centurion and Dogestry. If your
|
483
|
+
keys have the default names and are in located in the path represented by
|
484
|
+
`DOCKER_CERT_PATH` in your environment, this should just work. Otherwise you'll
|
485
|
+
need to be sure to `set :tlsverify, true` and *also* set the TLS cert names as
|
486
|
+
decribed above.
|
487
|
+
|
426
488
|
Development
|
427
489
|
-----------
|
428
490
|
|
491
|
+
Centurion supports a few features to make development easier when
|
492
|
+
building your deployment tooling or debugging your containers.
|
493
|
+
|
494
|
+
#### Overriding Environment Variables
|
495
|
+
|
429
496
|
Sometimes when you're doing development you want to try out some configuration
|
430
497
|
settings in environment variables that aren't in the config yet. Or perhaps you
|
431
498
|
want to override existing settings to test with. You can provide the
|
@@ -441,6 +508,24 @@ this functionality for production deployments. It will work, but it means that
|
|
441
508
|
the config is not the whole source of truth for your container configuration.
|
442
509
|
Caveat emptor.
|
443
510
|
|
511
|
+
#### Exporting Environment Variables Locally
|
512
|
+
|
513
|
+
Sometimes you need to test how your code works inside the container and you
|
514
|
+
need to have all of your configuration exported. Centurion includes an action
|
515
|
+
that will let you do that. It exports all of your environment settings for the
|
516
|
+
environment you specify. It then partially sanitizes them to preserve things
|
517
|
+
like `rbenv` settings. Then it executes `/bin/bash` locally.
|
518
|
+
|
519
|
+
The action is named `dev:export_only` and you call it like this:
|
520
|
+
|
521
|
+
```bash
|
522
|
+
$ bundle exec centurion -e development -p testing_project -a dev:export_only
|
523
|
+
$ bundle exec rake spec
|
524
|
+
```
|
525
|
+
|
526
|
+
It's important to note that the second line is actually being invoked with new
|
527
|
+
environment exported.
|
528
|
+
|
444
529
|
Future Additions
|
445
530
|
----------------
|
446
531
|
|
data/lib/centurion/deploy.rb
CHANGED
@@ -5,12 +5,9 @@ module Centurion; end
|
|
5
5
|
|
6
6
|
module Centurion::Deploy
|
7
7
|
FAILED_CONTAINER_VALIDATION = 100
|
8
|
-
INVALID_CGROUP_CPUSHARES_VALUE = 101
|
9
|
-
INVALID_CGROUP_MEMORY_VALUE = 102
|
10
8
|
|
11
|
-
def stop_containers(target_server,
|
12
|
-
|
13
|
-
old_containers = target_server.find_containers_by_public_port(public_port)
|
9
|
+
def stop_containers(target_server, service, timeout = 30)
|
10
|
+
old_containers = target_server.find_containers_by_public_port(service.public_ports.first)
|
14
11
|
info "Stopping container(s): #{old_containers.inspect}"
|
15
12
|
|
16
13
|
old_containers.each do |old_container|
|
@@ -19,10 +16,10 @@ module Centurion::Deploy
|
|
19
16
|
end
|
20
17
|
end
|
21
18
|
|
22
|
-
def wait_for_health_check_ok(health_check_method, target_server, port, endpoint, image_id, tag, sleep_time=5, retries=12)
|
19
|
+
def wait_for_health_check_ok(health_check_method, target_server, container_id, port, endpoint, image_id, tag, sleep_time=5, retries=12)
|
23
20
|
info 'Waiting for the port to come up'
|
24
21
|
1.upto(retries) do
|
25
|
-
if container_up?(target_server,
|
22
|
+
if container_up?(target_server, container_id) && health_check_method.call(target_server, port, endpoint)
|
26
23
|
info 'Container is up!'
|
27
24
|
break
|
28
25
|
end
|
@@ -37,20 +34,13 @@ module Centurion::Deploy
|
|
37
34
|
end
|
38
35
|
end
|
39
36
|
|
40
|
-
def container_up?(target_server,
|
37
|
+
def container_up?(target_server, container_id)
|
41
38
|
# The API returns a record set like this:
|
42
39
|
#[{"Command"=>"script/run ", "Created"=>1394470428, "Id"=>"41a68bda6eb0a5bb78bbde19363e543f9c4f0e845a3eb130a6253972051bffb0", "Image"=>"quay.io/newrelic/rubicon:5f23ac3fad7979cd1efdc9295e0d8c5707d1c806", "Names"=>["/happy_pike"], "Ports"=>[{"IP"=>"0.0.0.0", "PrivatePort"=>80, "PublicPort"=>8484, "Type"=>"tcp"}], "Status"=>"Up 13 seconds"}]
|
43
40
|
|
44
|
-
|
45
|
-
container = running_containers.pop
|
41
|
+
container = target_server.find_container_by_id(container_id)
|
46
42
|
|
47
|
-
|
48
|
-
# This _should_ never happen, but...
|
49
|
-
error "More than one container is bound to port #{port} on #{target_server}!"
|
50
|
-
return false
|
51
|
-
end
|
52
|
-
|
53
|
-
if container && container['Ports'].any? { |bind| bind['PublicPort'].to_i == port.to_i }
|
43
|
+
if container
|
54
44
|
info "Found container up for #{Time.now.to_i - container['Created'].to_i} seconds"
|
55
45
|
return true
|
56
46
|
end
|
@@ -74,149 +64,61 @@ module Centurion::Deploy
|
|
74
64
|
false
|
75
65
|
end
|
76
66
|
|
77
|
-
def is_a_uint64?(value)
|
78
|
-
result = false
|
79
|
-
if !value.is_a? Integer
|
80
|
-
return result
|
81
|
-
end
|
82
|
-
if value < 0 || value > 0xFFFFFFFFFFFFFFFF
|
83
|
-
return result
|
84
|
-
end
|
85
|
-
return true
|
86
|
-
end
|
87
|
-
|
88
67
|
def wait_for_load_balancer_check_interval
|
89
68
|
sleep(fetch(:rolling_deploy_check_interval, 5))
|
90
69
|
end
|
91
70
|
|
92
|
-
def cleanup_containers(target_server,
|
93
|
-
|
94
|
-
old_containers = target_server.old_containers_for_port(public_port)
|
71
|
+
def cleanup_containers(target_server, service)
|
72
|
+
old_containers = target_server.old_containers_for_name(service.name)
|
95
73
|
old_containers.shift(2)
|
96
74
|
|
97
|
-
info "
|
75
|
+
info "Service name #{service.name}"
|
98
76
|
old_containers.each do |old_container|
|
99
77
|
info "Removing old container #{old_container['Id'][0..7]} (#{old_container['Names'].join(',')})"
|
100
78
|
target_server.remove_container(old_container['Id'])
|
101
79
|
end
|
102
80
|
end
|
103
81
|
|
104
|
-
def
|
105
|
-
|
106
|
-
|
107
|
-
error "Invalid value for CGroup memory constraint: #{memory}, value must be a between 0 and 18446744073709551615"
|
108
|
-
exit(INVALID_CGROUP_MEMORY_VALUE)
|
109
|
-
end
|
110
|
-
|
111
|
-
if cpu_shares && ! is_a_uint64?(cpu_shares)
|
112
|
-
error "Invalid value for CGroup CPU constraint: #{cpu_shares}, value must be between 0 and 18446744073709551615"
|
113
|
-
exit(INVALID_CGROUP_CPUSHARES_VALUE)
|
114
|
-
end
|
82
|
+
def hostname_proc
|
83
|
+
hostname = fetch(:container_hostname)
|
84
|
+
return nil if hostname.nil?
|
115
85
|
|
116
|
-
|
117
|
-
|
118
|
-
|
119
|
-
|
120
|
-
|
121
|
-
container_config.merge!('Cmd' => command) if command
|
122
|
-
container_config.merge!('Memory' => memory) if memory
|
123
|
-
container_config.merge!('CpuShares' => cpu_shares) if cpu_shares
|
124
|
-
|
125
|
-
if port_bindings
|
126
|
-
container_config['ExposedPorts'] ||= {}
|
127
|
-
port_bindings.keys.each do |port|
|
128
|
-
container_config['ExposedPorts'][port] = {}
|
129
|
-
end
|
130
|
-
end
|
131
|
-
|
132
|
-
if env_vars
|
133
|
-
container_config['Env'] = env_vars.map do |k,v|
|
134
|
-
"#{k}=#{interpolate_var(v, target_server)}"
|
135
|
-
end
|
136
|
-
end
|
137
|
-
|
138
|
-
if volumes
|
139
|
-
container_config['Volumes'] = volumes.inject({}) do |memo, v|
|
140
|
-
memo[v.split(/:/).last] = {}
|
141
|
-
memo
|
142
|
-
end
|
143
|
-
container_config['VolumesFrom'] = 'parent'
|
86
|
+
if hostname.respond_to?(:call)
|
87
|
+
hostname
|
88
|
+
else
|
89
|
+
->(h) { hostname }
|
144
90
|
end
|
145
|
-
|
146
|
-
container_config
|
147
|
-
end
|
148
|
-
|
149
|
-
def start_new_container(target_server, image_id, port_bindings, volumes, env_vars=nil, command=nil, memory=nil, cpu_shares=nil)
|
150
|
-
container_config = container_config_for(target_server, image_id, port_bindings, env_vars, volumes, command, memory, cpu_shares)
|
151
|
-
start_container_with_config(target_server, volumes, port_bindings, container_config)
|
152
|
-
end
|
153
|
-
|
154
|
-
def launch_console(target_server, image_id, port_bindings, volumes, env_vars=nil)
|
155
|
-
container_config = container_config_for(target_server, image_id, port_bindings, env_vars, volumes, ['/bin/bash']).merge(
|
156
|
-
'AttachStdin' => true,
|
157
|
-
'Tty' => true,
|
158
|
-
'OpenStdin' => true,
|
159
|
-
)
|
160
|
-
|
161
|
-
container = start_container_with_config(target_server, volumes, port_bindings, container_config)
|
162
|
-
|
163
|
-
target_server.attach(container['Id'])
|
164
91
|
end
|
165
92
|
|
166
|
-
|
167
|
-
|
168
|
-
|
169
|
-
|
170
|
-
host_config['RestartPolicy'] = {}
|
93
|
+
def start_new_container(server, service, restart_policy)
|
94
|
+
container_config = service.build_config(server.hostname, &hostname_proc)
|
95
|
+
info "Creating new container for #{container_config['Image'][0..7]}"
|
96
|
+
container = server.create_container(container_config, service.name)
|
171
97
|
|
172
|
-
|
173
|
-
restart_policy_name = 'on-failure' unless ["always", "on-failure", "no"].include?(restart_policy_name)
|
98
|
+
host_config = service.build_host_config(restart_policy)
|
174
99
|
|
175
|
-
|
100
|
+
info "Starting new container #{container['Id'][0..7]}"
|
101
|
+
server.start_container(container['Id'], host_config)
|
176
102
|
|
177
|
-
|
178
|
-
|
103
|
+
info "Inspecting new container #{container['Id'][0..7]}:"
|
104
|
+
(server.inspect_container(container['Id']) || {}).each_pair do |key,value|
|
105
|
+
info "\t#{key} => #{value.inspect}"
|
106
|
+
end
|
179
107
|
|
180
|
-
|
108
|
+
container
|
181
109
|
end
|
182
110
|
|
183
|
-
def
|
111
|
+
def launch_console(server, service)
|
112
|
+
container_config = service.build_console_config(server.hostname)
|
184
113
|
info "Creating new container for #{container_config['Image'][0..7]}"
|
185
|
-
new_container = target_server.create_container(container_config, fetch(:name))
|
186
114
|
|
187
|
-
|
115
|
+
container = server.create_container(container_config, service.name)
|
188
116
|
|
189
|
-
|
190
|
-
host_config['Binds'] = volumes if volumes && !volumes.empty?
|
117
|
+
host_config = service.build_host_config
|
191
118
|
|
192
|
-
|
193
|
-
|
119
|
+
info "Starting new container #{container['Id'][0..7]}"
|
120
|
+
server.start_container(container['Id'], host_config)
|
194
121
|
|
195
|
-
|
196
|
-
dns = fetch(:custom_dns)
|
197
|
-
host_config['Dns'] = dns if dns
|
198
|
-
|
199
|
-
# Restart Policy
|
200
|
-
host_config = build_host_config_restart_policy(host_config)
|
201
|
-
|
202
|
-
info "Starting new container #{new_container['Id'][0..7]}"
|
203
|
-
target_server.start_container(new_container['Id'], host_config)
|
204
|
-
|
205
|
-
info "Inspecting new container #{new_container['Id'][0..7]}:"
|
206
|
-
info target_server.inspect_container(new_container['Id'])
|
207
|
-
|
208
|
-
new_container
|
122
|
+
server.attach(container['Id'])
|
209
123
|
end
|
210
|
-
|
211
|
-
def interpolate_var(val, target_server)
|
212
|
-
val.gsub('%DOCKER_HOSTNAME%', target_server.hostname)
|
213
|
-
.gsub('%DOCKER_HOST_IP%', host_ip(target_server.hostname))
|
214
|
-
end
|
215
|
-
|
216
|
-
def host_ip(hostname)
|
217
|
-
@host_ip ||= {}
|
218
|
-
return @host_ip[hostname] if @host_ip.has_key?(hostname)
|
219
|
-
@host_ip[hostname] = Socket.getaddrinfo(hostname, nil).first[2]
|
220
|
-
end
|
221
|
-
|
222
124
|
end
|