docker-armada 2.0.53

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (44) hide show
  1. checksums.yaml +7 -0
  2. data/.gitignore +12 -0
  3. data/.travis.yml +5 -0
  4. data/Gemfile +5 -0
  5. data/LICENSE +20 -0
  6. data/README.md +293 -0
  7. data/Rakefile +9 -0
  8. data/Thorfile +1 -0
  9. data/armada.gemspec +37 -0
  10. data/bin/armada +5 -0
  11. data/lib/armada.rb +37 -0
  12. data/lib/armada/clean.rb +2 -0
  13. data/lib/armada/clean/containers.rb +30 -0
  14. data/lib/armada/clean/images.rb +29 -0
  15. data/lib/armada/cli.rb +40 -0
  16. data/lib/armada/cli/clean.rb +23 -0
  17. data/lib/armada/cli/deploy.rb +32 -0
  18. data/lib/armada/cli/inspect.rb +28 -0
  19. data/lib/armada/configuration.rb +33 -0
  20. data/lib/armada/connection.rb +3 -0
  21. data/lib/armada/connection/docker.rb +22 -0
  22. data/lib/armada/connection/health_check.rb +66 -0
  23. data/lib/armada/connection/remote.rb +26 -0
  24. data/lib/armada/deploy.rb +2 -0
  25. data/lib/armada/deploy/parallel.rb +58 -0
  26. data/lib/armada/deploy/rolling.rb +60 -0
  27. data/lib/armada/deploy_dsl.rb +156 -0
  28. data/lib/armada/docker.rb +6 -0
  29. data/lib/armada/docker/config.rb +55 -0
  30. data/lib/armada/docker/container.rb +120 -0
  31. data/lib/armada/docker/host.rb +68 -0
  32. data/lib/armada/docker/image.rb +86 -0
  33. data/lib/armada/thor.rb +1 -0
  34. data/lib/armada/ui.rb +15 -0
  35. data/lib/armada/utils.rb +2 -0
  36. data/lib/armada/utils/array.rb +9 -0
  37. data/lib/armada/utils/time.rb +23 -0
  38. data/lib/armada/version.rb +3 -0
  39. data/spec/connection/health_check_spec.rb +36 -0
  40. data/spec/deploy_dsl_spec.rb +84 -0
  41. data/spec/docker/container_spec.rb +124 -0
  42. data/spec/docker/image_spec.rb +110 -0
  43. data/spec/spec_helper.rb +12 -0
  44. metadata +289 -0
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: 0bc76cc4632cb663329bafd287254200a8c8cf44
4
+ data.tar.gz: 8858416bd0e9bf2fbac8fb54f564b586c45d3ba5
5
+ SHA512:
6
+ metadata.gz: c0d905bc4873c50aa9678e31995a8895a17aefea9ae45466ebd3927e19e55a047282afbb584180a4e9eff44dea84241a9d655fd58ce1bb64e6316c8736fe41b0
7
+ data.tar.gz: 68b4fec515215b7656d4e7aaeb320a13fa7ad601b0b4e34703248201c0dd4f826425117e80aa3688f4688eb1490d6bcc6cc687baedd0e2cd5a531927006281f6
data/.gitignore ADDED
@@ -0,0 +1,12 @@
1
+ .bundle
2
+ .config
3
+ coverage/
4
+ *.gem
5
+ pkg/
6
+ *.rbc
7
+ *.swp
8
+ *.swo
9
+ tmp/
10
+ .DS_Store
11
+ VERSION
12
+ Gemfile.lock
data/.travis.yml ADDED
@@ -0,0 +1,5 @@
1
+ language: ruby
2
+ rvm:
3
+ - 2.1.0
4
+ - 1.9.3
5
+ script: bundle exec rspec spec
data/Gemfile ADDED
@@ -0,0 +1,5 @@
1
+ source 'https://rubygems.org'
2
+
3
+ gemspec
4
+ gem 'geminabox'
5
+ gem 'rspec_junit_formatter', group: "test"
data/LICENSE ADDED
@@ -0,0 +1,20 @@
1
+ Copyright (c) 2014 Rally Software Development Corp
2
+
3
+ Permission is hereby granted, free of charge, to any person obtaining
4
+ a copy of this software and associated documentation files (the
5
+ "Software"), to deal in the Software without restriction, including
6
+ without limitation the rights to use, copy, modify, merge, publish,
7
+ distribute, sublicense, and/or sell copies of the Software, and to
8
+ permit persons to whom the Software is furnished to do so, subject to
9
+ the following conditions:
10
+
11
+ The above copyright notice and this permission notice shall be
12
+ included in all copies or substantial portions of the Software.
13
+
14
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
15
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
16
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
17
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
18
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
19
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
20
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,293 @@
1
+ [![TravisCI](https://travis-ci.org/RallySoftware/armada.svg)](https://travis-ci.org/RallySoftware/armada)
2
+
3
+
4
+ ## Description
5
+
6
+ Armada is a docker deployment tool which we originally forked from the [NewRelic Centurion](https://github.com/newrelic/centurion) project. It has since seen a huge refactor occur where we started using the [swipely/docker-api](https://github.com/swipely/docker-api) gem for interacting with our docker hosts instead of the mix of docker-cli and api calls that Centurion makes. The DSL is largely unchanged as it works really well for our intended purposes.
7
+
8
+ ## Disclaimer
9
+ This gem is used in production for deployments at Rally Software. We like the structure it gives us and the simplicity of the DSL. If you feel that we are missing something please open an issue and let's have a discussion about it. If you find a bug please submit an issue and if possible a reproducible test case.
10
+
11
+ ## Installation
12
+
13
+ ```
14
+ $ gem install armada
15
+ ```
16
+
17
+ ## Writing your service descriptor
18
+ Descriptors are in the form of a Rake file that uses a built-in DSL to make them
19
+ easy to write. Here's a sample config for a project called `myservice` that
20
+ would go into `$CWD/myservice.rake`:
21
+
22
+ **Armada will look in the current working directory for your service descriptor. We recommend placing this descriptor in your service's repository**
23
+
24
+ ```ruby
25
+ namespace :environment do
26
+ task :common do
27
+ env_vars LOG_DIR: '/home/myservice/logs/myservice'
28
+ container_name 'myservice'
29
+ set :image, 'quay.io/rallysoftware/myservice'
30
+ set :health_check_endpoint, '/metrics/healthcheck'
31
+ set :health_check_port, 3000
32
+ host_port 3000, container_port: 3000
33
+ end
34
+
35
+ desc 'Boulder environment'
36
+ task :bld => :common do
37
+ set_current_environment(:bld)
38
+ host 'bld-myservice-01:3534'
39
+ host 'bld-myservice-02:3534'
40
+ host 'bld-myservice-03:3534'
41
+ end
42
+
43
+ desc 'Production Environment'
44
+ task :prod => :common do
45
+ set_current_environment(:prod)
46
+ host 'prod-myservice-01:3534'
47
+ host 'prod-myservice-02:3534'
48
+ host 'prod-myservice-03:3534'
49
+ end
50
+
51
+ end
52
+ ```
53
+
54
+ ### Common Task
55
+ The `common` task is used to DRY up your descriptor file. It will always be loaded first allowing you to specify common elements of your deployment here and not repeat them throughout the rest of the file. You could also specify common elements here and then override them in later tasks if you need.
56
+
57
+ ### Tasks
58
+ Each task should represent a logical unit of seperation from the rest. For instance, in the above descriptor we are describing each of the environments where the `myservice` project can reside.
59
+
60
+ ### Armada DSL
61
+ Armada provides a few convenience methods for adding items such as host and environment variables to a list.
62
+
63
+ #### host_port - Exposing container ports to the host system
64
+ The `host_port` method takes 2 parameters - the `port` on the host system and a map of options. The map of options has 3 values that can be set:
65
+ * `host_ip` - The ip address of the host interface. This way you can bind your host port to a particular ip address. Default is `0.0.0.0`
66
+ * `container_port` - The exposed port you are trying to map. **REQUIRED**
67
+ * `type` - The type of port you are exposing. Default is `tcp`.
68
+
69
+ **You can call this method multiple times to specify multiple exposed ports.**
70
+ **If your container exposes a port and you do not want to map it to a static port on the host, Armada will make sure docker dynamically assigns it a port.**
71
+
72
+ Examples:
73
+
74
+ ```ruby
75
+ host_port 3000, container_port: 3000
76
+ host_port 8282, container_port: 8080, type: 'udp'
77
+ host_port 5991, host_ip: '123.456.789' container_port: 7001, type: 'udp'
78
+ ```
79
+
80
+ #### host_volume - Mapping container volumes to host volumes
81
+ The `host_volume` method takes two parameters - the host volume and a map of options. The map of options has 1 value that can be set.
82
+ * `container_volume` - The container volume to map to.
83
+
84
+ **You can call this method multiple times to specify multiple volumes**
85
+
86
+ Examples:
87
+ ```ruby
88
+ host_volume '/var/log', container_volume: '/var/log:rw'
89
+ host_volume '/var/log', container_volume: '/var/log:ro'
90
+ ```
91
+
92
+ #### env_vars - Key value pairs that are passed in as environment variables
93
+ The `env_vars` method take 1 parameter - a map of key value pairs.
94
+
95
+ Examples:
96
+ ```ruby
97
+ env_vars JAVA_OPTS: '-Xmx2g -server -XX:+UseConcMarkSweepGC'
98
+ env_vars DB_USER: 'someuser'
99
+ ```
100
+
101
+ **You can call this method multiple times to specify multiple environment variables**
102
+
103
+ #### host - Specifies a host for a given task to interact with
104
+ The `host` method takes 1 parameter which is a string containing the `host` and `port` of the docker api you would like to interact with.
105
+
106
+ Examples:
107
+ ```ruby
108
+ host 'bld-docker-01:4243'
109
+ ```
110
+
111
+ **You can call this method multiple times to specify multiple hosts**
112
+
113
+ #### container_name - Override the container name
114
+ The `container_name` method takes 1 parameter which is a string to name the container when it is created on the host. Currently this is how we identify which container to shutdown during a rolling deploy.
115
+
116
+ Examples:
117
+ ```ruby
118
+ container_name 'myservice'
119
+ ```
120
+
121
+ #### Docker Restart Policy
122
+ You can add your own Docker Restart policy, see the [API documentation](https://docs.docker.com/reference/api/docker_remote_api_v1.15/#create-a-container)
123
+
124
+ ```ruby
125
+ restart_policy { "Name" => "always" }
126
+ restart_policy { "Name" => "on-failure", "MaximumRetryCount" => 5 }
127
+ ```
128
+
129
+ #### Setting other descriptor values
130
+ Some configuration options are not set using a DSL method. Instead you must call the `set` method. The current list of these options are:
131
+
132
+ ```ruby
133
+ set :image, 'quay.io/myorg/myservice'
134
+ set :tag, '0.1.0'
135
+ set :health_check_endpoint, '/_metrics/healthcheck'
136
+ set :health_check_port, 3100
137
+ set :health_check_retries, 60 #number of times to check if the container is up and healthy
138
+ set :health_check_delay, 1 #number of seconds to wait between each retry
139
+ ```
140
+ **health_check_port will work even if you do not specify a contianer -> host port mapping. Armada will determine the dynamically assigned port to the expected container health check port and use that when performing the health check.**
141
+
142
+ #### Raw Container Config
143
+ If you want to use a docker feature not yet exposed through the armadafile, you can include a raw container config, and the rest of the armadafile will be applied on top of it.
144
+
145
+ ```ruby
146
+ container_config { "Cmd" => [ "date" ] }
147
+ container_config { "Privileged" => false }
148
+ ```
149
+
150
+ ## CLI
151
+ The CLI is written using [Thor](http://whatisthor.com/). Below is current commands that can be executed using the Armada gem.
152
+
153
+ ### Help
154
+ Examples:
155
+ ```bash
156
+ aramda help -- print the main help menu
157
+ armada deploy help parallel -- print the help menu for the parallel subcommand
158
+ armada deploy help
159
+ ```
160
+
161
+ ### Deploy
162
+ The following tasks can be used when deploying a container to a set of docker hosts.
163
+
164
+ #### Parallel
165
+ Deploys a project to all hosts in parallel. The following steps are run within their own thread. This means that some steps may complete on some hosts faster than other. Also means that all machines will be down at roughly the same time.
166
+
167
+ Command:
168
+ ```bash
169
+ armada deploy parallel <project> <environment> <options>
170
+ ```
171
+
172
+ Steps performed by this task:
173
+ 1. Pull the version of the image you wish to deploy. If `--no-pull` is specified this will not occur.
174
+ 1. Stop all running container(s) with the name you are wishing to use.
175
+ 1. Start the new container(s)
176
+ 1. Wait for the container(s) to come up
177
+ 1. If you used the `--health-check` option it will then wait until the health check passes
178
+
179
+ Options:
180
+ * `hosts` - This will override the hosts defined in the descriptor
181
+ * `image` - This will override the image defined in the descriptor
182
+ * `tag` - This will override the tag defined in the descriptor
183
+ * `username` - The username for the private registry of your image, if specified you must also specify `password`
184
+ * `password` - The password for the private registry of your image, if specified you must also specify `username`
185
+ * `health-check` - Default is true. You can specify `--no-health-check` to not perform a health check during a rolling deploy.
186
+ * `env-vars` - This allows for new or overriding env vars to be passed in from the command line. This option can only be specified once, but may take mulitple values.
187
+ * `ssh-gateway` - This allows you to perform commands against a remote docker host(s) using a gateway.
188
+ * `ssh-gateway-user` - The user opening the gateway
189
+ * `pull` - Allows you to specify whether to pull the specified image or not. Defaults to true.
190
+ * `dockercfg` - This is the path to the .dockercfg file which can be used instead of specifying the username and password for the registry. Defaults to `~/.dockercfg`
191
+
192
+ Examples:
193
+ ```bash
194
+ armada deploy parallel foo prod --hosts my-prod-host:5555 --username username --password secretsauce --health-check
195
+ armada deploy parallel foo prod --env-vars PORT:4343 DB_USER:"FOO"
196
+ ```
197
+
198
+ #### Rolling
199
+ This will deploy the project in a rolling fashion. Meaning it will only act on 1 host at a time.
200
+
201
+ Command:
202
+ ```bash
203
+ armada deploy rolling <project> <environment> <options>
204
+ ```
205
+
206
+ Steps performed by this task:
207
+ 1. Pull the version of the image you wish to deploy. If `--no-pull` is specified this will not occur.
208
+ 1. Stop all running container(s) with the name you are wishing to use.
209
+ 1. Start the new container(s)
210
+ 1. Wait for the container(s) to come up
211
+ 1. Perform a health check. You can specify the `--no-health-check` option to skip this step.
212
+ 1. Move to next host (if applicable), sequentially repeating the steps above until all hosts are complete.
213
+
214
+ Options:
215
+ * `hosts` - This will override the hosts defined in the descriptor
216
+ * `image` - This will override the image defined in the descriptor
217
+ * `tag` - This will override the tag defined in the descriptor
218
+ * `username` - The username for the private registry of your image, if specified you must also specify `password`
219
+ * `password` - The password for the private registry of your image, if specified you must also specify `username`
220
+ * `health-check` - Default is true. You can specify `--no-health-check` to not perform a health check during a rolling deploy.
221
+ * `env-vars` - This allows for new or overriding env vars to be passed in from the command line. This option can only be specified once, but may take mulitple values.
222
+ * `ssh-gateway` - This allows you to perform commands against a remote docker host(s) using a gateway.
223
+ * `ssh-gateway-user` - The user opening the gateway
224
+ * `pull` - Allows you to specify whether to pull the specified image or not. Defaults to true.
225
+ * `dockercfg` - This is the path to the .dockercfg file which can be used instead of specifying the username and password for the registry. Defaults to `~/.dockercfg`
226
+
227
+ Examples:
228
+ ```bash
229
+ armada deploy rolling foo prod --hosts my-prod-host:5555 --username username --password secretsauce --no-health-check
230
+ armada deploy rolling foo prod --env-vars PORT:4343 DB_USER:"FOO"
231
+ ```
232
+
233
+ ### Clean
234
+ The clean commands offer a way to clean up stale images or old containers from a host or set of hosts. You can issue these commands through a gateway if you like.
235
+
236
+ #### Containers
237
+ This will remove all containers that are not running or paused.
238
+
239
+ Options:
240
+ * `hosts` - The list of hosts you wish to perform this action against
241
+ * `ssh-gateway` - This allows you to perform commands against a remote docker host(s) using a gateway.
242
+ * `ssh-gateway-user` - The user opening the gateway
243
+ * `force` - Force the action to take place. ** If this option is not specified it will perform a dry run**
244
+
245
+ Examples:
246
+ ```bash
247
+ armada clean containers --hosts my-docker-host-01:3435
248
+ ```
249
+
250
+ #### Images
251
+ This will remove all images that are considered "orphaned". This means they are tagged as <none> if you run the `docker images` command. You will notice that this command will print out an error like the following:
252
+
253
+ ```bash
254
+ bld-docker-01 -- unable to remove image 9d535e81db1e because of the following error - Expected([200, 201, 202, 203, 204, 304]) <=> Actual(409 Conflict)
255
+ ```
256
+ This is because it is not the parent image. We are working to resolve this problem. You may also have to run this command a few times to get all the images cleaned up.
257
+
258
+ Options:
259
+ * `hosts` - The list of hosts you wish to perform this action against
260
+ * `ssh-gateway` - This allows you to perform commands against a remote docker host(s) using a gateway.
261
+ * `ssh-gateway-user` - The user opening the gateway
262
+ * `force` - Force the action to take place. ** If this option is not specified it will perform a dry run**
263
+
264
+ Examples:
265
+ ```bash
266
+ armada clean images --hosts my-docker-host-01:3435
267
+ ```
268
+
269
+ ## Maintainers
270
+ [Jonathan Chauncey (jchauncey)](https://github.com/jchauncey)
271
+ [Darrell Hamilton (zeroem)](https://github.com/zeroem)
272
+
273
+ ## License
274
+ Copyright (c) 2014 Rally Software Development Corp
275
+
276
+ Permission is hereby granted, free of charge, to any person obtaining
277
+ a copy of this software and associated documentation files (the
278
+ "Software"), to deal in the Software without restriction, including
279
+ without limitation the rights to use, copy, modify, merge, publish,
280
+ distribute, sublicense, and/or sell copies of the Software, and to
281
+ permit persons to whom the Software is furnished to do so, subject to
282
+ the following conditions:
283
+
284
+ The above copyright notice and this permission notice shall be
285
+ included in all copies or substantial portions of the Software.
286
+
287
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
288
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
289
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
290
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
291
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
292
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
293
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/Rakefile ADDED
@@ -0,0 +1,9 @@
1
+ $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), 'lib'))
2
+
3
+ require 'rake'
4
+ require 'rspec/core/rake_task'
5
+ require 'bundler/gem_tasks'
6
+
7
+ RSpec::Core::RakeTask.new do |t|
8
+ t.pattern = 'spec/**/*_spec.rb'
9
+ end
data/Thorfile ADDED
@@ -0,0 +1 @@
1
+ require 'thor/scmversion'
data/armada.gemspec ADDED
@@ -0,0 +1,37 @@
1
+ # coding: utf-8
2
+ lib = File.expand_path('../lib', __FILE__)
3
+ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
4
+ require 'armada/version'
5
+
6
+ Gem::Specification.new do |spec|
7
+ spec.name = 'docker-armada'
8
+ spec.version = Armada::VERSION
9
+ spec.authors = ['Jonathan Chauncey', 'Matt Farrar', 'Darrell Hamilton']
10
+ spec.summary = 'Deploy utility for docker containers'
11
+ spec.description = 'Deploy utility for docker containers'
12
+ spec.homepage = 'https://github.com/RallySoftware/armada'
13
+ spec.license = 'MIT'
14
+
15
+ spec.files = `git ls-files -z`.split("\x0")
16
+ spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
17
+ spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
18
+ spec.require_paths = ['lib']
19
+
20
+ spec.add_dependency 'excon', '~> 0.33'
21
+ spec.add_dependency 'net-ssh'
22
+ spec.add_dependency 'net-ssh-gateway'
23
+ spec.add_dependency 'docker-api', '~> 1.13'
24
+ spec.add_dependency 'thor', '~> 0.19'
25
+ spec.add_dependency 'awesome_print'
26
+ spec.add_dependency 'table_print'
27
+ spec.add_dependency 'conjur-cli'
28
+
29
+ spec.add_development_dependency 'bundler'
30
+ spec.add_development_dependency 'rake'
31
+ spec.add_development_dependency 'rspec', '~> 2.14.0'
32
+ spec.add_development_dependency 'thor-scmversion', '< 1.6.0'
33
+ spec.add_development_dependency 'geminabox', '~> 0.10'
34
+ spec.add_development_dependency 'webmock'
35
+
36
+ spec.required_ruby_version = '>= 1.9.3'
37
+ end
data/bin/armada ADDED
@@ -0,0 +1,5 @@
1
+ #!/usr/bin/env ruby
2
+ $:.push File.expand_path("../../lib", __FILE__)
3
+ require 'armada'
4
+
5
+ Armada::Cli.start
data/lib/armada.rb ADDED
@@ -0,0 +1,37 @@
1
+ require 'thor'
2
+ require 'net/ssh/gateway'
3
+ require 'awesome_print'
4
+ require 'table_print'
5
+
6
+ require_relative 'armada/cli'
7
+ require_relative 'armada/clean'
8
+ require_relative 'armada/configuration'
9
+ require_relative 'armada/connection'
10
+ require_relative 'armada/deploy'
11
+ require_relative 'armada/docker'
12
+ require_relative 'armada/deploy_dsl'
13
+ require_relative 'armada/ui'
14
+ require_relative 'armada/utils'
15
+ require_relative 'armada/thor'
16
+
17
+ module Armada
18
+ Thor::Base.shell.send(:include, Armada::UI)
19
+
20
+ class << self
21
+ def root
22
+ @root ||= Pathname.new(File.expand_path('../', File.dirname(__FILE__)))
23
+ end
24
+
25
+ def executable_name
26
+ File.basename($PROGRAM_NAME)
27
+ end
28
+
29
+ def ui
30
+ @ui ||= Thor::Base.shell.new
31
+ end
32
+ end
33
+ end
34
+
35
+ Excon.defaults[:connect_timeout] = 120
36
+ Excon.defaults[:read_timeout] = 120
37
+ Excon.defaults[:write_timeout] = 120