centurion 1.0.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,10 @@
1
+ .bundle
2
+ .config
3
+ coverage/
4
+ *.gem
5
+ Gemfile.lock
6
+ pkg/
7
+ *.rbc
8
+ *.swp
9
+ *.swo
10
+ tmp/
@@ -0,0 +1,42 @@
1
+ Contributors
2
+ ============
3
+
4
+ Post-release
5
+ ------------
6
+
7
+ Your name could be here!
8
+
9
+ Pre-release
10
+ -----------
11
+
12
+ A lot of work was done on Centurion before its public release. Those statistics
13
+ are not reflected in the current history. Many of the ideas for the project
14
+ originally came from Nic Benders and the Insights team at New Relic. At the
15
+ time of public release, here's how the contributors list looked:
16
+
17
+ * 343 commits
18
+ * 12 branches
19
+ * 0 releases
20
+ * 19 contributors
21
+
22
+ ```
23
+ kmatthias 97 commits / 3,870 ++ / 3,658 --
24
+ nic 36 commits / 1,481 ++ / 971 --
25
+ bryan 15 commits / 355 ++ / 236 --
26
+ andrew 47 commits / 326 ++ / 55 --
27
+ jdearing 31 commits / 300 ++ / 152 --
28
+ jon 17 commits / 242 ++ / 26 --
29
+ dkerr 7 commits / 212 ++ / 6 --
30
+ aaron 2 commits / 111 ++ / 23 --
31
+ bkayser 7 commits / 106 ++ / 74 --
32
+ jlepper 7 commits / 96 ++ / 46 --
33
+ didip 5 commits / 93 ++ / 18 --
34
+ dcelis 4 commits / 66 ++ / 12 --
35
+ jthurman 2 commits / 40 ++ / 6 --
36
+ amjith 7 commits / 38 ++ / 22 --
37
+ brett 1 commit / 26 ++ / 0 --
38
+ merlyn 2 commits / 15 ++ / 2 --
39
+ jonathan 1 commit / 13 ++ / 10 --
40
+ ehyland 5 commits / 6 ++ / 2 --
41
+ franky 1 commit / 2 ++ / 0 --
42
+ ```
data/Gemfile ADDED
@@ -0,0 +1,6 @@
1
+ source 'https://WiPJLMV2QdeRQ2XRu3Yt@gem.fury.io/newrelic/'
2
+ source 'https://rubygems.org'
3
+
4
+ # Specify your gem's dependencies in centurion.gemspec
5
+ gemspec
6
+ gem 'rspec_junit_formatter', group: "test"
data/LICENSE ADDED
@@ -0,0 +1,19 @@
1
+ Copyright (c) 2014 New Relic, Inc. All Rights Reserved.
2
+
3
+ Permission is hereby granted, free of charge, to any person obtaining a copy
4
+ of this software and associated documentation files (the "Software"), to deal
5
+ in the Software without restriction, including without limitation the rights
6
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
7
+ copies of the Software, and to permit persons to whom the Software is
8
+ furnished to do so, subject to the following conditions:
9
+
10
+ The above copyright notice and this permission notice shall be included in
11
+ all copies or substantial portions of the Software.
12
+
13
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
14
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
15
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
16
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
17
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
18
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
19
+ THE SOFTWARE.
@@ -0,0 +1,239 @@
1
+ Centurion
2
+ =========
3
+
4
+ A deployment tool for Docker. Takes containers from a Docker registry and runs
5
+ them on a fleet of hosts with the correct environment variables, host mappings,
6
+ and port mappings. Supports rolling deployments out of the box, and makes it
7
+ easy to ship applications to Docker servers.
8
+
9
+ We're using it to run our production infrastructure.
10
+
11
+ Centurion works in a two part deployment process where the build process ships
12
+ a container to the registry, and Centurion ships containers from the registry
13
+ to the Docker fleet. Registry support is handled by the Docker command line
14
+ tools directly so you can use anything they currently support via the normal
15
+ registry mechanism.
16
+
17
+ If you haven't been using a registry, you should read up on how to do that
18
+ before trying to deploy anything with Centurion. Docker, Inc [provide
19
+ repositories](https://index.docker.io/), including the main public repository.
20
+ Alternatively, you can [host your
21
+ own](https://github.com/dotcloud/docker-registry), or
22
+ [Quay.io](https://quay.io) is another commercial option.
23
+
24
+ Status
25
+ ------
26
+
27
+ This project is under active development! The initial release on GitHub contains
28
+ one roll-up commit of all our internal code. But all internal development will
29
+ now be on public GitHub. See the CONTRIBUTORS file for the contributors to the
30
+ original internal project.
31
+
32
+ Installation
33
+ ------------
34
+
35
+ Centurion is a Ruby gem. It assumes that you have a working, modern-ish Ruby
36
+ (1.9.3 or higher). On Ubuntu 12.04 you can install this with the `ruby-1.9.1`
37
+ system package, for example. On OSX this is best accomplished via `rbenv` and
38
+ `ruby-build` which can be installed with [Homebrew](http://brew.sh/) or from
39
+ [GitHub](https://github.com/sstephenson/rbenv).
40
+
41
+ Once you have a running, modern Ruby, you simply:
42
+
43
+ ```
44
+ $ gem install centurion
45
+ ```
46
+
47
+ With rbenv you will now need to do an `rbenv rehash` and the commands should
48
+ be available. With a non-rbenv install, assuming the gem dir is in your path,
49
+ the commands should just work now.
50
+
51
+ Configuration
52
+ -------------
53
+
54
+ Centurion expects to find configuration tasks in the current working directory.
55
+ Soon it will also support reading configuration from etcd.
56
+
57
+ We recommend putting all your configuration for multiple applications into a
58
+ single repo rather than spreading it around by project. This allows a central
59
+ choke point on configuration changes between applications and tends to work
60
+ well with the hand-off in many organizations between the build and deploy
61
+ steps. If you only have one application, or don't need this you can
62
+ decentralize the config into each repo.
63
+
64
+ It will look for configuration files in either `./config/centurion` or `.`.
65
+
66
+ The pattern at New Relic is to have a configs repo with a `Gemfile` that
67
+ sources the Centurion gem. If you want Centurion to set up the structure for
68
+ you and to create a sample config, you can simply run `centurionize` once you
69
+ have the Ruby Gem installed.
70
+
71
+ Centurion ships with a simple scaffolding tool that will setup a new config repo for
72
+ you, as well as scaffold individual project configs. Here's how you run it:
73
+
74
+ ```bash
75
+ $ centurionize -p <your_project>
76
+ ```
77
+
78
+ `centurionize` relies on Bundler being installed already. Running the command
79
+ will have the following effects:
80
+
81
+ * Ensure that a `config/centurion` directory exists
82
+ * Scaffold an example config for your project (you can specify the registry)
83
+ * Ensure that a Gemfile is present
84
+ * Ensure that Centurion is in the Gemfile (if absent it just appends it)
85
+
86
+ Any time you add a new project you can scaffold it in the same manner even
87
+ in the same repo.
88
+
89
+ ###Writing configs
90
+
91
+ If you used `centurionize` you will have a base config scaffolded for you.
92
+ But you'll still need to specify all of your configuration.
93
+
94
+ Configs are in the form of a Rake task that uses a built-in DSL to make them
95
+ easy to write. Here's a sample config for a project called "radio-radio" that
96
+ would go into `config/centurion/radio-radio.rake`:
97
+
98
+ ```ruby
99
+ namespace :environment do
100
+ task :common do
101
+ set :image, 'example.com/newrelic/radio-radio'
102
+ host 'docker-server-1.example.com'
103
+ host 'docker-server-2.example.com'
104
+ end
105
+
106
+ desc 'Staging environment'
107
+ task :staging => :common do
108
+ set_current_environment(:staging)
109
+ env_var YOUR_ENV: 'staging'
110
+ env_var MY_DB: 'radio-db.example.com'
111
+ host_port 10234, container_port: 9292
112
+ host_port 10235, container_port: 9293
113
+ hot_volume '/mnt/volume1', container_volume: '/mnt/volume2'
114
+ end
115
+
116
+ desc 'Production environment'
117
+ task :production => :common do
118
+ set_current_environment(:production)
119
+ env_var YOUR_ENV: 'production'
120
+ env_var MY_DB: 'radio-db-prod.example.com'
121
+ host_port 22234, container_port: 9292
122
+ host_port 23235, container_port: 9293
123
+ end
124
+ end
125
+ ```
126
+
127
+ This sets up a staging and production environment and defines a `common` task
128
+ that will be run in either case. Note the dependency call in the task
129
+ definition for the `production` and `staging` tasks. Additionally, it
130
+ defines some host ports to map and sets which servers to deploy to. Some
131
+ configuration will provided to the containers at startup time, in the form of
132
+ environment variables.
133
+
134
+ All of the DSL items (`host_port`, `host_volume`, `env_var`, `host`) can be
135
+ specified more than once and will append to the configuration.
136
+
137
+ Deploying
138
+ ---------
139
+
140
+ Centurion supports a number of tasks out of the box that make working with
141
+ distributed containers easy. Here are some examples:
142
+
143
+ ###Do a rolling deployment to a fleet of Docker servers
144
+
145
+ A rolling deployment will stop and start each container one at a time to make
146
+ sure that the application stays available from the viewpoint of the load
147
+ balancer. Currently assumes a valid response in the 200 range on
148
+ `/status/check` by default. This is configurable by adding
149
+ `set(:status_endpoint, '/somewhere')` in your config.
150
+
151
+ ````bash
152
+ $ bundle exec centurion -p radio-radio -e staging -a rolling_deploy
153
+ ````
154
+
155
+ ###Deploy a project to a fleet of Docker servers
156
+
157
+ This will hard stop, then start containers on all the specified hosts. This
158
+ is not recommended for apps where one endpoint needs to be available at all
159
+ times.
160
+
161
+ ````bash
162
+ $ bundle exec centurion -p radio-radio -e staging -a deploy
163
+ ````
164
+
165
+ ###Deploy a bash console on a host
166
+
167
+ This will give you a command line shell with all of your existing environment
168
+ passed to the container. The `CMD` from the `Dockerfile` will be replaced
169
+ with `/bin/bash`. It will use the first host from the host list.
170
+
171
+ ````bash
172
+ $ bundle exec centurion -p radio-radio -e staging -a deploy_console
173
+ ````
174
+
175
+ ###List all the tags running on your servers for a particular project
176
+
177
+ Returns a nicely-formatted list of all the current tags and which machines they
178
+ are running on. Gives a unique list of tags across all hosts as well. This is
179
+ useful for validating the state of the deployment in the case where something
180
+ goes wrong mid-deploy.
181
+
182
+ ```bash
183
+ $ bundle exec centurion -p radio-radio -e staging -a list:running_container_tags
184
+ ```
185
+
186
+ ###List all the containers currently running for this project
187
+
188
+ Returns a (as yet not very nicely formatted) list of all the containers for
189
+ this project on each of the servers from the config.
190
+
191
+ ```bash
192
+ $ bundle exec centurion -p radio-radio -e staging -a list:running_containers
193
+ ```
194
+
195
+ ###List registry containers
196
+
197
+ Returns a list of all the containers for this project in the registry.
198
+
199
+ ````bash
200
+ $ bundle exec centurion -p radio-radio -e staging -a list
201
+ ````
202
+
203
+ Future Additions
204
+ ----------------
205
+
206
+ We're currently looking at the following feature additions:
207
+
208
+ * [etcd](https://github.com/coreos/etcd) integration for configs and discovery
209
+ * Add the ability to show all the available tasks on the command line
210
+ * Certificate authentication
211
+ * Customized tasks
212
+ * Dynamic host allocation to a pool of servers
213
+
214
+ Contributions
215
+ -------------
216
+
217
+ Contributions are more than welcome. Bug reports with specific reproduction
218
+ steps are great. If you have a code contribution you'd like to make, open a
219
+ pull request with suggested code.
220
+
221
+ Pull requests should:
222
+
223
+ * Clearly state their intent in the title
224
+ * Have a description that explains the need for the changes
225
+ * Include tests!
226
+ * Not break the public API
227
+
228
+ If you are simply looking to contribute to the project, taking on one of the
229
+ items in the "Future Additions" section above would be a great place to start.
230
+ Ping us to let us know you're working on it by opening a GitHub Issue on the
231
+ project.
232
+
233
+ By contributing to this project you agree that you are granting New Relic a
234
+ non-exclusive, non-revokable, no-cost license to use the code, algorithms,
235
+ patents, and ideas in that code in our products if we so choose. You also agree
236
+ the code is provided as-is and you provide no warranties as to its fitness or
237
+ correctness for any purpose
238
+
239
+ Copyright (c) 2014 New Relic, Inc. All rights reserved.
@@ -0,0 +1,15 @@
1
+ $: << File.expand_path("lib")
2
+ require 'bundler/gem_tasks'
3
+
4
+ begin
5
+ require 'rspec/core/rake_task'
6
+
7
+ RSpec::Core::RakeTask.new(:spec) do |t|
8
+ t.rspec_opts = %w[--color --format=documentation]
9
+ t.pattern = "spec/**/*_spec.rb"
10
+ end
11
+
12
+ task :default => [:spec]
13
+ rescue LoadError
14
+ # don't generate Rspec tasks if we don't have it installed
15
+ end
@@ -0,0 +1,70 @@
1
+ #!/usr/bin/env ruby
2
+ $: << File.join(File.dirname(__FILE__), '..', 'lib')
3
+ require_relative '../lib/centurion'
4
+ require 'capistrano_dsl'
5
+
6
+ self.extend Capistrano::DSL
7
+ self.extend Centurion::DeployDSL
8
+ self.extend Centurion::Logging
9
+
10
+ #
11
+ # Initialize Rake engine
12
+ #
13
+ require 'rake'
14
+ Rake.application.options.trace = true
15
+
16
+ task_dir = File.expand_path(File.join(File.dirname(__FILE__), *%w{.. lib tasks}))
17
+ Dir.glob(File.join(task_dir, '*.rake')).each { |file| load file }
18
+
19
+ possible_environments = %w[development integration staging production local_integration]
20
+ def possible_environments.to_s
21
+ join(', ').sub(/, (\w+)$/, ', or \1')
22
+ end
23
+
24
+ #
25
+ # Trollup option setup
26
+ #
27
+ require 'trollop'
28
+
29
+ opts = Trollop::options do
30
+ opt :project, 'project (dirac, rubicon...)', type: String, required: true, short: '-p'
31
+ opt :environment, "environment (#{possible_environments})", type: String, required: true, short: '-e'
32
+ opt :action, 'action (deploy, list...)', type: String, default: 'list', short: '-a'
33
+ opt :image, 'image (analytics/rubicon...)', type: String, required: false, short: '-i'
34
+ opt :tag, 'tag (latest...)', type: String, required: false, short: '-t'
35
+ opt :hosts, 'hosts, comma separated', type: String, required: false, short: '-h'
36
+ opt :docker_path, 'path to docker executable (default: docker)', type: String, default: 'docker', short: '-d'
37
+ end
38
+
39
+ unless possible_environments.include?(opts[:environment])
40
+ Trollop::die :environment, "is unknown; must be #{possible_environments}"
41
+ end
42
+
43
+ set_current_environment(opts[:environment].to_sym)
44
+ set :project, opts[:project]
45
+ set :environment, opts[:environment]
46
+
47
+ # Load the per-project config and execute the task for the current environment
48
+ projects_dir = File.join(Dir.getwd(), 'config', 'centurion')
49
+ config_file = "#{opts[:project]}.rake"
50
+ if File.exists?(File.join(projects_dir, config_file))
51
+ load File.join(File.join(projects_dir, config_file))
52
+ elsif File.exists?(config_file)
53
+ load config_file
54
+ else
55
+ raise "Can't find '#{config_file}'!"
56
+ end
57
+ invoke("environment:#{opts[:environment]}")
58
+
59
+ # Override the config with command line values if given
60
+ set :image, opts[:image] if opts[:image]
61
+ set :tag, opts[:tag] if opts[:tag]
62
+ set :hosts, opts[:hosts].split(",") if opts[:hosts]
63
+
64
+ # Default tag should be "latest"
65
+ set :tag, 'latest' unless any?(:tag)
66
+
67
+ # Specify a path to docker executable
68
+ set :docker_path, opts[:docker_path]
69
+
70
+ invoke(opts[:action])
@@ -0,0 +1,60 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ require 'fileutils'
4
+ require 'trollop'
5
+
6
+ opts = Trollop::options do
7
+ opt :project, 'The short name (no spaces) of the project to scaffold', required: true, type: String
8
+ opt :registry_base, 'The base url for your registry (ex: example.com/yourcompany/) slash terminated', default: ''
9
+ opt :centurion_dir, 'The base dir for centurion configs', default: File.join(Dir.getwd, 'config', 'centurion')
10
+ end
11
+
12
+ project_file = File.join(opts[:centurion_dir], "#{opts[:project]}.rake")
13
+
14
+ unless Dir.exists?(opts[:centurion_dir])
15
+ puts "Creating #{opts[:centurion_dir]}"
16
+ FileUtils.mkdir_p(opts[:centurion_dir])
17
+ end
18
+
19
+ unless File.exists?(project_file)
20
+ puts "Writing example config to #{project_file}"
21
+ File.write(project_file, <<-EOS.gsub(/^\s{4}/, ''))
22
+ namespace :environment do
23
+ task :common do
24
+ set :image, '#{opts[:registry_base]}#{opts[:project]}'
25
+ end
26
+
27
+ desc 'Staging environment'
28
+ task :staging => :common do
29
+ set_current_environment(:staging)
30
+ #env_var YOUR_ENV: 'staging'
31
+ #host_port 10234, container_port: 9292
32
+ #host 'docker-server-staging-1.example.com'
33
+ #host 'docker-server-staging-2.example.com'
34
+ end
35
+
36
+ desc 'Production environment'
37
+ task :production => :common do
38
+ set_current_environment(:production)
39
+ #env_var YOUR_ENV: 'production'
40
+ #host_port 23235, container_port: 9293
41
+ #host 'docker-server-prod-1.example.com'
42
+ #host 'docker-server-prod-2.example.com'
43
+ #host_volume '/mnt/volume1', container_volume: '/mnt/volume1'
44
+ end
45
+ end
46
+ EOS
47
+ end
48
+
49
+ gemfile = File.join(opts[:centurion_dir], *%w{.. .. Gemfile})
50
+ unless File.exists?(gemfile)
51
+ raise "Error creating Gemfile: $!" unless system("bash -c 'cd #{File.dirname(gemfile)} && bundle init'")
52
+ end
53
+
54
+ unless File.read(gemfile) =~ /centurion/s
55
+ puts 'Adding Centurion to the Gemfile'
56
+ File.open(gemfile, 'a') { |f| f << "gem 'centurion'" }
57
+ puts "\n\nRemember to run `bundle install` before running Centurion\n\n"
58
+ end
59
+
60
+ puts 'Done!'