centurion 1.0.6

Sign up to get free protection for your applications and to get access to all the features.
@@ -0,0 +1,10 @@
1
+ .bundle
2
+ .config
3
+ coverage/
4
+ *.gem
5
+ Gemfile.lock
6
+ pkg/
7
+ *.rbc
8
+ *.swp
9
+ *.swo
10
+ tmp/
@@ -0,0 +1,42 @@
1
+ Contributors
2
+ ============
3
+
4
+ Post-release
5
+ ------------
6
+
7
+ Your name could be here!
8
+
9
+ Pre-release
10
+ -----------
11
+
12
+ A lot of work was done on Centurion before its public release. Those statistics
13
+ are not reflected in the current history. Many of the ideas for the project
14
+ originally came from Nic Benders and the Insights team at New Relic. At the
15
+ time of public release, here's how the contributors list looked:
16
+
17
+ * 343 commits
18
+ * 12 branches
19
+ * 0 releases
20
+ * 19 contributors
21
+
22
+ ```
23
+ kmatthias 97 commits / 3,870 ++ / 3,658 --
24
+ nic 36 commits / 1,481 ++ / 971 --
25
+ bryan 15 commits / 355 ++ / 236 --
26
+ andrew 47 commits / 326 ++ / 55 --
27
+ jdearing 31 commits / 300 ++ / 152 --
28
+ jon 17 commits / 242 ++ / 26 --
29
+ dkerr 7 commits / 212 ++ / 6 --
30
+ aaron 2 commits / 111 ++ / 23 --
31
+ bkayser 7 commits / 106 ++ / 74 --
32
+ jlepper 7 commits / 96 ++ / 46 --
33
+ didip 5 commits / 93 ++ / 18 --
34
+ dcelis 4 commits / 66 ++ / 12 --
35
+ jthurman 2 commits / 40 ++ / 6 --
36
+ amjith 7 commits / 38 ++ / 22 --
37
+ brett 1 commit / 26 ++ / 0 --
38
+ merlyn 2 commits / 15 ++ / 2 --
39
+ jonathan 1 commit / 13 ++ / 10 --
40
+ ehyland 5 commits / 6 ++ / 2 --
41
+ franky 1 commit / 2 ++ / 0 --
42
+ ```
data/Gemfile ADDED
@@ -0,0 +1,6 @@
1
+ source 'https://WiPJLMV2QdeRQ2XRu3Yt@gem.fury.io/newrelic/'
2
+ source 'https://rubygems.org'
3
+
4
+ # Specify your gem's dependencies in centurion.gemspec
5
+ gemspec
6
+ gem 'rspec_junit_formatter', group: "test"
data/LICENSE ADDED
@@ -0,0 +1,19 @@
1
+ Copyright (c) 2014 New Relic, Inc. All Rights Reserved.
2
+
3
+ Permission is hereby granted, free of charge, to any person obtaining a copy
4
+ of this software and associated documentation files (the "Software"), to deal
5
+ in the Software without restriction, including without limitation the rights
6
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
7
+ copies of the Software, and to permit persons to whom the Software is
8
+ furnished to do so, subject to the following conditions:
9
+
10
+ The above copyright notice and this permission notice shall be included in
11
+ all copies or substantial portions of the Software.
12
+
13
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
14
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
15
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
16
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
17
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
18
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
19
+ THE SOFTWARE.
@@ -0,0 +1,239 @@
1
+ Centurion
2
+ =========
3
+
4
+ A deployment tool for Docker. Takes containers from a Docker registry and runs
5
+ them on a fleet of hosts with the correct environment variables, host mappings,
6
+ and port mappings. Supports rolling deployments out of the box, and makes it
7
+ easy to ship applications to Docker servers.
8
+
9
+ We're using it to run our production infrastructure.
10
+
11
+ Centurion works in a two part deployment process where the build process ships
12
+ a container to the registry, and Centurion ships containers from the registry
13
+ to the Docker fleet. Registry support is handled by the Docker command line
14
+ tools directly so you can use anything they currently support via the normal
15
+ registry mechanism.
16
+
17
+ If you haven't been using a registry, you should read up on how to do that
18
+ before trying to deploy anything with Centurion. Docker, Inc [provide
19
+ repositories](https://index.docker.io/), including the main public repository.
20
+ Alternatively, you can [host your
21
+ own](https://github.com/dotcloud/docker-registry), or
22
+ [Quay.io](https://quay.io) is another commercial option.
23
+
24
+ Status
25
+ ------
26
+
27
+ This project is under active development! The initial release on GitHub contains
28
+ one roll-up commit of all our internal code. But all internal development will
29
+ now be on public GitHub. See the CONTRIBUTORS file for the contributors to the
30
+ original internal project.
31
+
32
+ Installation
33
+ ------------
34
+
35
+ Centurion is a Ruby gem. It assumes that you have a working, modern-ish Ruby
36
+ (1.9.3 or higher). On Ubuntu 12.04 you can install this with the `ruby-1.9.1`
37
+ system package, for example. On OSX this is best accomplished via `rbenv` and
38
+ `ruby-build` which can be installed with [Homebrew](http://brew.sh/) or from
39
+ [GitHub](https://github.com/sstephenson/rbenv).
40
+
41
+ Once you have a running, modern Ruby, you simply:
42
+
43
+ ```
44
+ $ gem install centurion
45
+ ```
46
+
47
+ With rbenv you will now need to do an `rbenv rehash` and the commands should
48
+ be available. With a non-rbenv install, assuming the gem dir is in your path,
49
+ the commands should just work now.
50
+
51
+ Configuration
52
+ -------------
53
+
54
+ Centurion expects to find configuration tasks in the current working directory.
55
+ Soon it will also support reading configuration from etcd.
56
+
57
+ We recommend putting all your configuration for multiple applications into a
58
+ single repo rather than spreading it around by project. This allows a central
59
+ choke point on configuration changes between applications and tends to work
60
+ well with the hand-off in many organizations between the build and deploy
61
+ steps. If you only have one application, or don't need this you can
62
+ decentralize the config into each repo.
63
+
64
+ It will look for configuration files in either `./config/centurion` or `.`.
65
+
66
+ The pattern at New Relic is to have a configs repo with a `Gemfile` that
67
+ sources the Centurion gem. If you want Centurion to set up the structure for
68
+ you and to create a sample config, you can simply run `centurionize` once you
69
+ have the Ruby Gem installed.
70
+
71
+ Centurion ships with a simple scaffolding tool that will setup a new config repo for
72
+ you, as well as scaffold individual project configs. Here's how you run it:
73
+
74
+ ```bash
75
+ $ centurionize -p <your_project>
76
+ ```
77
+
78
+ `centurionize` relies on Bundler being installed already. Running the command
79
+ will have the following effects:
80
+
81
+ * Ensure that a `config/centurion` directory exists
82
+ * Scaffold an example config for your project (you can specify the registry)
83
+ * Ensure that a Gemfile is present
84
+ * Ensure that Centurion is in the Gemfile (if absent it just appends it)
85
+
86
+ Any time you add a new project you can scaffold it in the same manner even
87
+ in the same repo.
88
+
89
+ ###Writing configs
90
+
91
+ If you used `centurionize` you will have a base config scaffolded for you.
92
+ But you'll still need to specify all of your configuration.
93
+
94
+ Configs are in the form of a Rake task that uses a built-in DSL to make them
95
+ easy to write. Here's a sample config for a project called "radio-radio" that
96
+ would go into `config/centurion/radio-radio.rake`:
97
+
98
+ ```ruby
99
+ namespace :environment do
100
+ task :common do
101
+ set :image, 'example.com/newrelic/radio-radio'
102
+ host 'docker-server-1.example.com'
103
+ host 'docker-server-2.example.com'
104
+ end
105
+
106
+ desc 'Staging environment'
107
+ task :staging => :common do
108
+ set_current_environment(:staging)
109
+ env_var YOUR_ENV: 'staging'
110
+ env_var MY_DB: 'radio-db.example.com'
111
+ host_port 10234, container_port: 9292
112
+ host_port 10235, container_port: 9293
113
+ hot_volume '/mnt/volume1', container_volume: '/mnt/volume2'
114
+ end
115
+
116
+ desc 'Production environment'
117
+ task :production => :common do
118
+ set_current_environment(:production)
119
+ env_var YOUR_ENV: 'production'
120
+ env_var MY_DB: 'radio-db-prod.example.com'
121
+ host_port 22234, container_port: 9292
122
+ host_port 23235, container_port: 9293
123
+ end
124
+ end
125
+ ```
126
+
127
+ This sets up a staging and production environment and defines a `common` task
128
+ that will be run in either case. Note the dependency call in the task
129
+ definition for the `production` and `staging` tasks. Additionally, it
130
+ defines some host ports to map and sets which servers to deploy to. Some
131
+ configuration will provided to the containers at startup time, in the form of
132
+ environment variables.
133
+
134
+ All of the DSL items (`host_port`, `host_volume`, `env_var`, `host`) can be
135
+ specified more than once and will append to the configuration.
136
+
137
+ Deploying
138
+ ---------
139
+
140
+ Centurion supports a number of tasks out of the box that make working with
141
+ distributed containers easy. Here are some examples:
142
+
143
+ ###Do a rolling deployment to a fleet of Docker servers
144
+
145
+ A rolling deployment will stop and start each container one at a time to make
146
+ sure that the application stays available from the viewpoint of the load
147
+ balancer. Currently assumes a valid response in the 200 range on
148
+ `/status/check` by default. This is configurable by adding
149
+ `set(:status_endpoint, '/somewhere')` in your config.
150
+
151
+ ````bash
152
+ $ bundle exec centurion -p radio-radio -e staging -a rolling_deploy
153
+ ````
154
+
155
+ ###Deploy a project to a fleet of Docker servers
156
+
157
+ This will hard stop, then start containers on all the specified hosts. This
158
+ is not recommended for apps where one endpoint needs to be available at all
159
+ times.
160
+
161
+ ````bash
162
+ $ bundle exec centurion -p radio-radio -e staging -a deploy
163
+ ````
164
+
165
+ ###Deploy a bash console on a host
166
+
167
+ This will give you a command line shell with all of your existing environment
168
+ passed to the container. The `CMD` from the `Dockerfile` will be replaced
169
+ with `/bin/bash`. It will use the first host from the host list.
170
+
171
+ ````bash
172
+ $ bundle exec centurion -p radio-radio -e staging -a deploy_console
173
+ ````
174
+
175
+ ###List all the tags running on your servers for a particular project
176
+
177
+ Returns a nicely-formatted list of all the current tags and which machines they
178
+ are running on. Gives a unique list of tags across all hosts as well. This is
179
+ useful for validating the state of the deployment in the case where something
180
+ goes wrong mid-deploy.
181
+
182
+ ```bash
183
+ $ bundle exec centurion -p radio-radio -e staging -a list:running_container_tags
184
+ ```
185
+
186
+ ###List all the containers currently running for this project
187
+
188
+ Returns a (as yet not very nicely formatted) list of all the containers for
189
+ this project on each of the servers from the config.
190
+
191
+ ```bash
192
+ $ bundle exec centurion -p radio-radio -e staging -a list:running_containers
193
+ ```
194
+
195
+ ###List registry containers
196
+
197
+ Returns a list of all the containers for this project in the registry.
198
+
199
+ ````bash
200
+ $ bundle exec centurion -p radio-radio -e staging -a list
201
+ ````
202
+
203
+ Future Additions
204
+ ----------------
205
+
206
+ We're currently looking at the following feature additions:
207
+
208
+ * [etcd](https://github.com/coreos/etcd) integration for configs and discovery
209
+ * Add the ability to show all the available tasks on the command line
210
+ * Certificate authentication
211
+ * Customized tasks
212
+ * Dynamic host allocation to a pool of servers
213
+
214
+ Contributions
215
+ -------------
216
+
217
+ Contributions are more than welcome. Bug reports with specific reproduction
218
+ steps are great. If you have a code contribution you'd like to make, open a
219
+ pull request with suggested code.
220
+
221
+ Pull requests should:
222
+
223
+ * Clearly state their intent in the title
224
+ * Have a description that explains the need for the changes
225
+ * Include tests!
226
+ * Not break the public API
227
+
228
+ If you are simply looking to contribute to the project, taking on one of the
229
+ items in the "Future Additions" section above would be a great place to start.
230
+ Ping us to let us know you're working on it by opening a GitHub Issue on the
231
+ project.
232
+
233
+ By contributing to this project you agree that you are granting New Relic a
234
+ non-exclusive, non-revokable, no-cost license to use the code, algorithms,
235
+ patents, and ideas in that code in our products if we so choose. You also agree
236
+ the code is provided as-is and you provide no warranties as to its fitness or
237
+ correctness for any purpose
238
+
239
+ Copyright (c) 2014 New Relic, Inc. All rights reserved.
@@ -0,0 +1,15 @@
1
+ $: << File.expand_path("lib")
2
+ require 'bundler/gem_tasks'
3
+
4
+ begin
5
+ require 'rspec/core/rake_task'
6
+
7
+ RSpec::Core::RakeTask.new(:spec) do |t|
8
+ t.rspec_opts = %w[--color --format=documentation]
9
+ t.pattern = "spec/**/*_spec.rb"
10
+ end
11
+
12
+ task :default => [:spec]
13
+ rescue LoadError
14
+ # don't generate Rspec tasks if we don't have it installed
15
+ end
@@ -0,0 +1,70 @@
1
+ #!/usr/bin/env ruby
2
+ $: << File.join(File.dirname(__FILE__), '..', 'lib')
3
+ require_relative '../lib/centurion'
4
+ require 'capistrano_dsl'
5
+
6
+ self.extend Capistrano::DSL
7
+ self.extend Centurion::DeployDSL
8
+ self.extend Centurion::Logging
9
+
10
+ #
11
+ # Initialize Rake engine
12
+ #
13
+ require 'rake'
14
+ Rake.application.options.trace = true
15
+
16
+ task_dir = File.expand_path(File.join(File.dirname(__FILE__), *%w{.. lib tasks}))
17
+ Dir.glob(File.join(task_dir, '*.rake')).each { |file| load file }
18
+
19
+ possible_environments = %w[development integration staging production local_integration]
20
+ def possible_environments.to_s
21
+ join(', ').sub(/, (\w+)$/, ', or \1')
22
+ end
23
+
24
+ #
25
+ # Trollup option setup
26
+ #
27
+ require 'trollop'
28
+
29
+ opts = Trollop::options do
30
+ opt :project, 'project (dirac, rubicon...)', type: String, required: true, short: '-p'
31
+ opt :environment, "environment (#{possible_environments})", type: String, required: true, short: '-e'
32
+ opt :action, 'action (deploy, list...)', type: String, default: 'list', short: '-a'
33
+ opt :image, 'image (analytics/rubicon...)', type: String, required: false, short: '-i'
34
+ opt :tag, 'tag (latest...)', type: String, required: false, short: '-t'
35
+ opt :hosts, 'hosts, comma separated', type: String, required: false, short: '-h'
36
+ opt :docker_path, 'path to docker executable (default: docker)', type: String, default: 'docker', short: '-d'
37
+ end
38
+
39
+ unless possible_environments.include?(opts[:environment])
40
+ Trollop::die :environment, "is unknown; must be #{possible_environments}"
41
+ end
42
+
43
+ set_current_environment(opts[:environment].to_sym)
44
+ set :project, opts[:project]
45
+ set :environment, opts[:environment]
46
+
47
+ # Load the per-project config and execute the task for the current environment
48
+ projects_dir = File.join(Dir.getwd(), 'config', 'centurion')
49
+ config_file = "#{opts[:project]}.rake"
50
+ if File.exists?(File.join(projects_dir, config_file))
51
+ load File.join(File.join(projects_dir, config_file))
52
+ elsif File.exists?(config_file)
53
+ load config_file
54
+ else
55
+ raise "Can't find '#{config_file}'!"
56
+ end
57
+ invoke("environment:#{opts[:environment]}")
58
+
59
+ # Override the config with command line values if given
60
+ set :image, opts[:image] if opts[:image]
61
+ set :tag, opts[:tag] if opts[:tag]
62
+ set :hosts, opts[:hosts].split(",") if opts[:hosts]
63
+
64
+ # Default tag should be "latest"
65
+ set :tag, 'latest' unless any?(:tag)
66
+
67
+ # Specify a path to docker executable
68
+ set :docker_path, opts[:docker_path]
69
+
70
+ invoke(opts[:action])
@@ -0,0 +1,60 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ require 'fileutils'
4
+ require 'trollop'
5
+
6
+ opts = Trollop::options do
7
+ opt :project, 'The short name (no spaces) of the project to scaffold', required: true, type: String
8
+ opt :registry_base, 'The base url for your registry (ex: example.com/yourcompany/) slash terminated', default: ''
9
+ opt :centurion_dir, 'The base dir for centurion configs', default: File.join(Dir.getwd, 'config', 'centurion')
10
+ end
11
+
12
+ project_file = File.join(opts[:centurion_dir], "#{opts[:project]}.rake")
13
+
14
+ unless Dir.exists?(opts[:centurion_dir])
15
+ puts "Creating #{opts[:centurion_dir]}"
16
+ FileUtils.mkdir_p(opts[:centurion_dir])
17
+ end
18
+
19
+ unless File.exists?(project_file)
20
+ puts "Writing example config to #{project_file}"
21
+ File.write(project_file, <<-EOS.gsub(/^\s{4}/, ''))
22
+ namespace :environment do
23
+ task :common do
24
+ set :image, '#{opts[:registry_base]}#{opts[:project]}'
25
+ end
26
+
27
+ desc 'Staging environment'
28
+ task :staging => :common do
29
+ set_current_environment(:staging)
30
+ #env_var YOUR_ENV: 'staging'
31
+ #host_port 10234, container_port: 9292
32
+ #host 'docker-server-staging-1.example.com'
33
+ #host 'docker-server-staging-2.example.com'
34
+ end
35
+
36
+ desc 'Production environment'
37
+ task :production => :common do
38
+ set_current_environment(:production)
39
+ #env_var YOUR_ENV: 'production'
40
+ #host_port 23235, container_port: 9293
41
+ #host 'docker-server-prod-1.example.com'
42
+ #host 'docker-server-prod-2.example.com'
43
+ #host_volume '/mnt/volume1', container_volume: '/mnt/volume1'
44
+ end
45
+ end
46
+ EOS
47
+ end
48
+
49
+ gemfile = File.join(opts[:centurion_dir], *%w{.. .. Gemfile})
50
+ unless File.exists?(gemfile)
51
+ raise "Error creating Gemfile: $!" unless system("bash -c 'cd #{File.dirname(gemfile)} && bundle init'")
52
+ end
53
+
54
+ unless File.read(gemfile) =~ /centurion/s
55
+ puts 'Adding Centurion to the Gemfile'
56
+ File.open(gemfile, 'a') { |f| f << "gem 'centurion'" }
57
+ puts "\n\nRemember to run `bundle install` before running Centurion\n\n"
58
+ end
59
+
60
+ puts 'Done!'