orchestration 0.3.4 → 0.3.5
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/Makefile +3 -0
- data/README.md +195 -59
- data/TODO +11 -1
- data/config/locales/en.yml +4 -3
- data/lib/orchestration/docker_compose/app_service.rb +1 -1
- data/lib/orchestration/docker_compose/mongo_service.rb +5 -9
- data/lib/orchestration/install_generator.rb +3 -3
- data/lib/orchestration/services/app/configuration.rb +2 -2
- data/lib/orchestration/services/configuration_base.rb +5 -0
- data/lib/orchestration/services/database/configuration.rb +2 -5
- data/lib/orchestration/services/mongo/configuration.rb +33 -12
- data/lib/orchestration/services/rabbitmq/configuration.rb +1 -3
- data/lib/orchestration/templates/application.mk.erb +1 -1
- data/lib/orchestration/templates/deploy.mk.erb +31 -10
- data/lib/orchestration/templates/{Makefile.erb → orchestration.mk.erb} +51 -27
- data/lib/orchestration/version.rb +1 -1
- data/orchestration.gemspec +2 -1
- metadata +18 -4
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: efffd8ac9ad681e2dd8341b2fb72dd709a118a5d862c670b19e0c146e15aaf5a
|
4
|
+
data.tar.gz: f62fd59095ff2f6ee7b615e834107625ba51ae63241e7c3d0b1766e53ff28048
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 441ff4f44916715e7999c0899fa94c232f76e53e43e94cdd151f1e19f4a63e41d4175ab99f3efca4bc3b33aa3e3f5f29061f4957041943a013fa80937e2c750a
|
7
|
+
data.tar.gz: 1ceb6dac11bb36e7eca7c645a390655675e1c7860c743e9e189854a668a18abb488cbcd2ce619d13ec60d7a725541ce56e3bf062dd56eb03ae82a5c37f2ec1d4
|
data/Makefile
CHANGED
data/README.md
CHANGED
@@ -1,101 +1,237 @@
|
|
1
1
|
# Orchestration
|
2
2
|
|
3
|
-
|
3
|
+
_Orchestration_ is a toolkit for testing, building, and deploying _Ruby_ (including _Rails_) applications in _Docker_.
|
4
4
|
|
5
|
-
|
5
|
+
## Getting Started
|
6
6
|
|
7
|
-
|
7
|
+
[_Docker_](https://www.docker.com/get-started) and [_Docker Compose_](https://docs.docker.com/compose/install/) must be installed on your system.
|
8
8
|
|
9
|
-
|
9
|
+
### Install
|
10
10
|
|
11
|
-
|
12
|
-
* MongoDB
|
13
|
-
* PostgreSQL
|
14
|
-
* RabbitMQ
|
15
|
-
|
16
|
-
## Installation
|
17
|
-
|
18
|
-
Add this line to your application's Gemfile:
|
11
|
+
Add _Orchestration_ to your Gemfile:
|
19
12
|
|
20
13
|
```ruby
|
21
|
-
gem 'orchestration', '~> 0.3.
|
14
|
+
gem 'orchestration', '~> 0.3.5'
|
22
15
|
```
|
23
16
|
|
24
|
-
|
25
|
-
|
26
|
-
|
17
|
+
Install:
|
18
|
+
|
19
|
+
```bash
|
20
|
+
bundle install
|
27
21
|
```
|
28
22
|
|
29
|
-
|
23
|
+
### Setup
|
30
24
|
|
31
|
-
|
25
|
+
Generate configuration files:
|
32
26
|
|
33
|
-
|
27
|
+
```bash
|
28
|
+
bin/rake orchestration:install
|
29
|
+
```
|
34
30
|
|
35
|
-
|
36
|
-
* `.orchestration.yml` - _Orchestration_ internal configuration, e.g. _Docker_ username.
|
37
|
-
* `Makefile` - Adds `orchestration/Makefile` as an `include` to avoid clobbering any existing _make_ commands.
|
38
|
-
* `orchestration/docker-compose.yml` - a custom-made set of services to allow you to run your application's dependencies locally.
|
39
|
-
* `orchestration/Dockerfile` - a ready-to-use _Docker_ build script which should need minimal (if any) modification to build your _Rails_ project.
|
40
|
-
* `orchestration/entrypoint.sh` - Container setup for your Docker application.
|
41
|
-
* `orchestration/Makefile` - provides easy access to all _Orchestration_ utilities.
|
42
|
-
* `orchestration/yaml.bash` - A _bash_ _YAML_ parser (used by _make_ utilities).
|
31
|
+
Commit changes:
|
43
32
|
|
44
|
-
|
33
|
+
```bash
|
34
|
+
git add .
|
35
|
+
git commit -m "Add Orchestration gem"
|
36
|
+
```
|
45
37
|
|
46
|
-
|
38
|
+
## Usage
|
39
|
+
|
40
|
+
Start your dependencies:
|
47
41
|
|
48
|
-
|
42
|
+
```bash
|
43
|
+
make start
|
44
|
+
```
|
49
45
|
|
50
|
-
|
51
|
-
|
46
|
+
Log in to your _Docker_ registry, then build and push your image:
|
47
|
+
```bash
|
48
|
+
docker login
|
49
|
+
make build push
|
50
|
+
```
|
52
51
|
|
53
|
-
|
52
|
+
Make a compact, portable, production-ready tarball:
|
53
|
+
```
|
54
|
+
make bundle
|
55
|
+
```
|
54
56
|
|
57
|
+
Copy tarball to your server and unpack:
|
55
58
|
```bash
|
56
|
-
|
57
|
-
|
59
|
+
tar xf deploy.tar
|
60
|
+
cd <your-app-name>/
|
58
61
|
```
|
59
62
|
|
60
|
-
|
61
|
-
|
63
|
+
Add required config to `.env` file:
|
64
|
+
```
|
65
|
+
# .env
|
66
|
+
SECRET_KEY_BASE=<your-secure-token>
|
67
|
+
LISTEN_PORT=8080
|
68
|
+
VIRTUAL_HOST=yourdomain.com
|
69
|
+
```
|
62
70
|
|
63
|
-
|
71
|
+
Load three instances of your container, load-balanced by _Nginx_, with dependecies:
|
72
|
+
```
|
73
|
+
make start instances=3
|
74
|
+
```
|
64
75
|
|
65
|
-
|
66
|
-
```
|
67
|
-
|
76
|
+
Or deploy to _Docker Swarm_:
|
77
|
+
```
|
78
|
+
make deploy
|
68
79
|
```
|
69
80
|
|
70
|
-
|
81
|
+
## Table of Contents
|
82
|
+
|
83
|
+
<!-- toc -->
|
84
|
+
|
85
|
+
- [Configuration Files](#configuration-files)
|
86
|
+
* [Makefile](#makefile)
|
87
|
+
* [.orchestration.yml](#orchestrationyml)
|
88
|
+
* [.env](#env)
|
89
|
+
* [orchestration/Dockerfile](#orchestrationdockerfile)
|
90
|
+
* [orchestration/entrypoint.sh](#orchestrationentrypointsh)
|
91
|
+
* [orchestration/docker-compose.yml](#orchestrationdocker-composeyml)
|
92
|
+
* [config/unicorn.rb](#configunicornrb)
|
93
|
+
- [Building](#building)
|
94
|
+
- [Build Environment](#build-environment)
|
95
|
+
- [Commands](#commands)
|
96
|
+
- [Dependencies](#dependencies)
|
97
|
+
|
98
|
+
<!-- tocstop -->
|
99
|
+
|
100
|
+
## Configuration Files
|
101
|
+
|
102
|
+
_Orchestration_ autogenerates boilerplate configuration based on your application's requirements and configuration.
|
103
|
+
|
104
|
+
When supported dependencies are detected they will be created as services in your _Docker Compose_ configurations ready for use in testing, development, and production.
|
105
|
+
|
106
|
+
The following files are created on setup:
|
107
|
+
|
108
|
+
### Makefile
|
109
|
+
|
110
|
+
Contains an `include` for the main _Orchestration_ `Makefile`. If this file already exists then the `include` will be added to the top of the file.
|
111
|
+
|
112
|
+
### .orchestration.yml
|
113
|
+
|
114
|
+
_Orchestration_-specific configuration such as your _Docker_ registry and username.
|
115
|
+
|
116
|
+
### .env
|
117
|
+
|
118
|
+
Specify any environment variables (e.g. `SECRET_KEY_BASE`) your application will need to run in production mode.
|
119
|
+
|
120
|
+
The following two variables _must_ be defined:
|
71
121
|
|
72
|
-
To build and push your image run:
|
73
122
|
```bash
|
74
|
-
|
123
|
+
VIRTUAL_HOST=localhost
|
124
|
+
LISTEN_PORT=3000
|
75
125
|
```
|
76
126
|
|
77
|
-
|
127
|
+
When running in production mode your application will be load-balanced by _Nginx_ proxy and available at http://localhost:3000/
|
128
|
+
|
129
|
+
Take a look at `orchestration/docker-compose.production.yml` to see what variables will be exposed to various containers.
|
130
|
+
|
131
|
+
### orchestration/Dockerfile
|
132
|
+
|
133
|
+
The basic requirements of a typical _Rails_ application. It is optimised for build speed and will automatically build assets (with our without `Webpacker`).
|
134
|
+
|
135
|
+
### orchestration/entrypoint.sh
|
136
|
+
|
137
|
+
Entrypoint script to handle user switching, permissions, stale pidfiles, etc.
|
138
|
+
|
139
|
+
### orchestration/docker-compose.yml
|
140
|
+
|
141
|
+
Along with the base `docker-compose.yml` a separate configuration is created for each environment. An override file is also generated.
|
142
|
+
|
143
|
+
See related documentation:
|
144
|
+
|
145
|
+
https://docs.docker.com/compose/extends/
|
146
|
+
|
147
|
+
* `orchestration/docker-compose.yml`
|
148
|
+
* `orchestration/docker-compose.test.yml`
|
149
|
+
* `orchestration/docker-compose.development.yml`
|
150
|
+
* `orchestration/docker-compose.production.yml`
|
151
|
+
* `orchestration/docker-compose.override.yml`
|
152
|
+
|
153
|
+
You can modify these files to suit your requirements.
|
154
|
+
|
155
|
+
The famous [`jwilder/nginx-proxy`](https://github.com/jwilder/nginx-proxy) is used to load-balance replicas of your application when running in production.
|
156
|
+
|
157
|
+
### config/unicorn.rb
|
158
|
+
|
159
|
+
If not already present, a [Unicorn](https://bogomips.org/unicorn/) configuration will be created. This is the default server when running in production.
|
160
|
+
|
161
|
+
## Building
|
162
|
+
|
163
|
+
_Orchestration_ provides tools for building your application as a _Docker_ image.
|
164
|
+
|
78
165
|
```bash
|
79
|
-
|
80
|
-
$ make docker-push
|
166
|
+
make build
|
81
167
|
```
|
82
168
|
|
83
|
-
|
169
|
+
Running `make build` does the following:
|
84
170
|
|
85
|
-
|
171
|
+
* Takes a snapshot of your application from current _Git_ `HEAD`. Only committed files are included.
|
172
|
+
* Copies your `Gemfile` and installs your bundle (optimised for _Docker_ image caching).
|
173
|
+
* Tags your image with your configured username/organisation, repository, and the current commit hash (abbreviated) of `HEAD`, e.g. `myorg/myapp:abc123`
|
86
174
|
|
87
|
-
|
88
|
-
|
175
|
+
Your image can then be pushed to your configured registry (use `docker login` before running):
|
176
|
+
|
177
|
+
```
|
178
|
+
make push
|
179
|
+
```
|
180
|
+
|
181
|
+
## Build Environment
|
182
|
+
|
183
|
+
The following environment variables will be passed as `ARG` variables when building your image:
|
184
|
+
|
185
|
+
```
|
186
|
+
BUNDLE_BITBUCKET__ORG
|
187
|
+
BUNDLE_GITHUB__COM
|
89
188
|
```
|
90
189
|
|
91
|
-
|
190
|
+
See related documentation:
|
92
191
|
|
93
|
-
|
94
|
-
|
95
|
-
|
96
|
-
|
97
|
-
|
98
|
-
|
192
|
+
* https://help.github.com/articles/creating-a-personal-access-token-for-the-command-line/
|
193
|
+
* https://confluence.atlassian.com/bitbucket/app-passwords-828781300.html
|
194
|
+
|
195
|
+
## Commands
|
196
|
+
|
197
|
+
_Orchestration_ provides a number of `make` commands to help you work with your application in _Docker_.
|
198
|
+
|
199
|
+
All commands respect `RAILS_ENV` or `RACK_ENV`. Alternatively you can pass `env` to any command:
|
200
|
+
```bash
|
201
|
+
make config env=production
|
99
202
|
```
|
100
203
|
|
101
|
-
|
204
|
+
The following commands are implemented:
|
205
|
+
|
206
|
+
| Command | Description |
|
207
|
+
|---|---|
|
208
|
+
| `start` | Start all containers and wait for their services to become available. In production mode, pass `instances=N` to start `N` replicas of your app. |
|
209
|
+
| `stop` | Stop all containers. |
|
210
|
+
| `logs` | Tail logs for all containers. |
|
211
|
+
| `config` | Output the full configuration for the current environment with all variables substituted. |
|
212
|
+
| `compose` | Output full `docker-compose` command. Run arbitrary commands for your environment, e.g. `$(make compose env=test) ps --services` |
|
213
|
+
| `test-setup` | Launch test dependency containers, wait for them to become ready, run database migrations. Call before running tests in a CI environment. |
|
214
|
+
| `wait` | Wait for all dependencies to be ready (i.e. verify that database is up and accepting connections, etc.). |
|
215
|
+
| `wait-database` | Wait for database container (supported: _PostgreSQL_ and _MySQL_) to become available. |
|
216
|
+
| `wait-mongo` | Wait for _Mongo_ container to become available.
|
217
|
+
| `wait-rabbitmq` | Wait for _RabbitMQ_ container to become available. |
|
218
|
+
| `wait-nginx_proxy` | Wait for _Nginx_ container to become available (`production` only). |
|
219
|
+
| `wait-app` | Wait for main application container to become available (`production` only). |
|
220
|
+
| `build` | Build your application as a _Docker_ image. |
|
221
|
+
| `push` | Push the current version of your application image to a _Docker_ registry. |
|
222
|
+
| `bundle` | Create `deploy.tar` which contains pre-cooked production configurations and `Makefile` ready to deploy your application on any machine with _Docker_ and _Docker Compose_ installed.
|
223
|
+
|
224
|
+
## Dependencies
|
225
|
+
|
226
|
+
Dependencies are automatically detected. The following services are currently supported:
|
227
|
+
|
228
|
+
| Service | Configuration File |
|
229
|
+
|---|---|
|
230
|
+
| _PostrgeSQL_ | `config/database.yml` |
|
231
|
+
| _MySQL_ | `config/database.yml` |
|
232
|
+
| _RabbitMQ_ | `config/rabbitmq.yml` |
|
233
|
+
| _Mongo_ | `config/mongoid.yml` |
|
234
|
+
|
235
|
+
Running `bin/rake orchestration:install` will automatically add services to your _Compose_ configurations that reflect your configuration files.
|
236
|
+
|
237
|
+
For _RabbitMQ_, `config/rabbitmq.yml` should contain `host` and `port` fields for each environment.
|
data/TODO
CHANGED
@@ -1,9 +1,19 @@
|
|
1
1
|
Refactor docker-compose services - these really belong in
|
2
2
|
lib/orchestration/services/<service-name>/docker_compose.rb
|
3
3
|
|
4
|
-
|
4
|
+
Add default logging section to each service ?
|
5
5
|
|
6
6
|
Redis support
|
7
7
|
|
8
8
|
Use `Paint` instead of `Colorize` for colouring output - better licensing and no
|
9
9
|
monkeypatching `String` etc.
|
10
|
+
|
11
|
+
Make unicorn `config.rb` resilient to database connectivity failures
|
12
|
+
|
13
|
+
Group healthcheck rake tasks - run all at once in the same process. Create a
|
14
|
+
master process that generates config files and calls each healthcheck in batch.
|
15
|
+
|
16
|
+
Provide a way of declaratively configuring bespoke healthchecks.
|
17
|
+
|
18
|
+
Auto-healthcheck any services with a local port bind using the Listener
|
19
|
+
healthcheck.
|
data/config/locales/en.yml
CHANGED
@@ -16,6 +16,7 @@ en:
|
|
16
16
|
mongo:
|
17
17
|
waiting: "Waiting for Mongo: %{config}"
|
18
18
|
ready: "Mongo is ready."
|
19
|
+
bad_config: "Unable to parse Mongo config: %{path}. Expected section for one of: %{expected}"
|
19
20
|
|
20
21
|
nginx_proxy:
|
21
22
|
waiting: "Waiting for Nginx proxy: %{config}"
|
@@ -31,9 +32,9 @@ en:
|
|
31
32
|
|
32
33
|
settings:
|
33
34
|
docker:
|
34
|
-
|
35
|
-
description: "Docker registry username"
|
36
|
-
prompt: "
|
35
|
+
organization:
|
36
|
+
description: "Docker registry organization/username"
|
37
|
+
prompt: "organization"
|
37
38
|
|
38
39
|
repository:
|
39
40
|
description: "Project name (will be used as Docker registry repository)"
|
@@ -31,17 +31,13 @@ module Orchestration
|
|
31
31
|
end
|
32
32
|
|
33
33
|
def local_port
|
34
|
-
|
35
|
-
.fetch('hosts')
|
36
|
-
.first
|
37
|
-
.partition(':')
|
38
|
-
port.empty? ? remote_port : port
|
34
|
+
@config.port.nil? ? remote_port : @config.port
|
39
35
|
end
|
40
36
|
|
41
|
-
def
|
42
|
-
|
43
|
-
|
44
|
-
|
37
|
+
def client
|
38
|
+
Services::Mong::Configuration::CONFIG_KEYS.each do |key|
|
39
|
+
return @config.settings.fetch(key) if @config.settings.key?(key)
|
40
|
+
end
|
45
41
|
end
|
46
42
|
|
47
43
|
def remote_port
|
@@ -17,7 +17,7 @@ module Orchestration
|
|
17
17
|
|
18
18
|
def orchestration_configuration
|
19
19
|
path = @env.orchestration_configuration_path
|
20
|
-
@terminal.ask_setting('docker.
|
20
|
+
@terminal.ask_setting('docker.organization')
|
21
21
|
@terminal.ask_setting('docker.repository', @env.default_app_name)
|
22
22
|
relpath = relative_path(path)
|
23
23
|
return @terminal.write(:create, relpath) unless @settings.exist?
|
@@ -27,7 +27,7 @@ module Orchestration
|
|
27
27
|
end
|
28
28
|
|
29
29
|
def orchestration_makefile
|
30
|
-
content = template('
|
30
|
+
content = template('orchestration.mk', makefile_environment)
|
31
31
|
path = @env.orchestration_root.join('Makefile')
|
32
32
|
path.exist? ? update_file(path, content) : create_file(path, content)
|
33
33
|
end
|
@@ -57,7 +57,7 @@ module Orchestration
|
|
57
57
|
def gitignore
|
58
58
|
path = @env.root.join('.gitignore')
|
59
59
|
globs = %w[.build/ .deploy/ Gemfile Gemfile.lock]
|
60
|
-
entries = [
|
60
|
+
entries = %w[.env deploy.tar] + globs.map do |entry|
|
61
61
|
"#{@env.orchestration_dir_name}/#{entry}"
|
62
62
|
end
|
63
63
|
|
@@ -52,13 +52,10 @@ module Orchestration
|
|
52
52
|
ERB.new(content).result
|
53
53
|
end
|
54
54
|
|
55
|
-
def yaml(content)
|
56
|
-
YAML.safe_load(content, [], [], true) # true: Allow aliases
|
57
|
-
end
|
58
|
-
|
59
55
|
def adapter_for(name)
|
60
56
|
{
|
61
57
|
'mysql2' => adapters::Mysql2,
|
58
|
+
'mysql' => adapters::Mysql2,
|
62
59
|
'postgresql' => adapters::Postgresql,
|
63
60
|
'sqlite3' => adapters::Sqlite3
|
64
61
|
}.fetch(name).new
|
@@ -79,7 +76,7 @@ module Orchestration
|
|
79
76
|
end
|
80
77
|
|
81
78
|
def host
|
82
|
-
return nil if @adapter
|
79
|
+
return nil if @adapter && @adapter.name == 'sqlite3'
|
83
80
|
|
84
81
|
super
|
85
82
|
end
|
@@ -8,19 +8,29 @@ module Orchestration
|
|
8
8
|
|
9
9
|
self.service_name = 'mongo'
|
10
10
|
|
11
|
+
CONFIG_KEYS = %w[clients sessions hosts].freeze
|
12
|
+
|
11
13
|
def initialize(env, service_name = nil)
|
12
14
|
super
|
13
15
|
@settings = nil
|
14
16
|
return unless defined?(Mongoid)
|
15
17
|
return unless File.exist?(@env.mongoid_configuration_path)
|
16
18
|
|
17
|
-
@settings =
|
19
|
+
@settings = if clients_key.nil?
|
20
|
+
hosts_config # Host was configured at top level.
|
21
|
+
else
|
22
|
+
{ clients_key => hosts_config }
|
23
|
+
end
|
18
24
|
end
|
19
25
|
|
20
26
|
def friendly_config
|
21
27
|
"[mongoid] #{host}:#{local_port}/#{database}"
|
22
28
|
end
|
23
29
|
|
30
|
+
def port
|
31
|
+
DockerCompose::MongoService::PORT
|
32
|
+
end
|
33
|
+
|
24
34
|
private
|
25
35
|
|
26
36
|
def hosts_config
|
@@ -33,28 +43,39 @@ module Orchestration
|
|
33
43
|
end
|
34
44
|
|
35
45
|
def database
|
36
|
-
config
|
37
|
-
|
46
|
+
env_config = config.fetch(@env.environment)
|
47
|
+
return env_config.fetch('database') if env_config.key?('database')
|
48
|
+
|
49
|
+
bad_config_error if clients_key.nil?
|
50
|
+
|
51
|
+
env_config
|
38
52
|
.fetch(clients_key)
|
39
53
|
.fetch('default')
|
40
54
|
.fetch('database')
|
41
55
|
end
|
42
56
|
|
43
|
-
def port
|
44
|
-
DockerCompose::MongoService::PORT
|
45
|
-
end
|
46
|
-
|
47
57
|
def clients_key
|
48
|
-
|
58
|
+
env_config = config.fetch(@env.environment)
|
49
59
|
|
50
60
|
# Support older Mongoid versions
|
51
|
-
|
61
|
+
CONFIG_KEYS.each do |key|
|
62
|
+
return key if env_config.key?(key)
|
63
|
+
end
|
64
|
+
|
65
|
+
nil
|
52
66
|
end
|
53
67
|
|
54
68
|
def config
|
55
|
-
@config ||=
|
56
|
-
|
57
|
-
|
69
|
+
@config ||= yaml(File.read(@env.mongoid_configuration_path))
|
70
|
+
end
|
71
|
+
|
72
|
+
def bad_config_error
|
73
|
+
raise ArgumentError,
|
74
|
+
I18n.t(
|
75
|
+
'orchestration.mongo.bad_config',
|
76
|
+
path: @env.mongoid_configuration_path,
|
77
|
+
expected: CONFIG_KEYS.join(', ')
|
78
|
+
)
|
58
79
|
end
|
59
80
|
end
|
60
81
|
end
|
@@ -1,34 +1,55 @@
|
|
1
|
-
.
|
1
|
+
-include .env
|
2
|
+
export
|
2
3
|
|
3
|
-
|
4
|
+
verify-environment:
|
5
|
+
ifndef VIRTUAL_HOST
|
6
|
+
@$(error VIRTUAL_HOST must be defined in environment)
|
7
|
+
endif
|
8
|
+
|
9
|
+
ifndef LISTEN_PORT
|
10
|
+
@$(error LISTEN_PORT must be defined in environment)
|
11
|
+
endif
|
12
|
+
|
13
|
+
compose_base:=HOST_UID=$(shell id -u) \
|
4
14
|
DOCKER_USERNAME=%%USERNAME%% \
|
5
15
|
DOCKER_REPOSITORY=%%REPOSITORY%%:%%VERSION%% \
|
6
16
|
docker-compose \
|
7
17
|
-p %%REPOSITORY%% \
|
8
18
|
-f docker-compose.yml
|
9
19
|
|
10
|
-
|
20
|
+
compose:=${compose_base} -f docker-compose.production.yml -f docker-compose.override.yml
|
11
21
|
|
22
|
+
.PHONY: deploy
|
12
23
|
deploy:
|
13
24
|
@echo "Deploying application to Docker swarm..."
|
14
|
-
@${
|
25
|
+
@${compose} config | docker stack deploy -c - %%REPOSITORY%%
|
15
26
|
|
27
|
+
.PHONY: stop
|
16
28
|
stop:
|
17
29
|
@echo "Stopping containers..."
|
18
|
-
@${
|
30
|
+
@${compose} down
|
19
31
|
|
32
|
+
.PHONY: start
|
20
33
|
start:
|
21
34
|
@echo "Launching application..."
|
22
|
-
@${
|
35
|
+
@${compose} up -d --scale app=$${instances:-1}
|
23
36
|
|
37
|
+
.PHONY: config
|
24
38
|
config:
|
25
|
-
@${
|
39
|
+
@${compose} config
|
26
40
|
|
41
|
+
.PHONY: pull
|
27
42
|
pull:
|
28
|
-
@${
|
43
|
+
@${compose} pull
|
29
44
|
|
45
|
+
.PHONY: logs
|
30
46
|
logs:
|
31
|
-
@${
|
47
|
+
@${compose} logs -f
|
32
48
|
|
49
|
+
.PHONY: migrate
|
33
50
|
migrate:
|
34
|
-
@${
|
51
|
+
@${compose} run --rm app bundle exec rake db:migrate
|
52
|
+
|
53
|
+
.PHONY: compose
|
54
|
+
compose:
|
55
|
+
@echo ${compose}
|
@@ -1,19 +1,6 @@
|
|
1
|
-
.PHONY: start stop migrate bundle docker build push start logs compose config test-setup <%= wait_commands.join(' ') %>
|
2
|
-
|
3
1
|
### Environment setup ###
|
4
2
|
|
5
|
-
|
6
|
-
docker_repository:=$(shell bash ./<%= env.orchestration_dir_name %>/yaml.bash docker_repository)
|
7
|
-
|
8
|
-
compose_base:=HOST_UID=$(shell id -u) \
|
9
|
-
DOCKER_USERNAME=${docker_username} \
|
10
|
-
DOCKER_REPOSITORY=${docker_repository} \
|
11
|
-
docker-compose \
|
12
|
-
-p ${docker_repository} \
|
13
|
-
-f <%= env.orchestration_dir_name %>/docker-compose.yml
|
14
|
-
|
15
|
-
git_branch:=$(if $(branch),$(branch),$(shell git rev-parse --abbrev-ref HEAD))
|
16
|
-
git_version:=$(shell git rev-parse --short --verify ${git_branch})
|
3
|
+
SHELL:=/bin/bash
|
17
4
|
|
18
5
|
-include .env
|
19
6
|
export
|
@@ -26,46 +13,71 @@ else ifeq (,$(env))
|
|
26
13
|
env:=development
|
27
14
|
endif
|
28
15
|
|
16
|
+
ifeq (,$(wildcard ./bin/rake))
|
17
|
+
rake:=RACK_ENV=${env} RAILS_ENV=${env} bundle exec rake
|
18
|
+
else
|
19
|
+
rake:=RACK_ENV=${env} RAILS_ENV=${env} bin/rake
|
20
|
+
endif
|
21
|
+
|
22
|
+
docker_organization:=$(shell bash ./<%= env.orchestration_dir_name %>/yaml.bash docker_organization)
|
23
|
+
docker_repository:=$(shell bash ./<%= env.orchestration_dir_name %>/yaml.bash docker_repository)
|
24
|
+
|
25
|
+
compose_base:=env HOST_UID=$(shell id -u) \
|
26
|
+
DOCKER_USERNAME=${docker_organization} \
|
27
|
+
DOCKER_REPOSITORY=${docker_repository} \
|
28
|
+
docker-compose \
|
29
|
+
-p ${docker_repository} \
|
30
|
+
-f <%= env.orchestration_dir_name %>/docker-compose.yml
|
31
|
+
|
32
|
+
git_branch:=$(if $(branch),$(branch),$(shell git rev-parse --abbrev-ref HEAD))
|
33
|
+
git_version:=$(shell git rev-parse --short --verify ${git_branch})
|
34
|
+
|
29
35
|
compose:=${compose_base} -f <%= env.orchestration_dir_name %>/docker-compose.${env}.yml -f <%= env.orchestration_dir_name %>/docker-compose.override.yml
|
30
|
-
rake:=RACK_ENV=${env} RAILS_ENV=${env} bin/rake
|
31
36
|
|
32
37
|
### Container management commands ###
|
33
38
|
|
39
|
+
.PHONY: start
|
34
40
|
start:
|
35
41
|
@echo "Starting containers..."
|
36
42
|
ifeq (${env},$(filter ${env},test development))
|
37
43
|
@${compose} up -d
|
38
44
|
else
|
39
|
-
@${compose} up -d --scale app=$${
|
45
|
+
@${compose} up -d --scale app=$${instances:-1}
|
40
46
|
endif
|
41
47
|
@$(MAKE) wait
|
42
48
|
|
49
|
+
.PHONY: stop
|
43
50
|
stop:
|
44
51
|
@echo "Stopping containers..."
|
45
52
|
@${compose} down
|
46
53
|
@echo "All containers stopped."
|
47
54
|
|
55
|
+
.PHONY: logs
|
48
56
|
logs:
|
49
57
|
@${compose} logs -f
|
50
58
|
|
59
|
+
.PHONY: config
|
51
60
|
config:
|
52
61
|
@${compose} config
|
53
62
|
|
63
|
+
.PHONY: compose
|
54
64
|
compose:
|
55
|
-
|
65
|
+
@echo ${compose}
|
56
66
|
|
67
|
+
.PHONY: test-setup
|
57
68
|
test-setup:
|
58
|
-
@$(MAKE) start
|
69
|
+
@$(MAKE) start migrate env=test
|
59
70
|
|
60
71
|
### Deployment utility commands ###
|
61
72
|
|
73
|
+
.PHONY: bundle
|
62
74
|
bundle:
|
63
75
|
@echo 'Building deployment bundle...'
|
64
76
|
@rm -rf <%= env.orchestration_dir_name %>/.deploy/
|
65
77
|
@mkdir -p <%= env.orchestration_dir_name %>/.deploy/${docker_repository}/
|
66
78
|
@sed -e "s/%%VERSION%%/${git_version}/g" \
|
67
79
|
-e "s/%%REPOSITORY%%/${docker_repository}/g" \
|
68
|
-
-e "s/%%
|
80
|
+
-e "s/%%ORGANIZATION%%/${docker_organization}/g" \
|
69
81
|
<%= env.orchestration_dir_name %>/deploy.mk > \
|
70
82
|
<%= env.orchestration_dir_name %>/.deploy/${docker_repository}/Makefile
|
71
83
|
@cp <%= env.orchestration_dir_name %>/docker-compose.yml \
|
@@ -77,10 +89,11 @@ bundle:
|
|
77
89
|
|
78
90
|
### Database utility commands ###
|
79
91
|
|
92
|
+
.PHONY: migrate
|
80
93
|
migrate:
|
81
94
|
@echo "Running migrations..."
|
82
95
|
ifeq (${env},$(filter ${env},test development))
|
83
|
-
|
96
|
+
@(${rake} db:create && ${rake} db:migrate) || ${rake} db:migrate
|
84
97
|
else
|
85
98
|
@${compose} run --rm app bin/rake db:migrate RAILS_ENV=${env}
|
86
99
|
endif
|
@@ -88,21 +101,30 @@ endif
|
|
88
101
|
|
89
102
|
### Service healthcheck commands ###
|
90
103
|
|
104
|
+
.PHONY: wait
|
91
105
|
wait: <%= wait_commands.join(' ') %>
|
92
106
|
@echo "All Containers ready."
|
93
107
|
|
108
|
+
## Generic Listener healthcheck for TCP services ##
|
109
|
+
|
110
|
+
wait-listener:
|
111
|
+
@${rake} orchestration:listener:wait service=${service}
|
112
|
+
|
94
113
|
## Test/development wait commands
|
95
114
|
|
115
|
+
.PHONY: wait-database
|
96
116
|
wait-database:
|
97
117
|
ifeq (${env},$(filter ${env},test development))
|
98
118
|
@${rake} orchestration:database:wait
|
99
119
|
endif
|
100
120
|
|
121
|
+
.PHONY: wait-mongo
|
101
122
|
wait-mongo:
|
102
123
|
ifeq (${env},$(filter ${env},test development))
|
103
124
|
@${rake} orchestration:mongo:wait
|
104
125
|
endif
|
105
126
|
|
127
|
+
.PHONY: wait-rabbitmq
|
106
128
|
wait-rabbitmq:
|
107
129
|
ifeq (${env},$(filter ${env},test development))
|
108
130
|
@${rake} orchestration:rabbitmq:wait
|
@@ -110,11 +132,13 @@ endif
|
|
110
132
|
|
111
133
|
## Production wait commands
|
112
134
|
|
135
|
+
.PHONY: wait-nginx_proxy
|
113
136
|
wait-nginx_proxy:
|
114
137
|
ifneq (${env},$(filter ${env},test development))
|
115
138
|
@${rake} orchestration:nginx_proxy:wait LISTEN_PORT=${LISTEN_PORT}
|
116
139
|
endif
|
117
140
|
|
141
|
+
.PHONY: wait-app
|
118
142
|
wait-app:
|
119
143
|
ifneq (${env},$(filter ${env},test development))
|
120
144
|
@${rake} orchestration:app:wait LISTEN_PORT=${LISTEN_PORT}
|
@@ -122,24 +146,24 @@ endif
|
|
122
146
|
|
123
147
|
### Docker build commands ###
|
124
148
|
|
125
|
-
|
126
|
-
|
149
|
+
.PHONY: build
|
127
150
|
build:
|
128
151
|
@echo "Preparing build from ${git_branch}"
|
129
152
|
@mkdir -p ./<%= env.orchestration_dir_name %>/.build
|
130
153
|
@git show ${git_branch}:./Gemfile > ./<%= env.orchestration_dir_name %>/.build/Gemfile
|
131
154
|
@git show ${git_branch}:./Gemfile.lock > ./<%= env.orchestration_dir_name %>/.build/Gemfile.lock
|
132
|
-
|
133
|
-
|
155
|
+
<% if defined?(Webpacker) %> @git show ${git_branch}:./package.json > ./<%= env.orchestration_dir_name %>/.build/package.json<% end %>
|
156
|
+
<% if defined?(Webpacker) %> @git show ${git_branch}:./yarn.lock > ./<%= env.orchestration_dir_name %>/.build/yarn.lock<% end %>
|
134
157
|
@echo "Building..."
|
135
158
|
@git archive --format tar.gz -o ./<%= env.orchestration_dir_name %>/.build/context.tar.gz ${git_branch}
|
136
159
|
@docker build \
|
137
160
|
--build-arg BUNDLE_GITHUB__COM \
|
138
161
|
--build-arg BUNDLE_BITBUCKET__ORG \
|
139
|
-
-t ${
|
140
|
-
-t ${
|
162
|
+
-t ${docker_organization}/${docker_repository} \
|
163
|
+
-t ${docker_organization}/${docker_repository}:${git_version} \
|
141
164
|
./<%= env.orchestration_dir_name %>/
|
142
165
|
@echo "Build complete."
|
143
166
|
|
167
|
+
.PHONY: push
|
144
168
|
push:
|
145
|
-
docker push ${
|
169
|
+
docker push ${docker_organization}/${docker_repository}:${git_version}
|
data/orchestration.gemspec
CHANGED
@@ -9,7 +9,7 @@ Gem::Specification.new do |spec|
|
|
9
9
|
spec.name = 'orchestration'
|
10
10
|
spec.version = Orchestration::VERSION
|
11
11
|
spec.authors = ['Bob Farrell']
|
12
|
-
spec.email = ['
|
12
|
+
spec.email = ['robertanthonyfarrell@gmail.com']
|
13
13
|
|
14
14
|
spec.summary = 'Docker orchestration toolkit'
|
15
15
|
spec.description = 'Tools to help launch apps in Docker'
|
@@ -30,6 +30,7 @@ Gem::Specification.new do |spec|
|
|
30
30
|
spec.add_runtime_dependency 'unicorn', '~> 5.4'
|
31
31
|
|
32
32
|
spec.add_development_dependency 'activerecord', '~> 5.2'
|
33
|
+
spec.add_development_dependency 'betterp', '~> 0.1.3'
|
33
34
|
spec.add_development_dependency 'bundler', '~> 1.16'
|
34
35
|
spec.add_development_dependency 'bunny', '~> 2.12'
|
35
36
|
spec.add_development_dependency 'byebug', '~> 10.0'
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: orchestration
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.3.
|
4
|
+
version: 0.3.5
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Bob Farrell
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2019-01-06 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: colorize
|
@@ -80,6 +80,20 @@ dependencies:
|
|
80
80
|
- - "~>"
|
81
81
|
- !ruby/object:Gem::Version
|
82
82
|
version: '5.2'
|
83
|
+
- !ruby/object:Gem::Dependency
|
84
|
+
name: betterp
|
85
|
+
requirement: !ruby/object:Gem::Requirement
|
86
|
+
requirements:
|
87
|
+
- - "~>"
|
88
|
+
- !ruby/object:Gem::Version
|
89
|
+
version: 0.1.3
|
90
|
+
type: :development
|
91
|
+
prerelease: false
|
92
|
+
version_requirements: !ruby/object:Gem::Requirement
|
93
|
+
requirements:
|
94
|
+
- - "~>"
|
95
|
+
- !ruby/object:Gem::Version
|
96
|
+
version: 0.1.3
|
83
97
|
- !ruby/object:Gem::Dependency
|
84
98
|
name: bundler
|
85
99
|
requirement: !ruby/object:Gem::Requirement
|
@@ -278,7 +292,7 @@ dependencies:
|
|
278
292
|
version: '3.4'
|
279
293
|
description: Tools to help launch apps in Docker
|
280
294
|
email:
|
281
|
-
-
|
295
|
+
- robertanthonyfarrell@gmail.com
|
282
296
|
executables:
|
283
297
|
- console
|
284
298
|
- rspec
|
@@ -347,13 +361,13 @@ files:
|
|
347
361
|
- lib/orchestration/services/rabbitmq/healthcheck.rb
|
348
362
|
- lib/orchestration/settings.rb
|
349
363
|
- lib/orchestration/templates/Dockerfile.erb
|
350
|
-
- lib/orchestration/templates/Makefile.erb
|
351
364
|
- lib/orchestration/templates/application.mk.erb
|
352
365
|
- lib/orchestration/templates/deploy.mk.erb
|
353
366
|
- lib/orchestration/templates/docker-compose.override.yml.erb
|
354
367
|
- lib/orchestration/templates/entrypoint.sh.erb
|
355
368
|
- lib/orchestration/templates/env.erb
|
356
369
|
- lib/orchestration/templates/nginx.tmpl.erb
|
370
|
+
- lib/orchestration/templates/orchestration.mk.erb
|
357
371
|
- lib/orchestration/templates/unicorn.rb.erb
|
358
372
|
- lib/orchestration/templates/yaml.bash.erb
|
359
373
|
- lib/orchestration/terminal.rb
|