orchestration 0.4.3 → 0.4.8
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +65 -5
- data/lib/orchestration/environment.rb +8 -1
- data/lib/orchestration/service_check.rb +12 -2
- data/lib/orchestration/services/mixins/configuration_base.rb +9 -1
- data/lib/orchestration/templates/Dockerfile.erb +8 -2
- data/lib/orchestration/templates/database.yml.erb +2 -2
- data/lib/orchestration/templates/entrypoint.sh.erb +2 -3
- data/lib/orchestration/templates/makefile_macros.mk.erb +0 -1
- data/lib/orchestration/templates/mongoid.yml.erb +2 -6
- data/lib/orchestration/templates/orchestration.mk.erb +16 -10
- data/lib/orchestration/templates/rabbitmq.yml.erb +2 -2
- data/lib/orchestration/version.rb +1 -1
- metadata +2 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 3658b497eae57e7f247ef8f703ddb90ca6c041a2aba5409c61e54291a8e80c22
|
4
|
+
data.tar.gz: 4a7a242492fd376b62ccb30c2dbf6e5e9163aa202e5fefbbb75b1fef4eab315c
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: d855b7c336167af5db5444f96319de4236d7bb19fdb2ad0554b0c1730485ff57f3d1e3e9d8317138fe60797ac4cf4011bf76f168c1e8948ce3e242e0a012d7c6
|
7
|
+
data.tar.gz: 98c840d9cdd7272fa2f167373b5df176f9bb0a6d6b22f4be897dd64290f51c17087a20c8f5765e1fdd68890beae7f9de74fdde839441d014d873dfb1effd1996
|
data/README.md
CHANGED
@@ -1,5 +1,11 @@
|
|
1
1
|
# Orchestration
|
2
2
|
|
3
|
+
```
|
4
|
+
I've got two tickets to the game
|
5
|
+
It'd be great if I could take you to it this Sunday
|
6
|
+
--Nickelback
|
7
|
+
```
|
8
|
+
|
3
9
|
## Overview
|
4
10
|
|
5
11
|
_Orchestration_ aims to provide a convenient and consistent process for working with _Rails_ and _Docker_ without obscuring underlying components.
|
@@ -29,7 +35,7 @@ The below screenshot demonstrates _Orchestration_ being installed in a brand new
|
|
29
35
|
Add _Orchestration_ to your Gemfile:
|
30
36
|
|
31
37
|
```ruby
|
32
|
-
gem 'orchestration', '~> 0.4.
|
38
|
+
gem 'orchestration', '~> 0.4.8'
|
33
39
|
```
|
34
40
|
|
35
41
|
Install:
|
@@ -144,7 +150,7 @@ make push
|
|
144
150
|
|
145
151
|
### Development
|
146
152
|
|
147
|
-
An [`.env`](https://docs.docker.com/compose/env-file/) is created automatically in your project root. This file is _not_ stored in version control. Set all application environment variables in this file.
|
153
|
+
An [`.env` file](https://docs.docker.com/compose/env-file/) is created automatically in your project root. This file is _not_ stored in version control. Set all application environment variables in this file.
|
148
154
|
|
149
155
|
#### Launching a development server
|
150
156
|
|
@@ -176,16 +182,59 @@ make test
|
|
176
182
|
|
177
183
|
Note that _Orchestration_ will wait for all services to become fully available (i.e. running and providing valid responses) before attempting to run tests. This is specifically intended to facilitate testing in continuous integration environments.
|
178
184
|
|
185
|
+
_(See [sidecar containers](#sidecar-containers) if you are running your test/development server inside _Docker_)_.
|
186
|
+
|
187
|
+
Dependencies will be launched and then tested for readiness. The retry limit and interval time for readiness tests can be controlled by the following environment variables:
|
188
|
+
|
189
|
+
```
|
190
|
+
ORCHESTRATION_RETRY_LIMIT # default: 10
|
191
|
+
ORCHESTRATION_RETRY_INTERVAL # default: 5 [seconds]
|
192
|
+
```
|
193
|
+
|
194
|
+
### (Local) Production
|
195
|
+
|
196
|
+
Run a production environment locally to simulate your deployment platform:
|
197
|
+
|
198
|
+
```
|
199
|
+
make start env=production
|
200
|
+
```
|
201
|
+
|
179
202
|
### Deployment to Docker Swarm
|
180
203
|
|
181
|
-
To deploy your application to _Docker Swarm_:
|
204
|
+
To deploy your application to a local _Docker Swarm_ use:
|
205
|
+
```
|
206
|
+
make deploy
|
207
|
+
```
|
208
|
+
|
209
|
+
#### Deploy to a remote swarm
|
210
|
+
|
211
|
+
To connect via _SSH_ to a remote swarm and deploy, pass the `manager` parameter:
|
182
212
|
```
|
183
213
|
make deploy manager=user@manager.swarm.example.com
|
184
214
|
```
|
185
215
|
|
186
|
-
|
216
|
+
#### Use a custom stack name
|
217
|
+
|
218
|
+
The [_Docker_ stack](https://docs.docker.com/engine/reference/commandline/stack/) name defaults to the name of your repository (as defined in `.orchesration.yml`) and the _Rails_ environment, e.g. `anvil_production`.
|
219
|
+
|
220
|
+
To override this default, pass the `project_name` parameter:
|
221
|
+
```
|
222
|
+
make deploy project_name=acme_anvil_production
|
223
|
+
```
|
224
|
+
|
225
|
+
This variable will also be available as `COMPOSE_PROJECT_NAME` for use within your `docker-compose.yml`. e.g. to explicitly name a network after the project name:
|
226
|
+
|
227
|
+
```yaml
|
228
|
+
networks:
|
229
|
+
myapp:
|
230
|
+
name: "${COMPOSE_PROJECT_NAME}"
|
231
|
+
```
|
232
|
+
|
233
|
+
#### Use a custom `.env` file
|
234
|
+
|
235
|
+
Specify a path to a local `.env` file (see [Docker Compose documentation](https://docs.docker.com/compose/environment-variables/#the-env-file)):
|
187
236
|
```
|
188
|
-
make deploy env_file=/path/to/.env
|
237
|
+
make deploy env_file=/path/to/.env
|
189
238
|
```
|
190
239
|
|
191
240
|
Note that the following two variables _must_ be set in the relevant `.env` file (will look in the current working directory if no path provided):
|
@@ -268,6 +317,17 @@ An [entrypoint](https://docs.docker.com/engine/reference/builder/#entrypoint) sc
|
|
268
317
|
* Adds a route `host.docker.internal` to the host machine running the container (mimicking the same route provided by _Docker_ itself on _Windows_ and _OS
|
269
318
|
X_).
|
270
319
|
|
320
|
+
<a name="sidecar-containers"></a>
|
321
|
+
## Sidecar Containers
|
322
|
+
|
323
|
+
If you need to start dependency services (databases, etc.) and connect to them from a _Docker_ container (an example use case of this would be running tests in _Jenkins_ using its _Docker_ agent) then the container that runs your tests must join the same _Docker_ network as your dependency services.
|
324
|
+
|
325
|
+
To do this automatically, pass the `sidecar` parameter to the `start` or `test` targets:
|
326
|
+
|
327
|
+
```bash
|
328
|
+
make test sidecar=1
|
329
|
+
```
|
330
|
+
|
271
331
|
<a name="rabbitmq-configuration"></a>
|
272
332
|
## RabbitMQ Configuration
|
273
333
|
|
@@ -18,7 +18,14 @@ module Orchestration
|
|
18
18
|
end
|
19
19
|
|
20
20
|
def database_url
|
21
|
-
|
21
|
+
case environment
|
22
|
+
when 'development'
|
23
|
+
ENV['DEVELOPMENT_DATABASE_URL'] || ENV['DATABASE_URL']
|
24
|
+
when 'test'
|
25
|
+
ENV['TEST_DATABASE_URL'] || ENV['DATABASE_URL']
|
26
|
+
else
|
27
|
+
ENV['DATABASE_URL']
|
28
|
+
end
|
22
29
|
end
|
23
30
|
|
24
31
|
def mongo_url
|
@@ -2,8 +2,8 @@
|
|
2
2
|
|
3
3
|
module Orchestration
|
4
4
|
class ServiceCheck
|
5
|
-
ATTEMPT_LIMIT = 10
|
6
|
-
RETRY_INTERVAL =
|
5
|
+
ATTEMPT_LIMIT = ENV.fetch('ORCHESTRATION_RETRY_LIMIT', '10').to_i
|
6
|
+
RETRY_INTERVAL = ENV.fetch('ORCHESTRATION_RETRY_INTERVAL', '5').to_i
|
7
7
|
|
8
8
|
def initialize(service, terminal, options = {})
|
9
9
|
@service = service
|
@@ -16,6 +16,8 @@ module Orchestration
|
|
16
16
|
end
|
17
17
|
|
18
18
|
def run
|
19
|
+
return echo_missing unless @service.configuration.configured?
|
20
|
+
|
19
21
|
echo_start
|
20
22
|
success = attempt_connection
|
21
23
|
echo_ready if success
|
@@ -37,6 +39,14 @@ module Orchestration
|
|
37
39
|
false
|
38
40
|
end
|
39
41
|
|
42
|
+
def echo_missing
|
43
|
+
@terminal.write(
|
44
|
+
@service_name.to_sym,
|
45
|
+
"#{@service.configuration.error} (skipping)",
|
46
|
+
:error
|
47
|
+
)
|
48
|
+
end
|
49
|
+
|
40
50
|
def echo_start
|
41
51
|
@terminal.write(@service_name.to_sym, '', :status)
|
42
52
|
end
|
@@ -3,7 +3,7 @@
|
|
3
3
|
module Orchestration
|
4
4
|
module Services
|
5
5
|
module ConfigurationBase
|
6
|
-
attr_reader :service_name, :env
|
6
|
+
attr_reader :service_name, :env, :error
|
7
7
|
|
8
8
|
def self.included(base)
|
9
9
|
base.extend(ClassMethods)
|
@@ -33,6 +33,14 @@ module Orchestration
|
|
33
33
|
@service_name
|
34
34
|
end
|
35
35
|
|
36
|
+
def configured?
|
37
|
+
port
|
38
|
+
true
|
39
|
+
rescue KeyError => error
|
40
|
+
@error = error
|
41
|
+
false
|
42
|
+
end
|
43
|
+
|
36
44
|
def port
|
37
45
|
return @env.app_port if @service_name == 'app'
|
38
46
|
|
@@ -1,8 +1,13 @@
|
|
1
1
|
FROM ruby:<%= ruby_version %>
|
2
2
|
ARG BUNDLE_BITBUCKET__ORG
|
3
3
|
ARG BUNDLE_GITHUB__COM
|
4
|
-
|
5
|
-
|
4
|
+
ARG GIT_COMMIT
|
5
|
+
RUN curl -sL https://deb.nodesource.com/setup_10.x | bash - \
|
6
|
+
&& apt-get update \
|
7
|
+
&& DEBIAN_FRONTEND=noninteractive apt-get install -y \
|
8
|
+
nodejs \
|
9
|
+
gosu \
|
10
|
+
sendmail \
|
6
11
|
&& rm -rf /var/lib/apt/lists/* \
|
7
12
|
&& gem install bundler \
|
8
13
|
&& mkdir /app<%if defined?(Webpacker) %> \
|
@@ -19,6 +24,7 @@ RUN . /root/.bashrc && yarn install
|
|
19
24
|
<% end %>
|
20
25
|
ADD .build/context.tar .
|
21
26
|
<% if defined?(Webpacker) %>RUN . /root/.bashrc && NODE_ENV=production RAILS_ENV=production yarn install && NODE_ENV=production RAILS_ENV=production SECRET_KEY_BASE=abc123 bundle exec rake assets:precompile<% elsif Rake::Task.tasks.map(&:name).include?('assets:precompile') %>RUN NODE_ENV=production RAILS_ENV=production SECRET_KEY_BASE=abc123 bundle exec rake assets:precompile<% end %>
|
27
|
+
RUN echo "${GIT_COMMIT}" > /app/GIT_COMMIT
|
22
28
|
HEALTHCHECK --interval=<%= healthcheck['interval'] %> \
|
23
29
|
--timeout=<%= healthcheck['timeout'] %> \
|
24
30
|
--start-period=<%= healthcheck['start_period'] %> \
|
@@ -21,8 +21,8 @@ test:
|
|
21
21
|
database: <%= compose.call(nil).database_adapter.credentials['database'] %>
|
22
22
|
# Useful for certain continuous integration environments (e.g. Jenkins in
|
23
23
|
# Docker) where the DB hostname may be a service name rather than `127.0.0.1`:
|
24
|
-
<%% if ENV.key?('
|
25
|
-
url: <%%= ENV['
|
24
|
+
<%% if ENV.key?('TEST_DATABASE_URL') %>
|
25
|
+
url: <%%= ENV['TEST_DATABASE_URL'] %>
|
26
26
|
<%% end %>
|
27
27
|
<% end %>
|
28
28
|
|
@@ -1,16 +1,12 @@
|
|
1
1
|
development:
|
2
2
|
clients:
|
3
3
|
default:
|
4
|
-
|
5
|
-
- 127.0.0.1:<%= compose.call('development').local_port('mongo') %>
|
6
|
-
database: development_db
|
4
|
+
uri: <%= "#{'<' + '%' + '='} ENV.fetch('MONGO_URL', 'mongodb://127.0.0.1:#{compose.call('development').local_port('mongo')}/development_db') #{'%' + '>'}" %>
|
7
5
|
|
8
6
|
test:
|
9
7
|
clients:
|
10
8
|
default:
|
11
|
-
|
12
|
-
- 127.0.0.1:<%= compose.call('test').local_port('mongo') %>
|
13
|
-
database: test_db
|
9
|
+
uri: <%= "#{'<' + '%' + '='} ENV.fetch('MONGO_URL', 'mongodb://127.0.0.1:#{compose.call('test').local_port('mongo')}/test_db') #{'%' + '>'}" %>
|
14
10
|
|
15
11
|
production:
|
16
12
|
clients:
|
@@ -39,11 +39,15 @@ endif
|
|
39
39
|
docker_organization=$(shell bash ${orchestration_dir}/yaml.bash docker_organization)
|
40
40
|
docker_repository=$(shell bash ${orchestration_dir}/yaml.bash docker_repository)
|
41
41
|
|
42
|
+
ifeq (,$(project_name))
|
43
|
+
project_name = ${docker_repository}_${env}
|
44
|
+
endif
|
45
|
+
|
42
46
|
compose_base=env HOST_UID=$(shell id -u) \
|
43
47
|
DOCKER_ORGANIZATION="${docker_organization}" \
|
44
48
|
DOCKER_REPOSITORY="${docker_repository}" \
|
49
|
+
COMPOSE_PROJECT_NAME="${project_name}" \
|
45
50
|
docker-compose \
|
46
|
-
-p "${docker_repository}_${env}" \
|
47
51
|
-f "${orchestration_dir}/docker-compose.yml"
|
48
52
|
|
49
53
|
git_branch ?= $(if $(branch),$(branch),$(shell git rev-parse --abbrev-ref HEAD))
|
@@ -62,19 +66,22 @@ all: build
|
|
62
66
|
### Container management commands ###
|
63
67
|
|
64
68
|
.PHONY: start
|
69
|
+
ifndef network
|
70
|
+
start: network := ${docker_repository}_${env}_default
|
71
|
+
endif
|
65
72
|
start: _clean-logs
|
66
73
|
@$(call print,'${yellow}Starting containers${reset} ...')
|
67
74
|
ifeq (${env},$(filter ${env},test development))
|
68
|
-
@${compose} up
|
69
|
-
@[ '${
|
75
|
+
@${compose} up --detach --force-recreate --renew-anon-volumes ${services} ${log} || ${fail}
|
76
|
+
@[ -n '${sidecar}' ] && \
|
70
77
|
( \
|
71
78
|
docker network connect '${network}' '$(shell hostname)' ${log} \
|
72
79
|
|| \
|
73
80
|
$(call println,'${yellow}Warning${reset}: Unable to join network: "${yellow}${network}${reset}". Container will not be able to connect to dependency services.') \
|
74
81
|
) \
|
75
|
-
|| ( [ '${
|
82
|
+
|| ( [ -z '${sidecar}' ] || ${fail} )
|
76
83
|
else
|
77
|
-
@${compose} up
|
84
|
+
@${compose} up --detach --scale app=$${instances:-1} ${services} ${log} || ${fail}
|
78
85
|
endif
|
79
86
|
@$(call printrawln,' ${green}started${reset} ${tick}')
|
80
87
|
@$(call println,'${yellow}Waiting for services to become available${reset} ...')
|
@@ -204,11 +211,9 @@ endif
|
|
204
211
|
ifndef manager
|
205
212
|
@$(call println_error,'Missing `manager` parameter: `make deploy manager=swarm-manager.example.com`')
|
206
213
|
endif
|
207
|
-
deploy: env := production
|
208
|
-
deploy: project_name = ${docker_repository}_${env}
|
209
214
|
deploy: path := $(shell mktemp -d)
|
210
|
-
deploy: RAILS_ENV
|
211
|
-
deploy: RACK_ENV
|
215
|
+
deploy: RAILS_ENV := ${env}
|
216
|
+
deploy: RACK_ENV := ${env}
|
212
217
|
deploy: DOCKER_TAG = ${git_version}
|
213
218
|
deploy:
|
214
219
|
@$(call println,'${yellow}Deploying stack via${reset} ${green}${manager}${reset} ...') && \
|
@@ -277,12 +282,13 @@ build:
|
|
277
282
|
<% if defined?(Webpacker) %> @git show ${git_branch}:./package.json > ${orchestration_dir}/.build/package.json 2>${stderr} || ${fail}<% end %>
|
278
283
|
<% if defined?(Webpacker) %> @git show ${git_branch}:./yarn.lock > ${orchestration_dir}/.build/yarn.lock 2>${stderr} || ${fail}<% end %>
|
279
284
|
@git archive --format 'tar' -o '${context}' '${git_branch}' ${log} || ${fail}
|
280
|
-
@temp=$$(mktemp -d) ; ( cd "$${temp}" &&
|
285
|
+
@temp=$$(mktemp -d) ; ( cd "$${temp}" && tar -uvf '${context}' . ) ${log} || ${fail}
|
281
286
|
@$(call printrawln,'${green}complete.${reset} ${tick}')
|
282
287
|
@$(call print,'${yellow}Building image${reset} ...')
|
283
288
|
@docker build \
|
284
289
|
--build-arg BUNDLE_GITHUB__COM \
|
285
290
|
--build-arg BUNDLE_BITBUCKET__ORG \
|
291
|
+
--build-arg GIT_COMMIT='${git_version}' \
|
286
292
|
-t ${docker_organization}/${docker_repository} \
|
287
293
|
-t ${docker_organization}/${docker_repository}:${git_version} \
|
288
294
|
${orchestration_dir}/ ${log_progress} || ${fail}
|
@@ -1,11 +1,11 @@
|
|
1
1
|
<% if compose.call('development').services.key?('rabbitmq') %>
|
2
2
|
development:
|
3
|
-
url: amqp://127.0.0.1
|
3
|
+
url: <%= "#{'<' + '%' + '='} ENV.fetch('RABBITMQ_URL', 'amqp://127.0.0.1:#{compose.call('development').local_port('rabbitmq')}') #{'%' + '>'}" %>
|
4
4
|
<% end %>
|
5
5
|
|
6
6
|
<% if compose.call('test').services.key?('rabbitmq') %>
|
7
7
|
test:
|
8
|
-
url: amqp://127.0.0.1
|
8
|
+
url: <%= "#{'<' + '%' + '='} ENV.fetch('RABBITMQ_URL', 'amqp://127.0.0.1:#{compose.call('test').local_port('rabbitmq')}') #{'%' + '>'}" %>
|
9
9
|
<% end %>
|
10
10
|
|
11
11
|
production:
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: orchestration
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.4.
|
4
|
+
version: 0.4.8
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Bob Farrell
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2019-
|
11
|
+
date: 2019-10-09 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: database_url
|