mrsk 0.0.2 → 0.0.3

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: cd8f29ada17aea2f0a3dbabe3a9b8917c4e951353eb7ada2022c5aa0c98a2dea
4
- data.tar.gz: 817eb173b1ea66238f868ddbb7683e651844767cda3ac5f74ce3bc933cb74cd8
3
+ metadata.gz: 0f5260e6f41ed0ae04931d800b7c2da3b0bc32f0042d2e140fcb2860b1a9e12a
4
+ data.tar.gz: 0c94b286b4a9ce9339678a4a4662c65dcefd6947795227c2464db50e7bf727e0
5
5
  SHA512:
6
- metadata.gz: 722b4f4b740c3110c620053e105a7ed32e3c185ef46b5b4ed08b8092c6b53b2baff5400340e102647a24db476c32d52dd262dd706895c1ae243ce490cd5a4a61
7
- data.tar.gz: 3e8abd162d7255a96fec5febab21f406bfc8f964a82b71f9c7f105b80807e46c07d6027f75abec515768eb22fd989af392e43b4bf2b0d7a811afe68d89372c0e
6
+ metadata.gz: 0f980ac1468ac3581f1b8bd8206c2471b5e79d7bc84c274f1bef2a761ceccc25905308774770a04156e68e5b1fd73d8d83926d235d287a11118fb9ea8f7fb619
7
+ data.tar.gz: 720ac72cfd88a494049a237140e774b37288f3a142cb70e3a672a91aa7094a019c0eb938c3ca9a146cd64b1d33d811b2c4b8d0373d7b9457bfaf8499bfd86943
data/README.md CHANGED
@@ -1,22 +1,18 @@
1
1
  # MRSK
2
2
 
3
- MRSK ships zero-downtime deploys of Rails apps packed as containers to any host. It uses the dynamic reverse-proxy Traefik to hold requests while the new application container is started and the old one is wound down. It works across multiple hosts at the same time, using SSHKit to execute commands.
3
+ MRSK ships zero-downtime deploys of Rails apps packed as containers to any host. It uses the dynamic reverse-proxy Traefik to hold requests while the new application container is started and the old one is wound down. It works seamlessly across multiple hosts, using SSHKit to execute commands.
4
4
 
5
5
  ## Installation
6
6
 
7
- Add the gem with `bundle add mrsk`, then run `rake mrsk:init`, and then edit the new file in `config/deploy.yml` to use the proper service name, image reference, servers to deploy on, and so on. It could look something like this:
7
+ Add the gem with `bundle add mrsk`, then run `rake mrsk:init`, and then edit the new file in `config/deploy.yml`. It could look as simple as this:
8
8
 
9
9
  ```yaml
10
10
  service: hey
11
11
  image: 37s/hey
12
12
  servers:
13
- - xxx.xxx.xxx.xxx
14
- - xxx.xxx.xxx.xxx
15
- env:
16
- DATABASE_URL: mysql2://db1/hey_production/
17
- REDIS_URL: redis://redis1:6379/1
13
+ - 192.168.0.1
14
+ - 192.168.0.2
18
15
  registry:
19
- server: registry.digitalocean.com
20
16
  username: <%= Rails.application.credentials.registry["username"] %>
21
17
  password: <%= Rails.application.credentials.registry["password"] %>
22
18
  ```
@@ -26,13 +22,13 @@ Then ensure your encrypted credentials have the registry username + password by
26
22
  ```
27
23
  registry:
28
24
  username: real-user-name
29
- password: real-password
25
+ password: real-registry-password-or-token
30
26
  ```
31
27
 
32
- Now you're ready to deploy a multi-arch image (FIXME: currently you need to manually run `docker buildx create --use` once first):
28
+ Now you're ready to deploy a multi-arch image to the servers:
33
29
 
34
30
  ```
35
- rake mrsk:deploy
31
+ ./bin/mrsk deploy
36
32
  ```
37
33
 
38
34
  This will:
@@ -48,33 +44,123 @@ This will:
48
44
 
49
45
  Voila! All the servers are now serving the app on port 80. If you're just running a single server, you're ready to go. If you're running multiple servers, you need to put a load balancer in front of them.
50
46
 
51
- ## Operations
47
+ ## Configuration
48
+
49
+ ### Using another registry than Docker Hub
50
+
51
+ The default registry for Docker is Docker Hub. If you'd like to use a different one, just configure the server, like so:
52
+
53
+ ```yaml
54
+ registry:
55
+ server: registry.digitalocean.com
56
+ username: <%= Rails.application.credentials.registry["username"] %>
57
+ password: <%= Rails.application.credentials.registry["password"] %>
58
+ ```
59
+
60
+ ### Using a different SSH user than root
61
+
62
+ The default SSH user is root, but you can change it using `ssh_user`:
63
+
64
+ ```yaml
65
+ ssh_user: app
66
+ ```
67
+
68
+ ### Adding custom env variables
69
+
70
+ You can inject custom env variables into the app containers using `env`:
71
+
72
+ ```yaml
73
+ env:
74
+ DATABASE_URL: mysql2://db1/hey_production/
75
+ REDIS_URL: redis://redis1:6379/1
76
+ ```
77
+
78
+ ### Splitting servers into different roles
79
+
80
+ If your application uses separate hosts for running jobs or other roles beyond the default web running, you can specify these hosts and their custom entrypoint command like so:
81
+
82
+ ```yaml
83
+ servers:
84
+ web:
85
+ - 192.168.0.1
86
+ - 192.168.0.2
87
+ job:
88
+ hosts:
89
+ - 192.168.0.3
90
+ - 192.168.0.4
91
+ cmd: bin/jobs
92
+ ```
93
+
94
+ Traefik will only be installed and run on the servers in the `web` role (and on all servers if no roles are defined).
52
95
 
53
- ### Running job hosts separately
96
+ ### Adding custom container labels
54
97
 
55
- If your application uses separate job running hosts, or other roles beyond the default web running, you can specify these hosts and their custom command like so:
98
+ You can specialize the default Traefik rules by setting custom labels on the containers that are being started:
99
+
100
+ ```
101
+ labels:
102
+ traefik.http.routers.hey.rule: '''Host(`app.hey.com`)'''
103
+ ```
104
+
105
+ (Note: The extra quotes are needed to ensure the rule is passed in correctly!)
106
+
107
+ This allows you to run multiple applications on the same server sharing the same Traefik instance and port.
108
+ See https://doc.traefik.io/traefik/routing/routers/#rule for a full list of available routing rules.
109
+
110
+ The labels can even be applied on a per-role basis:
56
111
 
57
112
  ```yaml
58
113
  servers:
59
114
  web:
60
- - xxx.xxx.xxx.xxx
61
- - xxx.xxx.xxx.xxx
115
+ - 192.168.0.1
116
+ - 192.168.0.2
62
117
  job:
63
118
  hosts:
64
- - xxx.xxx.xxx.xxx
65
- - xxx.xxx.xxx.xxx
119
+ - 192.168.0.3
120
+ - 192.168.0.4
66
121
  cmd: bin/jobs
122
+ labels:
123
+ my-custom-label: "50"
67
124
  ```
68
125
 
69
- The application will be deployed to all hosts, but only those in the `web` role will be labeled to run under traefik. If you want to run custom commands on all hosts in a role, you can use `rake mrsk:app:exec:rails CMD=about ROLES=job`.
126
+ ### Configuring remote builder for native multi-arch
70
127
 
71
- ### Executing commands
128
+ If you're developing on ARM64 (like Apple Silicon), but you want to deploy on AMD64 (x86 64-bit), you have to use multi-archecture images. By default, MRSK will setup a local buildx configuration that allows for this through QEMU emulation. This can be slow, especially on the first build.
72
129
 
73
- If you need to execute commands inside the Rails containers, you can use `rake mrsk:app:exec`, `rake mrsk:app:exec:once`, `rake mrsk:app:exec:rails`, and `rake mrsk:app:exec:once:rails`. Examples:
130
+ If you want to speed up this process by using a remote AMD64 host to natively build the AMD64 part of the image, while natively building the ARM64 part locally, you can do so using builder options like follows:
131
+
132
+ ```yaml
133
+ builder:
134
+ local:
135
+ arch: arm64
136
+ host: unix:///Users/dhh/.docker/run/docker.sock
137
+ remote:
138
+ arch: amd64
139
+ host: ssh://root@192.168.0.1
140
+ ```
141
+
142
+ Note: You must have Docker running on the remote host being used as a builder.
143
+
144
+ With that configuration in place, you can setup the local/remote configuration using `./bin/mrsk build:remote:create`. If you wish to remove the contexts and buildx instances again, you can run `./bin/mrsk build:remote:remove`. If you had already built using the standard emulation setup, run `./bin/mrsk build:remove` before doing `./bin/mrsk build:remote:create`.
145
+
146
+ ### Configuring native builder when multi-arch isn't needed
147
+
148
+ If you're developing on the same architecture as the one you're deploying on, you can speed up the build a lot by forgoing a multi-arch image. This can be done by configuring the builder like so:
149
+
150
+ ```yaml
151
+ builder:
152
+ multiarch: false
153
+ ```
154
+
155
+ ## Commands
156
+
157
+ ### Remote execution
158
+
159
+ If you need to execute commands inside the Rails containers, you can use `./bin/mrsk app:exec`, `./bin/mrsk app:exec:once`, `./bin/mrsk app:exec:rails`, and `./bin/mrsk app:exec:once:rails`. Examples:
74
160
 
75
161
  ```bash
76
162
  # Runs command on all servers
77
- rake mrsk:app:exec CMD='ruby -v'
163
+ ./bin/mrsk app:exec CMD='ruby -v'
78
164
  App Host: xxx.xxx.xxx.xxx
79
165
  ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x86_64-linux]
80
166
 
@@ -82,11 +168,11 @@ App Host: xxx.xxx.xxx.xxx
82
168
  ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x86_64-linux]
83
169
 
84
170
  # Runs command on first server
85
- rake mrsk:app:exec:once CMD='cat .ruby-version'
171
+ ./bin/mrsk app:exec:once CMD='cat .ruby-version'
86
172
  3.1.3
87
173
 
88
174
  # Runs Rails command on all servers
89
- rake mrsk:app:exec:rails CMD=about
175
+ ./bin/mrsk app:exec:rails CMD=about
90
176
  App Host: xxx.xxx.xxx.xxx
91
177
  About your application's environment
92
178
  Rails version 7.1.0.alpha
@@ -112,14 +198,18 @@ Database adapter sqlite3
112
198
  Database schema version 20221231233303
113
199
 
114
200
  # Runs Rails command on first server
115
- rake mrsk:app:exec:once:rails CMD='db:version'
201
+ ./bin/mrsk app:exec:once:rails CMD='db:version'
116
202
  database: storage/production.sqlite3
117
203
  Current version: 20221231233303
118
204
  ```
119
205
 
206
+ ### Running a Rails console on the primary host
207
+
208
+ If you need to interact with the production console for the app, you can use `./bin/mrsk app:console`, which will start a Rails console session on the primary host. Be mindful that this is a live wire! Any changes made to the production database will take effect immeditately.
209
+
120
210
  ### Inspecting
121
211
 
122
- You can see the state of your servers by running `rake mrsk:info`. It'll show something like this:
212
+ You can see the state of your servers by running `./bin/mrsk info`. It'll show something like this:
123
213
 
124
214
  ```
125
215
  Traefik Host: xxx.xxx.xxx.xxx
@@ -139,11 +229,11 @@ CONTAINER ID IMAGE
139
229
  1d3c91ed1f55 registry.digitalocean.com/user/app:6ef8a6a84c525b123c5245345a8483f86d05a123 "/rails/bin/docker-e…" 13 minutes ago Up 13 minutes 3000/tcp chat-6ef8a6a84c525b123c5245345a8483f86d05a123
140
230
  ```
141
231
 
142
- You can also see just info for app containers with `rake mrsk:app:info` or just for Traefik with `rake mrsk:traefik:info`.
232
+ You can also see just info for app containers with `./bin/mrsk app:info` or just for Traefik with `./bin/mrsk traefik:info`.
143
233
 
144
234
  ### Rollback
145
235
 
146
- If you've discovered a bad deploy, you can quickly rollback by reactivating the old, paused container image. You can see what old containers are available for rollback by running `rake mrsk:app:containers`. It'll give you a presentation similar to `rake mrsk:app:info`, but include all the old containers as well. Showing something like this:
236
+ If you've discovered a bad deploy, you can quickly rollback by reactivating the old, paused container image. You can see what old containers are available for rollback by running `./bin/mrsk app:containers`. It'll give you a presentation similar to `./bin/mrsk app:info`, but include all the old containers as well. Showing something like this:
147
237
 
148
238
  ```
149
239
  App Host: 164.92.105.119
@@ -157,20 +247,19 @@ badb1aa51db4 registry.digitalocean.com/user/app:6ef8a6a84c525b123c5245345a8483
157
247
  6f170d1172ae registry.digitalocean.com/user/app:e5d9d7c2b898289dfbc5f7f1334140d984eedae4 "/rails/bin/docker-e…" 31 minutes ago Exited (1) 27 minutes ago chat-e5d9d7c2b898289dfbc5f7f1334140d984eedae4
158
248
  ```
159
249
 
160
- From the example above, we can see that `e5d9d7c2b898289dfbc5f7f1334140d984eedae4` was the last version, so it's available as a rollback target. We can perform this rollback by running `rake mrsk:rollback VERSION=e5d9d7c2b898289dfbc5f7f1334140d984eedae4`. That'll stop `6ef8a6a84c525b123c5245345a8483f86d05a123` and then start `e5d9d7c2b898289dfbc5f7f1334140d984eedae4`. Because the old container is still available, this is very quick. Nothing to download from the registry.
250
+ From the example above, we can see that `e5d9d7c2b898289dfbc5f7f1334140d984eedae4` was the last version, so it's available as a rollback target. We can perform this rollback by running `./bin/mrsk rollback VERSION=e5d9d7c2b898289dfbc5f7f1334140d984eedae4`. That'll stop `6ef8a6a84c525b123c5245345a8483f86d05a123` and then start `e5d9d7c2b898289dfbc5f7f1334140d984eedae4`. Because the old container is still available, this is very quick. Nothing to download from the registry.
161
251
 
162
- Note that by default old containers are pruned after 3 days when you run `rake mrsk:deploy`.
252
+ Note that by default old containers are pruned after 3 days when you run `./bin/mrsk deploy`.
163
253
 
164
254
  ### Removing
165
255
 
166
- If you wish to remove the entire application, including Traefik, containers, images, and registry session, you can run `rake mrsk:remove`. This will leave the servers clean.
256
+ If you wish to remove the entire application, including Traefik, containers, images, and registry session, you can run `./bin/mrsk remove`. This will leave the servers clean.
167
257
 
168
258
  ## Stage of development
169
259
 
170
260
  This is alpha software. Lots of stuff is missing. Here are some of the areas we seek to improve:
171
261
 
172
262
  - Adapterize commands to work with Podman and other container runners
173
- - Possibly switching to a bin/mrsk command rather than raw rake
174
263
  - Integrate with cloud CI pipelines
175
264
 
176
265
  ## License
@@ -0,0 +1,56 @@
1
+ require "mrsk/configuration"
2
+ require "mrsk/commands/app"
3
+ require "mrsk/commands/builder"
4
+ require "mrsk/commands/prune"
5
+ require "mrsk/commands/traefik"
6
+ require "mrsk/commands/registry"
7
+
8
+ class Mrsk::Commander
9
+ attr_reader :config_file, :config, :verbose
10
+
11
+ def initialize(config_file:, verbose: false)
12
+ @config_file, @verbose = config_file, verbose
13
+ end
14
+
15
+ def config
16
+ @config ||= Mrsk::Configuration.load_file(config_file).tap { |config| setup_with(config) }
17
+ end
18
+
19
+
20
+ def app
21
+ @app ||= Mrsk::Commands::App.new(config)
22
+ end
23
+
24
+ def builder
25
+ @builder ||= Mrsk::Commands::Builder.new(config)
26
+ end
27
+
28
+ def traefik
29
+ @traefik ||= Mrsk::Commands::Traefik.new(config)
30
+ end
31
+
32
+ def registry
33
+ @registry ||= Mrsk::Commands::Registry.new(config)
34
+ end
35
+
36
+ def prune
37
+ @prune ||= Mrsk::Commands::Prune.new(config)
38
+ end
39
+
40
+
41
+ def verbosity(level)
42
+ old_level = SSHKit.config.output_verbosity
43
+ SSHKit.config.output_verbosity = level
44
+ yield
45
+ ensure
46
+ SSHKit.config.output_verbosity = old_level
47
+ end
48
+
49
+ private
50
+ # Lazy setup of SSHKit
51
+ def setup_with(config)
52
+ SSHKit::Backend::Netssh.configure { |ssh| ssh.ssh_options = config.ssh_options }
53
+ SSHKit.config.command_map[:docker] = "docker" # No need to use /usr/bin/env, just clogs up the logs
54
+ SSHKit.config.output_verbosity = :debug if verbose
55
+ end
56
+ end
@@ -1,14 +1,6 @@
1
- class Mrsk::Commands::App < Mrsk::Commands::Base
2
- def push
3
- # TODO: Run 'docker buildx create --use' when needed
4
- # TODO: Make multiarch an option so Linux users can enjoy speedier builds
5
- docker :buildx, :build, "--push", "--platform linux/amd64,linux/arm64", "-t", config.absolute_image, "."
6
- end
7
-
8
- def pull
9
- docker :pull, config.absolute_image
10
- end
1
+ require "mrsk/commands/base"
11
2
 
3
+ class Mrsk::Commands::App < Mrsk::Commands::Base
12
4
  def run(role: :web)
13
5
  role = config.role(role)
14
6
 
@@ -36,17 +28,22 @@ class Mrsk::Commands::App < Mrsk::Commands::Base
36
28
  end
37
29
 
38
30
  def logs
39
- [ "docker ps -q #{service_filter.join(" ")} | xargs docker logs -f" ]
31
+ [ "docker ps -q #{service_filter.join(" ")} | xargs docker logs -n 100 -t" ]
40
32
  end
41
33
 
42
- def exec(*command)
43
- docker :exec,
34
+ def exec(*command, interactive: false)
35
+ docker :exec,
36
+ ("-it" if interactive),
44
37
  "-e", redact("RAILS_MASTER_KEY=#{config.master_key}"),
45
38
  *config.env_args,
46
39
  config.service_with_version,
47
40
  *command
48
41
  end
49
42
 
43
+ def console
44
+ "ssh -t #{config.ssh_user}@#{config.primary_host} '#{exec("bin/rails", "c", interactive: true).join(" ")}'"
45
+ end
46
+
50
47
  def list_containers
51
48
  docker :container, :ls, "-a", *service_filter
52
49
  end
@@ -0,0 +1,27 @@
1
+ require "sshkit"
2
+
3
+ module Mrsk::Commands
4
+ class Base
5
+ attr_accessor :config
6
+
7
+ def initialize(config)
8
+ @config = config
9
+ end
10
+
11
+ private
12
+ def combine(*commands)
13
+ commands
14
+ .collect { |command| command + [ "&&" ] }.flatten # Join commands with &&
15
+ .tap { |commands| commands.pop } # Remove trailing &&
16
+ end
17
+
18
+ def docker(*args)
19
+ args.compact.unshift :docker
20
+ end
21
+
22
+ # Copied from SSHKit::Backend::Abstract#redact to be available inside Commands classes
23
+ def redact(arg) # Used in execute_command to hide redact() args a user passes in
24
+ arg.to_s.extend(SSHKit::Redaction) # to_s due to our inability to extend Integer, etc
25
+ end
26
+ end
27
+ end
@@ -0,0 +1,53 @@
1
+ require "mrsk/commands/builder/multiarch"
2
+
3
+ class Mrsk::Commands::Builder::Multiarch::Remote < Mrsk::Commands::Builder::Multiarch
4
+ def create
5
+ combine \
6
+ create_contexts,
7
+ create_local_buildx,
8
+ append_remote_buildx
9
+ end
10
+
11
+ def remove
12
+ combine \
13
+ remove_contexts,
14
+ super
15
+ end
16
+
17
+ private
18
+ def create_local_buildx
19
+ docker :buildx, :create, "--use", "--name", "mrsk", "mrsk-#{local["arch"]}", "--platform", "linux/#{local["arch"]}"
20
+ end
21
+
22
+ def append_remote_buildx
23
+ docker :buildx, :create, "--append", "--name", "mrsk", "mrsk-#{remote["arch"]}", "--platform", "linux/#{remote["arch"]}"
24
+ end
25
+
26
+ def create_contexts
27
+ combine \
28
+ create_context(local["arch"], local["host"]),
29
+ create_context(remote["arch"], remote["host"])
30
+ end
31
+
32
+ def create_context(arch, host)
33
+ docker :context, :create, "mrsk-#{arch}", "--description", "'MRSK #{arch} Native Host'", "--docker", "'host=#{host}'"
34
+ end
35
+
36
+ def remove_contexts
37
+ combine \
38
+ remove_context(local["arch"]),
39
+ remove_context(remote["arch"])
40
+ end
41
+
42
+ def remove_context(arch)
43
+ docker :context, :rm, "mrsk-#{arch}"
44
+ end
45
+
46
+ def local
47
+ config.builder["local"]
48
+ end
49
+
50
+ def remote
51
+ config.builder["remote"]
52
+ end
53
+ end
@@ -0,0 +1,25 @@
1
+ require "mrsk/commands/base"
2
+
3
+ class Mrsk::Commands::Builder::Multiarch < Mrsk::Commands::Base
4
+ def create
5
+ docker :buildx, :create, "--use", "--name", "mrsk"
6
+ end
7
+
8
+ def remove
9
+ docker :buildx, :rm, "mrsk"
10
+ end
11
+
12
+ def push
13
+ docker :buildx, :build, "--push", "--platform linux/amd64,linux/arm64", "-t", config.absolute_image, "."
14
+ end
15
+
16
+ def pull
17
+ docker :pull, config.absolute_image
18
+ end
19
+
20
+ def info
21
+ combine \
22
+ docker(:context, :ls),
23
+ docker(:buildx, :ls)
24
+ end
25
+ end
@@ -0,0 +1,25 @@
1
+ require "mrsk/commands/base"
2
+
3
+ class Mrsk::Commands::Builder::Native < Mrsk::Commands::Base
4
+ def create
5
+ # No-op on native
6
+ end
7
+
8
+ def remove
9
+ # No-op on native
10
+ end
11
+
12
+ def push
13
+ combine \
14
+ docker(:build, "-t", config.absolute_image, "."),
15
+ docker(:push, config.absolute_image)
16
+ end
17
+
18
+ def pull
19
+ docker :pull, config.absolute_image
20
+ end
21
+
22
+ def info
23
+ # No-op on native
24
+ end
25
+ end
@@ -0,0 +1,39 @@
1
+ require "mrsk/commands/base"
2
+
3
+ class Mrsk::Commands::Builder < Mrsk::Commands::Base
4
+ delegate :create, :remove, :push, :pull, :info, to: :target
5
+ delegate :native?, :multiarch?, :remote?, to: :name
6
+
7
+ def name
8
+ target.class.to_s.demodulize.downcase.inquiry
9
+ end
10
+
11
+ def target
12
+ case
13
+ when config.builder.nil?
14
+ multiarch
15
+ when config.builder["multiarch"] == false
16
+ native
17
+ when config.builder["local"] && config.builder["local"]
18
+ multiarch_remote
19
+ else
20
+ raise ArgumentError, "Builder configuration incorrect: #{config.builder.inspect}"
21
+ end
22
+ end
23
+
24
+ def native
25
+ @native ||= Mrsk::Commands::Builder::Native.new(config)
26
+ end
27
+
28
+ def multiarch
29
+ @multiarch ||= Mrsk::Commands::Builder::Multiarch.new(config)
30
+ end
31
+
32
+ def multiarch_remote
33
+ @multiarch_remote ||= Mrsk::Commands::Builder::Multiarch::Remote.new(config)
34
+ end
35
+ end
36
+
37
+ require "mrsk/commands/builder/native"
38
+ require "mrsk/commands/builder/multiarch"
39
+ require "mrsk/commands/builder/multiarch/remote"
@@ -0,0 +1,17 @@
1
+ require "mrsk/commands/base"
2
+ require "active_support/duration"
3
+ require "active_support/core_ext/numeric/time"
4
+
5
+ class Mrsk::Commands::Prune < Mrsk::Commands::Base
6
+ PRUNE_IMAGES_AFTER = 30.days.in_hours.to_i
7
+ PRUNE_CONTAINERS_AFTER = 3.days.in_hours.to_i
8
+
9
+ def images
10
+ docker :image, :prune, "-f", "--filter", "until=#{PRUNE_IMAGES_AFTER}h"
11
+ end
12
+
13
+ def containers
14
+ docker :image, :prune, "-f", "--filter", "until=#{PRUNE_IMAGES_AFTER}h"
15
+ docker :container, :prune, "-f", "--filter", "label=service=#{config.service}", "--filter", "'until=#{PRUNE_CONTAINERS_AFTER}h'"
16
+ end
17
+ end
@@ -1,3 +1,5 @@
1
+ require "mrsk/commands/base"
2
+
1
3
  class Mrsk::Commands::Registry < Mrsk::Commands::Base
2
4
  delegate :registry, to: :config
3
5
 
@@ -1,3 +1,5 @@
1
+ require "mrsk/commands/base"
2
+
1
3
  class Mrsk::Commands::Traefik < Mrsk::Commands::Base
2
4
  def run
3
5
  docker :run, "--name traefik",
@@ -22,7 +24,7 @@ class Mrsk::Commands::Traefik < Mrsk::Commands::Base
22
24
  end
23
25
 
24
26
  def logs
25
- docker :logs, "traefik"
27
+ docker :logs, "traefik", "-n", "100", "-t"
26
28
  end
27
29
 
28
30
  def remove_container
data/lib/mrsk/commands.rb CHANGED
@@ -1,25 +1,2 @@
1
- require "sshkit"
2
-
3
1
  module Mrsk::Commands
4
- class Base
5
- attr_accessor :config
6
-
7
- def initialize(config)
8
- @config = config
9
- end
10
-
11
- private
12
- def docker(*args)
13
- args.compact.unshift :docker
14
- end
15
-
16
- # Copied from SSHKit::Backend::Abstract#redact to be available inside Commands classes
17
- def redact(arg) # Used in execute_command to hide redact() args a user passes in
18
- arg.to_s.extend(SSHKit::Redaction) # to_s due to our inability to extend Integer, etc
19
- end
20
- end
21
2
  end
22
-
23
- require "mrsk/commands/app"
24
- require "mrsk/commands/traefik"
25
- require "mrsk/commands/registry"
@@ -11,6 +11,14 @@ class Mrsk::Configuration::Role
11
11
  @hosts ||= extract_hosts_from_config
12
12
  end
13
13
 
14
+ def labels
15
+ if name.web?
16
+ default_labels.merge(traefik_labels).merge(custom_labels)
17
+ else
18
+ default_labels.merge(custom_labels)
19
+ end
20
+ end
21
+
14
22
  def label_args
15
23
  argumentize "--label", labels
16
24
  end
@@ -31,14 +39,6 @@ class Mrsk::Configuration::Role
31
39
  end
32
40
  end
33
41
 
34
- def labels
35
- if name.web?
36
- default_labels.merge(traefik_labels)
37
- else
38
- default_labels
39
- end
40
- end
41
-
42
42
  def default_labels
43
43
  { "service" => config.service, "role" => name }
44
44
  end
@@ -53,6 +53,13 @@ class Mrsk::Configuration::Role
53
53
  }
54
54
  end
55
55
 
56
+ def custom_labels
57
+ Hash.new.tap do |labels|
58
+ labels.merge!(config.labels) if config.labels.present?
59
+ labels.merge!(specializations["labels"]) if specializations["labels"].present?
60
+ end
61
+ end
62
+
56
63
  def specializations
57
64
  if config.servers.is_a?(Array) || config.servers[name].is_a?(Array)
58
65
  { }
@@ -3,7 +3,7 @@ require "active_support/core_ext/string/inquiry"
3
3
  require "erb"
4
4
 
5
5
  class Mrsk::Configuration
6
- delegate :service, :image, :servers, :env, :registry, :ssh_user, to: :config, allow_nil: true
6
+ delegate :service, :image, :servers, :env, :labels, :registry, :builder, to: :config, allow_nil: true
7
7
 
8
8
  class << self
9
9
  def load_file(file)
@@ -75,11 +75,19 @@ class Mrsk::Configuration
75
75
 
76
76
 
77
77
  def env_args
78
- self.class.argumentize "-e", config.env if config.env.present?
78
+ if config.env.present?
79
+ self.class.argumentize "-e", config.env
80
+ else
81
+ []
82
+ end
83
+ end
84
+
85
+ def ssh_user
86
+ config.ssh_user || "root"
79
87
  end
80
88
 
81
89
  def ssh_options
82
- { user: config.ssh_user || "root", auth_methods: [ "publickey" ] }
90
+ { user: ssh_user, auth_methods: [ "publickey" ] }
83
91
  end
84
92
 
85
93
  def master_key
data/lib/mrsk/version.rb CHANGED
@@ -1,3 +1,3 @@
1
1
  module Mrsk
2
- VERSION = "0.0.2"
2
+ VERSION = "0.0.3"
3
3
  end
data/lib/mrsk.rb CHANGED
@@ -3,6 +3,4 @@ end
3
3
 
4
4
  require "mrsk/version"
5
5
  require "mrsk/engine"
6
-
7
- require "mrsk/configuration"
8
- require "mrsk/commands"
6
+ require "mrsk/commander"
@@ -1,32 +1,17 @@
1
1
  require_relative "setup"
2
2
 
3
- app = Mrsk::Commands::App.new(MRSK_CONFIG)
4
-
5
3
  namespace :mrsk do
6
4
  namespace :app do
7
- desc "Deliver a newly built app image to servers"
8
- task deliver: %i[ push pull ]
9
-
10
- desc "Build locally and push app image to registry"
11
- task :push do
12
- run_locally { execute *app.push } unless ENV["VERSION"]
13
- end
14
-
15
- desc "Pull app image from the registry onto servers"
16
- task :pull do
17
- on(MRSK_CONFIG.hosts) { execute *app.pull }
18
- end
19
-
20
5
  desc "Run app on servers (or start them if they've already been run)"
21
6
  task :run do
22
- MRSK_CONFIG.roles.each do |role|
7
+ MRSK.config.roles.each do |role|
23
8
  on(role.hosts) do |host|
24
9
  begin
25
- execute *app.run(role: role.name)
10
+ execute *MRSK.app.run(role: role.name)
26
11
  rescue SSHKit::Command::Failed => e
27
12
  if e.message =~ /already in use/
28
- puts "Container with same version already deployed on #{host}, starting that instead"
29
- execute *app.start, host: host
13
+ error "Container with same version already deployed on #{host}, starting that instead"
14
+ execute *MRSK.app.start, host: host
30
15
  else
31
16
  raise
32
17
  end
@@ -35,14 +20,14 @@ namespace :mrsk do
35
20
  end
36
21
  end
37
22
 
38
- desc "Start existing app on servers"
23
+ desc "Start existing app on servers (use VERSION=<git-hash> to designate which version)"
39
24
  task :start do
40
- on(MRSK_CONFIG.hosts) { execute *app.start, raise_on_non_zero_exit: false }
25
+ on(MRSK.config.hosts) { execute *MRSK.app.start, raise_on_non_zero_exit: false }
41
26
  end
42
27
 
43
28
  desc "Stop app on servers"
44
29
  task :stop do
45
- on(MRSK_CONFIG.hosts) { execute *app.stop, raise_on_non_zero_exit: false }
30
+ on(MRSK.config.hosts) { execute *MRSK.app.stop, raise_on_non_zero_exit: false }
46
31
  end
47
32
 
48
33
  desc "Start app on servers (use VERSION=<git-hash> to designate which version)"
@@ -50,48 +35,62 @@ namespace :mrsk do
50
35
 
51
36
  desc "Display information about app containers"
52
37
  task :info do
53
- on(MRSK_CONFIG.hosts) { |host| puts "App Host: #{host}\n" + capture(*app.info) + "\n\n" }
38
+ on(MRSK.config.hosts) { |host| puts "App Host: #{host}\n" + capture(*MRSK.app.info) + "\n\n" }
54
39
  end
55
40
 
56
41
  desc "Execute a custom task on servers passed in as CMD='bin/rake some:task'"
57
42
  task :exec do
58
- on(MRSK_CONFIG.hosts) { |host| puts "App Host: #{host}\n" + capture(*app.exec(ENV["CMD"])) + "\n\n" }
43
+ on(MRSK.config.hosts) { |host| puts "App Host: #{host}\n" + capture(*MRSK.app.exec(ENV["CMD"])) + "\n\n" }
44
+ end
45
+
46
+ desc "Start Rails Console on primary host"
47
+ task :console do
48
+ puts "Launching Rails console on #{MRSK.config.primary_host}..."
49
+ exec app.console
59
50
  end
60
51
 
61
52
  namespace :exec do
62
53
  desc "Execute Rails command on servers, like CMD='runner \"puts %(Hello World)\""
63
54
  task :rails do
64
- on(MRSK_CONFIG.hosts) { |host| puts "App Host: #{host}\n" + capture(*app.exec("bin/rails", ENV["CMD"])) + "\n\n" }
55
+ on(MRSK.config.hosts) { |host| puts "App Host: #{host}\n" + capture(*MRSK.app.exec("bin/rails", ENV["CMD"])) + "\n\n" }
65
56
  end
66
57
 
67
58
  desc "Execute a custom task on the first defined server"
68
59
  task :once do
69
- on(MRSK_CONFIG.primary_host) { |host| puts capture(*app.exec(ENV["CMD"])) }
60
+ on(MRSK.config.primary_host) { puts capture(*MRSK.app.exec(ENV["CMD"])) }
70
61
  end
71
62
 
72
63
  namespace :once do
73
64
  desc "Execute Rails command on the first defined server, like CMD='runner \"puts %(Hello World)\""
74
65
  task :rails do
75
- on(MRSK_CONFIG.primary_host) { puts capture(*app.exec("bin/rails", ENV["CMD"])) }
66
+ on(MRSK.config.primary_host) { puts capture(*MRSK.app.exec("bin/rails", ENV["CMD"])) }
76
67
  end
77
68
  end
78
69
  end
79
70
 
80
71
  desc "List all the app containers currently on servers"
81
72
  task :containers do
82
- on(MRSK_CONFIG.hosts) { |host| puts "App Host: #{host}\n" + capture(*app.list_containers) + "\n\n" }
73
+ on(MRSK.config.hosts) { |host| puts "App Host: #{host}\n" + capture(*MRSK.app.list_containers) + "\n\n" }
83
74
  end
84
75
 
85
- desc "Tail logs from app containers"
76
+ desc "Show last 100 log lines from app on servers"
86
77
  task :logs do
87
- on(MRSK_CONFIG.hosts) { execute *app.logs }
78
+ # FIXME: Catch when app containers aren't running
79
+ on(MRSK.config.hosts) { |host| puts "App Host: #{host}\n" + capture(*MRSK.app.logs) + "\n\n" }
88
80
  end
89
81
 
90
82
  desc "Remove app containers and images from servers"
91
- task remove: %i[ stop ] do
92
- on(MRSK_CONFIG.hosts) do
93
- execute *app.remove_containers
94
- execute *app.remove_images
83
+ task remove: %w[ remove:containers remove:images ]
84
+
85
+ namespace :remove do
86
+ desc "Remove app containers from servers"
87
+ task :containers do
88
+ on(MRSK.config.hosts) { execute *MRSK.app.remove_containers }
89
+ end
90
+
91
+ desc "Remove app images from servers"
92
+ task :images do
93
+ on(MRSK.config.hosts) { execute *MRSK.app.remove_images }
95
94
  end
96
95
  end
97
96
  end
@@ -0,0 +1,52 @@
1
+ require_relative "setup"
2
+
3
+ namespace :mrsk do
4
+ namespace :build do
5
+ desc "Deliver a newly built app image to servers"
6
+ task deliver: %i[ push pull ]
7
+
8
+ desc "Build locally and push app image to registry"
9
+ task :push do
10
+ run_locally do
11
+ begin
12
+ debug "Using builder: #{MRSK.builder.name}"
13
+ info "Building image may take a while (run with VERBOSE=1 for progress logging)"
14
+ execute *MRSK.builder.push
15
+ rescue SSHKit::Command::Failed => e
16
+ error "Missing compatible builder, so creating a new one first"
17
+ execute *MRSK.builder.create
18
+ execute *MRSK.builder.push
19
+ end
20
+ end unless ENV["VERSION"]
21
+ end
22
+
23
+ desc "Pull app image from the registry onto servers"
24
+ task :pull do
25
+ on(MRSK.config.hosts) { execute *MRSK.builder.pull }
26
+ end
27
+
28
+ desc "Create a local build setup"
29
+ task :create do
30
+ run_locally do
31
+ debug "Using builder: #{MRSK.builder.name}"
32
+ execute *MRSK.builder.create
33
+ end
34
+ end
35
+
36
+ desc "Remove local build setup"
37
+ task :remove do
38
+ run_locally do
39
+ debug "Using builder: #{MRSK.builder.name}"
40
+ execute *MRSK.builder.remove
41
+ end
42
+ end
43
+
44
+ desc "Show the name of the configured builder"
45
+ task :info do
46
+ run_locally do
47
+ puts "Builder: #{MRSK.builder.name} (#{MRSK.builder.target.class.name})"
48
+ puts capture(*MRSK.builder.info)
49
+ end
50
+ end
51
+ end
52
+ end
@@ -2,10 +2,10 @@ require_relative "setup"
2
2
 
3
3
  namespace :mrsk do
4
4
  desc "Deploy app for the first time to a fresh server"
5
- task fresh: %w[ server:bootstrap registry:login app:deliver traefik:run app:stop app:run ]
5
+ task fresh: %w[ server:bootstrap registry:login build:deliver traefik:run app:stop app:run ]
6
6
 
7
7
  desc "Push the latest version of the app, ensure Traefik is running, then restart app"
8
- task deploy: %w[ registry:login app:deliver traefik:run app:stop app:run prune ]
8
+ task deploy: %w[ registry:login build:deliver traefik:run app:stop app:run prune ]
9
9
 
10
10
  desc "Rollback to VERSION=x that was already run as a container on servers"
11
11
  task rollback: %w[ app:restart ]
@@ -16,9 +16,20 @@ namespace :mrsk do
16
16
  desc "Create config stub in config/deploy.yml"
17
17
  task :init do
18
18
  require "fileutils"
19
- FileUtils.cp_r \
20
- Pathname.new(File.expand_path("templates/deploy.yml", __dir__)),
21
- Rails.root.join("config/deploy.yml")
19
+
20
+ if (deploy_file = Rails.root.join("config/deploy.yml")).exist?
21
+ puts "Config file already exists in config/deploy.yml (remove first to create a new one)"
22
+ else
23
+ FileUtils.cp_r Pathname.new(File.expand_path("templates/deploy.yml", __dir__)), deploy_file
24
+ puts "Created configuration file in config/deploy.yml"
25
+ end
26
+
27
+ if (binstub = Rails.root.join("bin/mrsk")).exist?
28
+ puts "Binstub already exists in bin/mrsk (remove first to create a new one)"
29
+ else
30
+ FileUtils.cp_r Pathname.new(File.expand_path("templates/mrsk", __dir__)), binstub
31
+ puts "Created binstub file in bin/mrsk"
32
+ end
22
33
  end
23
34
 
24
35
  desc "Remove Traefik, app, and registry session from servers"
@@ -7,12 +7,12 @@ namespace :mrsk do
7
7
  namespace :prune do
8
8
  desc "Prune unused images older than 30 days"
9
9
  task :images do
10
- on(MRSK_CONFIG.hosts) { execute "docker image prune -f --filter 'until=720h'" }
10
+ on(MRSK.config.hosts) { MRSK.verbosity(:debug) { execute *MRSK.prune.images } }
11
11
  end
12
12
 
13
13
  desc "Prune stopped containers for the service older than 3 days"
14
14
  task :containers do
15
- on(MRSK_CONFIG.hosts) { execute "docker container prune -f --filter label=service=#{MRSK_CONFIG.service} --filter 'until=72h'" }
15
+ on(MRSK.config.hosts) { MRSK.verbosity(:debug) { execute *MRSK.prune.containers } }
16
16
  end
17
17
  end
18
18
  end
@@ -1,18 +1,16 @@
1
1
  require_relative "setup"
2
2
 
3
- registry = Mrsk::Commands::Registry.new(MRSK_CONFIG)
4
-
5
3
  namespace :mrsk do
6
4
  namespace :registry do
7
5
  desc "Login to the registry locally and remotely"
8
6
  task :login do
9
- run_locally { execute *registry.login }
10
- on(MRSK_CONFIG.hosts) { execute *registry.login }
7
+ run_locally { execute *MRSK.registry.login }
8
+ on(MRSK.config.hosts) { execute *MRSK.registry.login }
11
9
  end
12
10
 
13
11
  desc "Logout of the registry remotely"
14
12
  task :logout do
15
- on(MRSK_CONFIG.hosts) { execute *registry.logout }
13
+ on(MRSK.config.hosts) { execute *MRSK.registry.logout }
16
14
  end
17
15
  end
18
16
  end
@@ -5,7 +5,7 @@ namespace :mrsk do
5
5
  desc "Setup Docker on the remote servers"
6
6
  task :bootstrap do
7
7
  # FIXME: Detect when apt-get is not available and use the appropriate alternative
8
- on(MRSK_CONFIG.hosts) { execute "apt-get install docker.io -y" }
8
+ on(MRSK.config.hosts) { execute "apt-get install docker.io -y" }
9
9
  end
10
10
  end
11
11
  end
@@ -3,14 +3,4 @@ require "sshkit/dsl"
3
3
 
4
4
  include SSHKit::DSL
5
5
 
6
- if (config_file = Rails.root.join("config/deploy.yml")).exist?
7
- MRSK_CONFIG = Mrsk::Configuration.load_file(config_file)
8
-
9
- SSHKit::Backend::Netssh.configure { |ssh| ssh.ssh_options = MRSK_CONFIG.ssh_options }
10
-
11
- # No need to use /usr/bin/env, just clogs up the logs
12
- SSHKit.config.command_map[:docker] = "docker"
13
- else
14
- # MRSK is missing config/deploy.yml – run 'rake mrsk:init'
15
- MRSK_CONFIG = Mrsk::Configuration.new({}, validate: false)
16
- end
6
+ MRSK = Mrsk::Commander.new config_file: Rails.root.join("config/deploy.yml"), verbose: ENV["VERBOSE"]
@@ -3,19 +3,22 @@
3
3
  service: my-app
4
4
 
5
5
  # Name of the container image
6
- image: user/chat
6
+ image: user/my-app
7
7
 
8
- # All the servers targeted for deploy. You can reference a single server for a command by using SERVERS=xxx.xxx.xxx.xxx
8
+ # All the servers targeted for deploy. You can reference a single server for a command by using SERVERS=192.168.0.1
9
9
  servers:
10
- - xxx.xxx.xxx.xxx
10
+ - 192.168.0.1
11
11
 
12
12
  # The following envs are made available to the container when started
13
13
  env:
14
14
  # Remember never to put passwords or tokens directly into this file, use encrypted credentials
15
15
  # REDIS_URL: redis://x/y
16
16
 
17
+ # Where your images will be hosted
17
18
  registry:
18
19
  # Specify the registry server, if you're not using Docker Hub
19
20
  # server: registry.digitalocean.com / ghcr.io / ...
20
- username: <%= Rails.application.credentials.registry["username"] %>
21
- password: <%= Rails.application.credentials.registry["password"] %>
21
+
22
+ # Set credentials with bin/rails credentials:edit
23
+ username: my-user
24
+ password: my-password-should-go-in-credentials
@@ -0,0 +1,8 @@
1
+ #!/bin/bash
2
+
3
+ if [ "${*}" == "" ]; then
4
+ # Improve so list matches
5
+ exec bin/rake -T mrsk
6
+ else
7
+ exec bin/rake "mrsk:$@"
8
+ fi
@@ -1,22 +1,20 @@
1
1
  require_relative "setup"
2
2
 
3
- traefik = Mrsk::Commands::Traefik.new(MRSK_CONFIG)
4
-
5
3
  namespace :mrsk do
6
4
  namespace :traefik do
7
5
  desc "Run Traefik on servers"
8
6
  task :run do
9
- on(MRSK_CONFIG.role(:web).hosts) { execute *traefik.run, raise_on_non_zero_exit: false }
7
+ on(MRSK.config.role(:web).hosts) { execute *MRSK.traefik.run, raise_on_non_zero_exit: false }
10
8
  end
11
9
 
12
10
  desc "Start existing Traefik on servers"
13
11
  task :start do
14
- on(MRSK_CONFIG.role(:web).hosts) { execute *traefik.start, raise_on_non_zero_exit: false }
12
+ on(MRSK.config.role(:web).hosts) { execute *MRSK.traefik.start, raise_on_non_zero_exit: false }
15
13
  end
16
14
 
17
15
  desc "Stop Traefik on servers"
18
16
  task :stop do
19
- on(MRSK_CONFIG.role(:web).hosts) { execute *traefik.stop, raise_on_non_zero_exit: false }
17
+ on(MRSK.config.role(:web).hosts) { execute *MRSK.traefik.stop, raise_on_non_zero_exit: false }
20
18
  end
21
19
 
22
20
  desc "Restart Traefik on servers"
@@ -24,14 +22,19 @@ namespace :mrsk do
24
22
 
25
23
  desc "Display information about Traefik containers from servers"
26
24
  task :info do
27
- on(MRSK_CONFIG.role(:web).hosts) { |host| puts "Traefik Host: #{host}\n" + capture(*traefik.info) + "\n\n" }
25
+ on(MRSK.config.role(:web).hosts) { |host| puts "Traefik Host: #{host}\n" + capture(*MRSK.traefik.info) + "\n\n" }
26
+ end
27
+
28
+ desc "Show last 100 log lines from Traefik on servers"
29
+ task :logs do
30
+ on(MRSK.config.hosts) { |host| puts "Traefik Host: #{host}\n" + capture(*MRSK.traefik.logs) + "\n\n" }
28
31
  end
29
32
 
30
33
  desc "Remove Traefik container and image from servers"
31
34
  task remove: %i[ stop ] do
32
- on(MRSK_CONFIG.role(:web).hosts) do
33
- execute *traefik.remove_container
34
- execute *traefik.remove_image
35
+ on(MRSK.config.role(:web).hosts) do
36
+ execute *MRSK.traefik.remove_container
37
+ execute *MRSK.traefik.remove_image
35
38
  end
36
39
  end
37
40
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: mrsk
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.2
4
+ version: 0.0.3
5
5
  platform: ruby
6
6
  authors:
7
7
  - David Heinemeier Hansson
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2023-01-10 00:00:00.000000000 Z
11
+ date: 2023-01-13 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: railties
@@ -47,8 +47,15 @@ files:
47
47
  - MIT-LICENSE
48
48
  - README.md
49
49
  - lib/mrsk.rb
50
+ - lib/mrsk/commander.rb
50
51
  - lib/mrsk/commands.rb
51
52
  - lib/mrsk/commands/app.rb
53
+ - lib/mrsk/commands/base.rb
54
+ - lib/mrsk/commands/builder.rb
55
+ - lib/mrsk/commands/builder/multiarch.rb
56
+ - lib/mrsk/commands/builder/multiarch/remote.rb
57
+ - lib/mrsk/commands/builder/native.rb
58
+ - lib/mrsk/commands/prune.rb
52
59
  - lib/mrsk/commands/registry.rb
53
60
  - lib/mrsk/commands/traefik.rb
54
61
  - lib/mrsk/configuration.rb
@@ -56,12 +63,14 @@ files:
56
63
  - lib/mrsk/engine.rb
57
64
  - lib/mrsk/version.rb
58
65
  - lib/tasks/mrsk/app.rake
66
+ - lib/tasks/mrsk/build.rake
59
67
  - lib/tasks/mrsk/mrsk.rake
60
68
  - lib/tasks/mrsk/prune.rake
61
69
  - lib/tasks/mrsk/registry.rake
62
70
  - lib/tasks/mrsk/server.rake
63
71
  - lib/tasks/mrsk/setup.rb
64
72
  - lib/tasks/mrsk/templates/deploy.yml
73
+ - lib/tasks/mrsk/templates/mrsk
65
74
  - lib/tasks/mrsk/traefik.rake
66
75
  homepage: https://github.com/rails/mrsk
67
76
  licenses: