mrsk 0.13.0 → 0.13.2

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 6b258a4c44d5b010d6ab4004d0e051f9378dd4511203020a8fed2dc639527d9f
4
- data.tar.gz: 702b1c42fc0b095d18c37baa1811186ece2267b3d49067648129b76bef284be1
3
+ metadata.gz: 3a752ca316aea9ce851cc9b4ac96cd7fde613f94dd1b63e238c3320b10f82e8c
4
+ data.tar.gz: 689bba95ccc72b27b9cc85572b6e4d995f6bfed3d09be30c2076a81ed1eb07f6
5
5
  SHA512:
6
- metadata.gz: 60163759e4097ea91df33411e7dc32261c0bc82f609bc1f54308f4111e3f0efe4d9731b234175e293f5c8a6aef9e1a88156be88260fc0a545b4d276ddb4e753b
7
- data.tar.gz: 882d09991704e45f3c377dfafa0b15f2c6f1421085e5ca8096855bb03b109c6a91e4e47b7ec9270cdb31f7e43b3a18b23576fb0b2faa6367c82d9b3748550201
6
+ metadata.gz: cdbb9e9b58364ba933778bb18bf4576ee3c611fcbb3be9b341c2e21d53d08d37bd524c694e058be4456f0d7da837b488d20c009973017fa39f6c57fb5835d0ce
7
+ data.tar.gz: eb58c8ac236172f91719582f7b9606d2255c122be41c7f341a6a54e6b1680b4715f55ad99492ee47a57c38b74626b129d92bd77ed0d864042a7c457343aade13
data/README.md CHANGED
@@ -67,7 +67,7 @@ Voila! All the servers are now serving the app on port 80. If you're just runnin
67
67
 
68
68
  In the past decade+, there's been an explosion in commercial offerings that make deploying web apps easier. Heroku kicked it off with an incredible offering that stayed ahead of the competition seemingly forever. These days we have excellent alternatives like Fly.io and Render. And hosted Kubernetes is making things easier too on AWS, GCP, Digital Ocean, and elsewhere. But these are all offerings that have you renting computers in the cloud at a premium. If you want to run on your own hardware, or even just have a clear migration path to do so in the future, you need to carefully consider how locked in you get to these commercial platforms. Preferably before the bills swallow your business whole!
69
69
 
70
- MRSK seeks to bring the advance in ergonomics pioneered by these commercial offerings to deploying web apps anywhere. Whether that's low-cost cloud options without the managed-service markup from the likes of Digital Ocean, Hetzner, OVH, etc, or it's your own colocated bare metal. To MRSK, it's all the same. Feed the config file a list of IP addresses with vanilla Ubuntu servers that have seen no prep beyond an added SSH key, and you'll be running in literally minutes.
70
+ MRSK seeks to bring the advance in ergonomics pioneered by these commercial offerings to deploying web apps anywhere. Whether that's low-cost cloud options without the managed-service markup from the likes of Digital Ocean, Hetzner, OVH, etc., or it's your own colocated bare metal. To MRSK, it's all the same. Feed the config file a list of IP addresses with vanilla Ubuntu servers that have seen no prep beyond an added SSH key, and you'll be running in literally minutes.
71
71
 
72
72
  This approach gives you enormous portability. You can have your web app deployed on several clouds at ease like this. Or you can buy the baseline with your own hardware, then deploy to a cloud before a big seasonal spike to get more capacity. When you're not locked into a single provider from a tooling perspective, there are a lot of compelling options available.
73
73
 
@@ -670,7 +670,7 @@ This assumes the Cron settings are stored in `config/crontab`.
670
670
 
671
671
  ### Healthcheck
672
672
 
673
- MRSK uses Docker healtchecks to check the health of your application during deployment. Traefik uses this same healthcheck status to determine when a container is ready to receive traffic.
673
+ MRSK uses Docker healthchecks to check the health of your application during deployment. Traefik uses this same healthcheck status to determine when a container is ready to receive traffic.
674
674
 
675
675
  The healthcheck defaults to testing the HTTP response to the path `/up` on port 3000, up to 7 times. You can tailor this behaviour with the `healthcheck` setting:
676
676
 
@@ -840,7 +840,7 @@ Message: Automatic deploy lock
840
840
  You can also manually acquire and release the lock
841
841
 
842
842
  ```
843
- mrsk lock acquire -m "Doing maintanence"
843
+ mrsk lock acquire -m "Doing maintenance"
844
844
  ```
845
845
 
846
846
  ```
@@ -882,11 +882,13 @@ firing a JSON webhook. These variables include:
882
882
  - `MRSK_PERFORMER` - the local user performing the command (from `whoami`)
883
883
  - `MRSK_SERVICE_VERSION` - an abbreviated service and version for use in messages, e.g. app@150b24f
884
884
  - `MRSK_VERSION` - an full version being deployed
885
- - `MRSK_DESTINATION` - optional: destination, e.g. "staging"
886
885
  - `MRSK_HOSTS` - a comma separated list of the hosts targeted by the command
886
+ - `MRSK_COMMAND` - The command we are running
887
+ - `MRSK_SUBCOMMAND` - optional: The subcommand we are running
888
+ - `MRSK_DESTINATION` - optional: destination, e.g. "staging"
887
889
  - `MRSK_ROLE` - optional: role targeted, e.g. "web"
888
890
 
889
- There are three hooks:
891
+ There are four hooks:
890
892
 
891
893
  1. pre-connect
892
894
  Called before taking the deploy lock. For checks that need to run before connecting to remote hosts - e.g. DNS warming.
@@ -894,6 +896,9 @@ Called before taking the deploy lock. For checks that need to run before connect
894
896
  2. pre-build
895
897
  Used for pre-build checks - e.g. there are no uncommitted changes or that CI has passed.
896
898
 
899
+ 3. pre-deploy
900
+ For final checks before deploying, e.g. checking CI completed
901
+
897
902
  3. post-deploy - run after a deploy, redeploy or rollback
898
903
 
899
904
  This hook is also passed a `MRSK_RUNTIME` env variable.
data/lib/mrsk/cli/app.rb CHANGED
@@ -29,7 +29,7 @@ class Mrsk::Cli::App < Mrsk::Cli::Base
29
29
  execute *auditor.record("Booted app version #{version}"), verbosity: :debug
30
30
 
31
31
  old_version = capture_with_info(*app.current_running_version, raise_on_non_zero_exit: false).strip
32
- execute *app.start_or_run
32
+ execute *app.start_or_run(hostname: "#{host}-#{SecureRandom.hex(6)}")
33
33
 
34
34
  Mrsk::Utils::HealthcheckPoller.wait_for_healthy(pause_after_ready: true) { capture_with_info(*app.status(version: version)) }
35
35
 
data/lib/mrsk/cli/base.rb CHANGED
@@ -133,15 +133,39 @@ module Mrsk::Cli
133
133
  end
134
134
  end
135
135
 
136
- def run_hook(hook, **details)
136
+ def run_hook(hook, **extra_details)
137
137
  if !options[:skip_hooks] && MRSK.hook.hook_exists?(hook)
138
+ details = { hosts: MRSK.hosts.join(","), command: command, subcommand: subcommand }
139
+
138
140
  say "Running the #{hook} hook...", :magenta
139
141
  run_locally do
140
- MRSK.with_verbosity(:debug) { execute *MRSK.hook.run(hook, **details, hosts: MRSK.hosts.join(",")) }
142
+ MRSK.with_verbosity(:debug) { execute *MRSK.hook.run(hook, **details, **extra_details) }
141
143
  rescue SSHKit::Command::Failed
142
144
  raise HookError.new("Hook `#{hook}` failed")
143
145
  end
144
146
  end
145
147
  end
146
- end
148
+
149
+ def command
150
+ @mrsk_command ||= begin
151
+ invocation_class, invocation_commands = *first_invocation
152
+ if invocation_class == Mrsk::Cli::Main
153
+ invocation_commands[0]
154
+ else
155
+ Mrsk::Cli::Main.subcommand_classes.find { |command, clazz| clazz == invocation_class }[0]
156
+ end
157
+ end
158
+ end
159
+
160
+ def subcommand
161
+ @mrsk_subcommand ||= begin
162
+ invocation_class, invocation_commands = *first_invocation
163
+ invocation_commands[0] if invocation_class != Mrsk::Cli::Main
164
+ end
165
+ end
166
+
167
+ def first_invocation
168
+ instance_variable_get("@_invocations").first
169
+ end
170
+ end
147
171
  end
data/lib/mrsk/cli/lock.rb CHANGED
@@ -7,7 +7,7 @@ class Mrsk::Cli::Lock < Mrsk::Cli::Base
7
7
  end
8
8
 
9
9
  desc "acquire", "Acquire the deploy lock"
10
- option :message, aliases: "-m", type: :string, desc: "A lock mesasge", required: true
10
+ option :message, aliases: "-m", type: :string, desc: "A lock message", required: true
11
11
  def acquire
12
12
  message = options[:message]
13
13
  raise_if_locked do
data/lib/mrsk/cli/main.rb CHANGED
@@ -28,6 +28,8 @@ class Mrsk::Cli::Main < Mrsk::Cli::Base
28
28
  invoke "mrsk:cli:build:deliver", [], invoke_options
29
29
  end
30
30
 
31
+ run_hook "pre-deploy"
32
+
31
33
  say "Ensure Traefik is running...", :magenta
32
34
  invoke "mrsk:cli:traefik:boot", [], invoke_options
33
35
 
@@ -62,6 +64,8 @@ class Mrsk::Cli::Main < Mrsk::Cli::Base
62
64
  invoke "mrsk:cli:build:deliver", [], invoke_options
63
65
  end
64
66
 
67
+ run_hook "pre-deploy"
68
+
65
69
  say "Ensure app can pass healthcheck...", :magenta
66
70
  invoke "mrsk:cli:healthcheck:perform", [], invoke_options
67
71
 
@@ -86,6 +90,8 @@ class Mrsk::Cli::Main < Mrsk::Cli::Base
86
90
  old_version = nil
87
91
 
88
92
  if container_available?(version)
93
+ run_hook "pre-deploy"
94
+
89
95
  invoke "mrsk:cli:app:boot", [], invoke_options.merge(version: version)
90
96
  rolled_back = true
91
97
  else
@@ -12,7 +12,8 @@ class Mrsk::Cli::Prune < Mrsk::Cli::Base
12
12
  with_lock do
13
13
  on(MRSK.hosts) do
14
14
  execute *MRSK.auditor.record("Pruned images"), verbosity: :debug
15
- execute *MRSK.prune.images
15
+ execute *MRSK.prune.dangling_images
16
+ execute *MRSK.prune.tagged_images
16
17
  end
17
18
  end
18
19
  end
@@ -0,0 +1,82 @@
1
+ #!/bin/sh
2
+
3
+ # A sample pre-deploy hook
4
+ #
5
+ # Checks the Github status of the build, waiting for a pending build to complete for up to 720 seconds.
6
+ #
7
+ # Fails unless the combined status is "success"
8
+ #
9
+ # These environment variables are available:
10
+ # MRSK_RECORDED_AT
11
+ # MRSK_PERFORMER
12
+ # MRSK_VERSION
13
+ # MRSK_HOSTS
14
+ # MRSK_COMMAND
15
+ # MRSK_SUBCOMMAND
16
+ # MRSK_ROLE (if set)
17
+ # MRSK_DESTINATION (if set)
18
+
19
+ #!/usr/bin/env ruby
20
+
21
+ # Only check the build status for production deployments
22
+ if ENV["MRSK_COMMAND"] == "rollback" || ENV["MRSK_DESTINATION"] != "production"
23
+ exit 0
24
+ end
25
+
26
+ require "bundler/inline"
27
+
28
+ # true = install gems so this is fast on repeat invocations
29
+ gemfile(true, quiet: true) do
30
+ source "https://rubygems.org"
31
+
32
+ gem "octokit"
33
+ gem "faraday-retry"
34
+ end
35
+
36
+ MAX_ATTEMPTS = 72
37
+ ATTEMPTS_GAP = 10
38
+
39
+ def exit_with_error(message)
40
+ $stderr.puts message
41
+ exit 1
42
+ end
43
+
44
+ def first_status_url(combined_status, state)
45
+ first_status = combined_status[:statuses].find { |status| status[:state] == state }
46
+ first_status && first_status[:target_url]
47
+ end
48
+
49
+ remote_url = `git config --get remote.origin.url`.strip.delete_prefix("https://github.com/")
50
+ git_sha = `git rev-parse HEAD`.strip
51
+
52
+ repository = Octokit::Repository.from_url(remote_url)
53
+ github_client = Octokit::Client.new(access_token: ENV["GITHUB_TOKEN"])
54
+ attempts = 0
55
+
56
+ begin
57
+ loop do
58
+ combined_status = github_client.combined_status(remote_url, git_sha)
59
+ state = combined_status[:state]
60
+ first_status_url = first_status_url(combined_status, state)
61
+
62
+ case state
63
+ when "success"
64
+ puts "Build passed, see #{first_status_url}"
65
+ exit 0
66
+ when "failure"
67
+ exit_with_error "Build failed, see #{first_status_url}"
68
+ when "pending"
69
+ attempts += 1
70
+ end
71
+
72
+ puts "Waiting #{ATTEMPTS_GAP} more seconds for build to complete#{", see #{first_status_url}" if first_status_url}..."
73
+
74
+ if attempts == MAX_ATTEMPTS
75
+ exit_with_error "Build status is still pending, gave up after #{MAX_ATTEMPTS * ATTEMPTS_GAP} seconds"
76
+ end
77
+
78
+ sleep(ATTEMPTS_GAP)
79
+ end
80
+ rescue Octokit::NotFound
81
+ exit_with_error "Build status could not be found"
82
+ end
@@ -1,4 +1,6 @@
1
1
  class Mrsk::Commands::App < Mrsk::Commands::Base
2
+ ACTIVE_DOCKER_STATUSES = [ :running, :restarting ]
3
+
2
4
  attr_reader :role
3
5
 
4
6
  def initialize(config, role: nil)
@@ -6,17 +8,18 @@ class Mrsk::Commands::App < Mrsk::Commands::Base
6
8
  @role = role
7
9
  end
8
10
 
9
- def start_or_run
10
- combine start, run, by: "||"
11
+ def start_or_run(hostname: nil)
12
+ combine start, run(hostname: hostname), by: "||"
11
13
  end
12
14
 
13
- def run
15
+ def run(hostname: nil)
14
16
  role = config.role(self.role)
15
17
 
16
18
  docker :run,
17
19
  "--detach",
18
20
  "--restart unless-stopped",
19
21
  "--name", container_name,
22
+ *(["--hostname", hostname] if hostname),
20
23
  "-e", "MRSK_CONTAINER_NAME=\"#{container_name}\"",
21
24
  *role.env_args,
22
25
  *role.health_check_args,
@@ -92,7 +95,7 @@ class Mrsk::Commands::App < Mrsk::Commands::Base
92
95
 
93
96
 
94
97
  def current_running_container_id
95
- docker :ps, "--quiet", *filter_args(status: :running), "--latest"
98
+ docker :ps, "--quiet", *filter_args(statuses: ACTIVE_DOCKER_STATUSES), "--latest"
96
99
  end
97
100
 
98
101
  def container_id_for_version(version, only_running: false)
@@ -100,12 +103,12 @@ class Mrsk::Commands::App < Mrsk::Commands::Base
100
103
  end
101
104
 
102
105
  def current_running_version
103
- list_versions("--latest", status: :running)
106
+ list_versions("--latest", statuses: ACTIVE_DOCKER_STATUSES)
104
107
  end
105
108
 
106
- def list_versions(*docker_args, status: nil)
109
+ def list_versions(*docker_args, statuses: nil)
107
110
  pipe \
108
- docker(:ps, *filter_args(status: status), *docker_args, "--format", '"{{.Names}}"'),
111
+ docker(:ps, *filter_args(statuses: statuses), *docker_args, "--format", '"{{.Names}}"'),
109
112
  %(grep -oE "\\-[^-]+$"), # Extract SHA from "service-role-dest-SHA"
110
113
  %(cut -c 2-)
111
114
  end
@@ -150,15 +153,17 @@ class Mrsk::Commands::App < Mrsk::Commands::Base
150
153
  [ config.service, role, config.destination, version || config.version ].compact.join("-")
151
154
  end
152
155
 
153
- def filter_args(status: nil)
154
- argumentize "--filter", filters(status: status)
156
+ def filter_args(statuses: nil)
157
+ argumentize "--filter", filters(statuses: statuses)
155
158
  end
156
159
 
157
- def filters(status: nil)
160
+ def filters(statuses: nil)
158
161
  [ "label=service=#{config.service}" ].tap do |filters|
159
162
  filters << "label=destination=#{config.destination}" if config.destination
160
163
  filters << "label=role=#{role}" if role
161
- filters << "status=#{status}" if status
164
+ statuses&.each do |status|
165
+ filters << "status=#{status}"
166
+ end
162
167
  end
163
168
  end
164
169
  end
@@ -2,13 +2,20 @@ require "active_support/duration"
2
2
  require "active_support/core_ext/numeric/time"
3
3
 
4
4
  class Mrsk::Commands::Prune < Mrsk::Commands::Base
5
- def images
5
+ def dangling_images
6
6
  docker :image, :prune, "--force", "--filter", "label=service=#{config.service}", "--filter", "dangling=true"
7
7
  end
8
8
 
9
+ def tagged_images
10
+ pipe \
11
+ docker(:image, :ls, *service_filter, "--format", "'{{.ID}} {{.Repository}}:{{.Tag}}'"),
12
+ "grep -v -w \"#{active_image_list}\"",
13
+ "while read image tag; do docker rmi $tag; done"
14
+ end
15
+
9
16
  def containers(keep_last: 5)
10
17
  pipe \
11
- docker(:ps, "-q", "-a", "--filter", "label=service=#{config.service}", *stopped_containers_filters),
18
+ docker(:ps, "-q", "-a", *service_filter, *stopped_containers_filters),
12
19
  "tail -n +#{keep_last + 1}",
13
20
  "while read container_id; do docker rm $container_id; done"
14
21
  end
@@ -17,4 +24,15 @@ class Mrsk::Commands::Prune < Mrsk::Commands::Base
17
24
  def stopped_containers_filters
18
25
  [ "created", "exited", "dead" ].flat_map { |status| ["--filter", "status=#{status}"] }
19
26
  end
27
+
28
+ def active_image_list
29
+ # Pull the images that are used by any containers
30
+ # Append repo:latest - to avoid deleting the latest tag
31
+ # Append repo:<none> - to avoid deleting dangling images that are in use. Unused dangling images are deleted separately
32
+ "$(docker container ls -a --format '{{.Image}}\\|' --filter label=service=#{config.service} | tr -d '\\n')#{config.latest_image}\\|#{config.repository}:<none>"
33
+ end
34
+
35
+ def service_filter
36
+ [ "--filter", "label=service=#{config.service}" ]
37
+ end
20
38
  end
data/lib/mrsk/version.rb CHANGED
@@ -1,3 +1,3 @@
1
1
  module Mrsk
2
- VERSION = "0.13.0"
2
+ VERSION = "0.13.2"
3
3
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: mrsk
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.13.0
4
+ version: 0.13.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - David Heinemeier Hansson
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2023-05-25 00:00:00.000000000 Z
11
+ date: 2023-06-01 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: activesupport
@@ -190,6 +190,7 @@ files:
190
190
  - lib/mrsk/cli/templates/sample_hooks/post-deploy.sample
191
191
  - lib/mrsk/cli/templates/sample_hooks/pre-build.sample
192
192
  - lib/mrsk/cli/templates/sample_hooks/pre-connect.sample
193
+ - lib/mrsk/cli/templates/sample_hooks/pre-deploy.sample
193
194
  - lib/mrsk/cli/templates/template.env
194
195
  - lib/mrsk/cli/traefik.rb
195
196
  - lib/mrsk/commander.rb