mrsk 0.13.0 → 0.13.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 6b258a4c44d5b010d6ab4004d0e051f9378dd4511203020a8fed2dc639527d9f
-  data.tar.gz: 702b1c42fc0b095d18c37baa1811186ece2267b3d49067648129b76bef284be1
+  metadata.gz: 3a752ca316aea9ce851cc9b4ac96cd7fde613f94dd1b63e238c3320b10f82e8c
+  data.tar.gz: 689bba95ccc72b27b9cc85572b6e4d995f6bfed3d09be30c2076a81ed1eb07f6
 SHA512:
-  metadata.gz: 60163759e4097ea91df33411e7dc32261c0bc82f609bc1f54308f4111e3f0efe4d9731b234175e293f5c8a6aef9e1a88156be88260fc0a545b4d276ddb4e753b
-  data.tar.gz: 882d09991704e45f3c377dfafa0b15f2c6f1421085e5ca8096855bb03b109c6a91e4e47b7ec9270cdb31f7e43b3a18b23576fb0b2faa6367c82d9b3748550201
+  metadata.gz: cdbb9e9b58364ba933778bb18bf4576ee3c611fcbb3be9b341c2e21d53d08d37bd524c694e058be4456f0d7da837b488d20c009973017fa39f6c57fb5835d0ce
+  data.tar.gz: eb58c8ac236172f91719582f7b9606d2255c122be41c7f341a6a54e6b1680b4715f55ad99492ee47a57c38b74626b129d92bd77ed0d864042a7c457343aade13

data/README.md

@@ -67,7 +67,7 @@ Voila! All the servers are now serving the app on port 80. If you're just runnin
 
 In the past decade+, there's been an explosion in commercial offerings that make deploying web apps easier. Heroku kicked it off with an incredible offering that stayed ahead of the competition seemingly forever. These days we have excellent alternatives like Fly.io and Render. And hosted Kubernetes is making things easier too on AWS, GCP, Digital Ocean, and elsewhere. But these are all offerings that have you renting computers in the cloud at a premium. If you want to run on your own hardware, or even just have a clear migration path to do so in the future, you need to carefully consider how locked in you get to these commercial platforms. Preferably before the bills swallow your business whole!
 
-MRSK seeks to bring the advance in ergonomics pioneered by these commercial offerings to deploying web apps anywhere. Whether that's low-cost cloud options without the managed-service markup from the likes of Digital Ocean, Hetzner, OVH, etc, or it's your own colocated bare metal. To MRSK, it's all the same. Feed the config file a list of IP addresses with vanilla Ubuntu servers that have seen no prep beyond an added SSH key, and you'll be running in literally minutes.
+MRSK seeks to bring the advance in ergonomics pioneered by these commercial offerings to deploying web apps anywhere. Whether that's low-cost cloud options without the managed-service markup from the likes of Digital Ocean, Hetzner, OVH, etc., or it's your own colocated bare metal. To MRSK, it's all the same. Feed the config file a list of IP addresses with vanilla Ubuntu servers that have seen no prep beyond an added SSH key, and you'll be running in literally minutes.
 
 This approach gives you enormous portability. You can have your web app deployed on several clouds at ease like this. Or you can buy the baseline with your own hardware, then deploy to a cloud before a big seasonal spike to get more capacity. When you're not locked into a single provider from a tooling perspective, there are a lot of compelling options available.
 
@@ -670,7 +670,7 @@ This assumes the Cron settings are stored in `config/crontab`.
 
 ### Healthcheck
 
-MRSK uses Docker healtchecks to check the health of your application during deployment. Traefik uses this same healthcheck status to determine when a container is ready to receive traffic.
+MRSK uses Docker healthchecks to check the health of your application during deployment. Traefik uses this same healthcheck status to determine when a container is ready to receive traffic.
 
 The healthcheck defaults to testing the HTTP response to the path `/up` on port 3000, up to 7 times. You can tailor this behaviour with the `healthcheck` setting:
 
@@ -840,7 +840,7 @@ Message: Automatic deploy lock
 You can also manually acquire and release the lock
 
 ```
-mrsk lock acquire -m "Doing maintanence"
+mrsk lock acquire -m "Doing maintenance"
 ```
 
 ```
@@ -882,11 +882,13 @@ firing a JSON webhook. These variables include:
 - `MRSK_PERFORMER` - the local user performing the command (from `whoami`)
 - `MRSK_SERVICE_VERSION` - an abbreviated service and version for use in messages, e.g. app@150b24f
 - `MRSK_VERSION` - an full version being deployed
-- `MRSK_DESTINATION` - optional: destination, e.g. "staging"
 - `MRSK_HOSTS` - a comma separated list of the hosts targeted by the command
+- `MRSK_COMMAND` - The command we are running
+- `MRSK_SUBCOMMAND` - optional: The subcommand we are running
+- `MRSK_DESTINATION` - optional: destination, e.g. "staging"
 - `MRSK_ROLE` - optional: role targeted, e.g. "web"
 
-There are three hooks:
+There are four hooks:
 
 1. pre-connect
 Called before taking the deploy lock. For checks that need to run before connecting to remote hosts - e.g. DNS warming.
@@ -894,6 +896,9 @@ Called before taking the deploy lock. For checks that need to run before connect
 2. pre-build
 Used for pre-build checks - e.g. there are no uncommitted changes or that CI has passed.
 
+3. pre-deploy
+For final checks before deploying, e.g. checking CI completed
+
 3. post-deploy - run after a deploy, redeploy or rollback
 
 This hook is also passed a `MRSK_RUNTIME` env variable.

data/lib/mrsk/cli/app.rb

@@ -29,7 +29,7 @@ class Mrsk::Cli::App < Mrsk::Cli::Base
               execute *auditor.record("Booted app version #{version}"), verbosity: :debug
 
               old_version = capture_with_info(*app.current_running_version, raise_on_non_zero_exit: false).strip
-              execute *app.start_or_run
+              execute *app.start_or_run(hostname: "#{host}-#{SecureRandom.hex(6)}")
 
               Mrsk::Utils::HealthcheckPoller.wait_for_healthy(pause_after_ready: true) { capture_with_info(*app.status(version: version)) }
 

data/lib/mrsk/cli/base.rb

@@ -133,15 +133,39 @@ module Mrsk::Cli
         end
       end
 
-      def run_hook(hook, **details)
+      def run_hook(hook, **extra_details)
         if !options[:skip_hooks] && MRSK.hook.hook_exists?(hook)
+          details = { hosts: MRSK.hosts.join(","), command: command, subcommand: subcommand }
+
           say "Running the #{hook} hook...", :magenta
           run_locally do
-            MRSK.with_verbosity(:debug) { execute *MRSK.hook.run(hook, **details, hosts: MRSK.hosts.join(",")) }
+            MRSK.with_verbosity(:debug) { execute *MRSK.hook.run(hook, **details, **extra_details) }
           rescue SSHKit::Command::Failed
             raise HookError.new("Hook `#{hook}` failed")
           end
         end
       end
-  end
+
+      def command
+        @mrsk_command ||= begin
+          invocation_class, invocation_commands = *first_invocation
+          if invocation_class == Mrsk::Cli::Main
+            invocation_commands[0]
+          else
+            Mrsk::Cli::Main.subcommand_classes.find { |command, clazz| clazz == invocation_class }[0]
+          end
+        end
+      end
+
+      def subcommand
+        @mrsk_subcommand ||= begin
+          invocation_class, invocation_commands = *first_invocation
+          invocation_commands[0] if invocation_class != Mrsk::Cli::Main
+        end
+      end
+
+      def first_invocation
+        instance_variable_get("@_invocations").first
+      end
+    end
 end

data/lib/mrsk/cli/lock.rb

@@ -7,7 +7,7 @@ class Mrsk::Cli::Lock < Mrsk::Cli::Base
   end
 
   desc "acquire", "Acquire the deploy lock"
-  option :message, aliases: "-m", type: :string, desc: "A lock mesasge", required: true
+  option :message, aliases: "-m", type: :string, desc: "A lock message", required: true
   def acquire
     message = options[:message]
     raise_if_locked do

data/lib/mrsk/cli/main.rb

@@ -28,6 +28,8 @@ class Mrsk::Cli::Main < Mrsk::Cli::Base
           invoke "mrsk:cli:build:deliver", [], invoke_options
         end
 
+        run_hook "pre-deploy"
+
         say "Ensure Traefik is running...", :magenta
         invoke "mrsk:cli:traefik:boot", [], invoke_options
 
@@ -62,6 +64,8 @@ class Mrsk::Cli::Main < Mrsk::Cli::Base
           invoke "mrsk:cli:build:deliver", [], invoke_options
         end
 
+        run_hook "pre-deploy"
+
         say "Ensure app can pass healthcheck...", :magenta
         invoke "mrsk:cli:healthcheck:perform", [], invoke_options
 
@@ -86,6 +90,8 @@ class Mrsk::Cli::Main < Mrsk::Cli::Base
         old_version = nil
 
         if container_available?(version)
+          run_hook "pre-deploy"
+
           invoke "mrsk:cli:app:boot", [], invoke_options.merge(version: version)
           rolled_back = true
         else

data/lib/mrsk/cli/prune.rb

@@ -12,7 +12,8 @@ class Mrsk::Cli::Prune < Mrsk::Cli::Base
     with_lock do
       on(MRSK.hosts) do
         execute *MRSK.auditor.record("Pruned images"), verbosity: :debug
-        execute *MRSK.prune.images
+        execute *MRSK.prune.dangling_images
+        execute *MRSK.prune.tagged_images
       end
     end
   end

data/lib/mrsk/cli/templates/sample_hooks/pre-deploy.sample

@@ -0,0 +1,82 @@
+#!/bin/sh
+
+# A sample pre-deploy hook
+#
+# Checks the Github status of the build, waiting for a pending build to complete for up to 720 seconds.
+#
+# Fails unless the combined status is "success"
+#
+# These environment variables are available:
+# MRSK_RECORDED_AT
+# MRSK_PERFORMER
+# MRSK_VERSION
+# MRSK_HOSTS
+# MRSK_COMMAND
+# MRSK_SUBCOMMAND
+# MRSK_ROLE (if set)
+# MRSK_DESTINATION (if set)
+
+#!/usr/bin/env ruby
+
+# Only check the build status for production deployments
+if ENV["MRSK_COMMAND"] == "rollback" || ENV["MRSK_DESTINATION"] != "production"
+  exit 0
+end
+
+require "bundler/inline"
+
+# true = install gems so this is fast on repeat invocations
+gemfile(true, quiet: true) do
+  source "https://rubygems.org"
+
+  gem "octokit"
+  gem "faraday-retry"
+end
+
+MAX_ATTEMPTS = 72
+ATTEMPTS_GAP = 10
+
+def exit_with_error(message)
+  $stderr.puts message
+  exit 1
+end
+
+def first_status_url(combined_status, state)
+  first_status = combined_status[:statuses].find { |status| status[:state] == state }
+  first_status && first_status[:target_url]
+end
+
+remote_url = `git config --get remote.origin.url`.strip.delete_prefix("https://github.com/")
+git_sha = `git rev-parse HEAD`.strip
+
+repository = Octokit::Repository.from_url(remote_url)
+github_client = Octokit::Client.new(access_token: ENV["GITHUB_TOKEN"])
+attempts = 0
+
+begin
+  loop do
+    combined_status = github_client.combined_status(remote_url, git_sha)
+    state = combined_status[:state]
+    first_status_url = first_status_url(combined_status, state)
+
+    case state
+    when "success"
+      puts "Build passed, see #{first_status_url}"
+      exit 0
+    when "failure"
+      exit_with_error "Build failed, see #{first_status_url}"
+    when "pending"
+      attempts += 1
+    end
+
+    puts "Waiting #{ATTEMPTS_GAP} more seconds for build to complete#{", see #{first_status_url}" if first_status_url}..."
+
+    if attempts == MAX_ATTEMPTS
+      exit_with_error "Build status is still pending, gave up after #{MAX_ATTEMPTS * ATTEMPTS_GAP} seconds"
+    end
+
+    sleep(ATTEMPTS_GAP)
+  end
+rescue Octokit::NotFound
+  exit_with_error "Build status could not be found"
+end

data/lib/mrsk/commands/app.rb

@@ -1,4 +1,6 @@
 class Mrsk::Commands::App < Mrsk::Commands::Base
+  ACTIVE_DOCKER_STATUSES = [ :running, :restarting ]
+
   attr_reader :role
 
   def initialize(config, role: nil)
@@ -6,17 +8,18 @@ class Mrsk::Commands::App < Mrsk::Commands::Base
     @role = role
   end
 
-  def start_or_run
-    combine start, run, by: "||"
+  def start_or_run(hostname: nil)
+    combine start, run(hostname: hostname), by: "||"
   end
 
-  def run
+  def run(hostname: nil)
     role = config.role(self.role)
 
     docker :run,
       "--detach",
       "--restart unless-stopped",
       "--name", container_name,
+      *(["--hostname", hostname] if hostname),
       "-e", "MRSK_CONTAINER_NAME=\"#{container_name}\"",
       *role.env_args,
       *role.health_check_args,
@@ -92,7 +95,7 @@ class Mrsk::Commands::App < Mrsk::Commands::Base
 
 
   def current_running_container_id
-    docker :ps, "--quiet", *filter_args(status: :running), "--latest"
+    docker :ps, "--quiet", *filter_args(statuses: ACTIVE_DOCKER_STATUSES), "--latest"
   end
 
   def container_id_for_version(version, only_running: false)
@@ -100,12 +103,12 @@ class Mrsk::Commands::App < Mrsk::Commands::Base
   end
 
   def current_running_version
-    list_versions("--latest", status: :running)
+    list_versions("--latest", statuses: ACTIVE_DOCKER_STATUSES)
   end
 
-  def list_versions(*docker_args, status: nil)
+  def list_versions(*docker_args, statuses: nil)
     pipe \
-      docker(:ps, *filter_args(status: status), *docker_args, "--format", '"{{.Names}}"'),
+      docker(:ps, *filter_args(statuses: statuses), *docker_args, "--format", '"{{.Names}}"'),
       %(grep -oE "\\-[^-]+$"), # Extract SHA from "service-role-dest-SHA"
       %(cut -c 2-)
   end
@@ -150,15 +153,17 @@ class Mrsk::Commands::App < Mrsk::Commands::Base
       [ config.service, role, config.destination, version || config.version ].compact.join("-")
     end
 
-    def filter_args(status: nil)
-      argumentize "--filter", filters(status: status)
+    def filter_args(statuses: nil)
+      argumentize "--filter", filters(statuses: statuses)
     end
 
-    def filters(status: nil)
+    def filters(statuses: nil)
       [ "label=service=#{config.service}" ].tap do |filters|
         filters << "label=destination=#{config.destination}" if config.destination
         filters << "label=role=#{role}" if role
-        filters << "status=#{status}" if status
+        statuses&.each do |status|
+          filters << "status=#{status}"
+        end
       end
     end
 end

data/lib/mrsk/commands/prune.rb

@@ -2,13 +2,20 @@ require "active_support/duration"
 require "active_support/core_ext/numeric/time"
 
 class Mrsk::Commands::Prune < Mrsk::Commands::Base
-  def images
+  def dangling_images
     docker :image, :prune, "--force", "--filter", "label=service=#{config.service}", "--filter", "dangling=true"
   end
 
+  def tagged_images
+    pipe \
+      docker(:image, :ls, *service_filter, "--format", "'{{.ID}} {{.Repository}}:{{.Tag}}'"),
+      "grep -v -w \"#{active_image_list}\"",
+      "while read image tag; do docker rmi $tag; done"
+  end
+
   def containers(keep_last: 5)
     pipe \
-      docker(:ps, "-q", "-a", "--filter", "label=service=#{config.service}", *stopped_containers_filters),
+      docker(:ps, "-q", "-a", *service_filter, *stopped_containers_filters),
       "tail -n +#{keep_last + 1}",
       "while read container_id; do docker rm $container_id; done"
   end
@@ -17,4 +24,15 @@ class Mrsk::Commands::Prune < Mrsk::Commands::Base
     def stopped_containers_filters
       [ "created", "exited", "dead" ].flat_map { |status| ["--filter", "status=#{status}"] }
     end
+
+    def active_image_list
+      # Pull the images that are used by any containers
+      # Append repo:latest - to avoid deleting the latest tag
+      # Append repo:<none> - to avoid deleting dangling images that are in use. Unused dangling images are deleted separately
+      "$(docker container ls -a --format '{{.Image}}\\|' --filter label=service=#{config.service} | tr -d '\\n')#{config.latest_image}\\|#{config.repository}:<none>"
+    end
+
+    def service_filter
+      [ "--filter", "label=service=#{config.service}" ]
+    end
 end

data/lib/mrsk/version.rb

@@ -1,3 +1,3 @@
 module Mrsk
-  VERSION = "0.13.0"
+  VERSION = "0.13.2"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: mrsk
 version: !ruby/object:Gem::Version
-  version: 0.13.0
+  version: 0.13.2
 platform: ruby
 authors:
 - David Heinemeier Hansson
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2023-05-25 00:00:00.000000000 Z
+date: 2023-06-01 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activesupport
@@ -190,6 +190,7 @@ files:
 - lib/mrsk/cli/templates/sample_hooks/post-deploy.sample
 - lib/mrsk/cli/templates/sample_hooks/pre-build.sample
 - lib/mrsk/cli/templates/sample_hooks/pre-connect.sample
+- lib/mrsk/cli/templates/sample_hooks/pre-deploy.sample
 - lib/mrsk/cli/templates/template.env
 - lib/mrsk/cli/traefik.rb
 - lib/mrsk/commander.rb