resqued 0.12.4 → 0.13.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 767297afcad9765330342e4dc397b27cdae3ca7c40e6d51c26b389d2708534a2
-  data.tar.gz: dacfd4dcfc28b2f7d632d5dbfed1654fd226b30756f0e576efeee4560683c93e
+  metadata.gz: 7104c2e7f23dc24ebaf0fb864c8b53ae33ee02315c3a2baa91f60fea39eace6e
+  data.tar.gz: 3684ecfbc8f0285c9e4c9b6914f0940d74491cebf378651a679cfca7c4335183
 SHA512:
-  metadata.gz: 272ea62737a31ca4c3a32d4fbc8e263a2c6642f03d962febb68a6fc61dc06139486ebff33decb224c4c0abce2e0f4ccd2f2df820534babacce8e7d9a78bbee68
-  data.tar.gz: aa952102dc7c5e41c859c792bc4b1b5ce362d7f5d2edcc01b52f3c7dbe51d09f1752dcd40708b7d16fd1970fed4c9bb5a5229903ba2dda2be3fa9dc11b436205
+  metadata.gz: e5e4cf0be88ae6fc02951ab7a7262d67cc9dac404696265a7cf785a2646ead92faa6eed0c686fc56b0844fa958753a18161d5d8d1d23474fa805ca52e79fd0f3
+  data.tar.gz: 51cf8397eec968b97b525d83506e3fbb8c84d0b11c989089c926b6242da0bb54dd0ad02ada99cfc370463d1aaaa94fb6257adcd6904e298a87cf352e9c6841c4

data/CHANGES.md

@@ -1,5 +1,15 @@
 Starting with version 0.6.1, resqued uses semantic versioning to indicate incompatibilities between the master process, listener process, and configuration.
 
+v0.13.0
+-------
+* Re-exec on SIGUSR1 instead of SIGHUP. There should be no compatibility
+  problem between 0.12 and 0.13, but the behavior will be different. If you
+  start 0.12 and then upgrade to 0.13, resqued will re-exec on HUP until it
+  hits 0.13. If you downgrade to 0.12, resqued's master will stay at 0.13 even
+  though the listener changes to 0.12; `kill -USR1 $pid` to get the 0.13 master
+  to roll back to 0.12, too. Note: take care not to `kill -USR1` a 0.12
+  resqued, as that will cause it to shut down. (#71)
+
 v0.12.4
 -------
 * Add support for redis client gem version 5. (#70)

data/README.md

@@ -2,7 +2,7 @@
 
 Resqued is a multi-process daemon that controls and monitors a pool of resque workers. It works well with slow jobs and continuous delivery.
 
-[![travis status](https://travis-ci.org/spraints/resqued.png)](https://travis-ci.org/spraints/resqued)
+![CI status](https://github.com/spraints/resqued/actions/workflows/ruby.yml/badge.svg)
 
 ## Installation
 
@@ -57,7 +57,11 @@ This time, you'd end up with something similar to this:
 
 ## Launching in production.
 
-Resqued restarts when its master process receives `SIGHUP`. It restarts by re-execing the command that you initially ran. There are two main recommendations for running in production.
+Resqued consists of three types of processes: master, listener, and worker. The master process only loads code from the resqued gem, and it tries to converge on running exactly one listener. The listener process loads your app's code (via the `before_fork` hook) and launches the workers you've configured. Each worker is a single process.
+
+Resqued restarts on SIGHUP by starting a new listener, and then replacing workers from the old pool with workers in the new pool. Note that this means that if you change the version of Resqued in your bundle, the master and listener processes will be using different versions. This is completely safe (except as noted in CHANGES.md). But, to help with transitions between versions, you can use `kill -USR1` to tell the master process to re-exec itself, which will get the master and listener processes in sync again.
+
+There are two main recommendations for running in production:
 
 * If you use bundler to install resqued, tell it to generate a binstub for resqued. Invoke this binstub (e.g. `bin/resqued`) when you start resqued.
 
@@ -71,18 +75,17 @@ If your application is running from a symlinked dir (for example, [capistrano's
 
 * If you're invoking resqued from something that resolves symlinks in `pwd`, you'll also want to explicitly set the `PWD` environment variable.
 
-Rolling all of the above advice together, here's a sample that you could use in an upstart script for resqued:
+Putting all of the above advice together, here's a sample that you could use as a systemd unit:
 
 ```
-# fragment of /etc/init/resqued.conf
-
-kill signal QUIT
+# fragment of resqued.service
 
-env BUNDLE_GEMFILE=/opt/app/current/Gemfile
-env PWD=/opt/app/current
-chdir /opt/app/current
+[Service]
+Type=simple
 
-exec bin/resqued -c config/resqued.rb -p /opt/app/shared/tmp/pids/resqued.pid
+WorkingDirectory=/opt/app/current
+ExecStart=bin/resqued config/resqued.rb
+ExecReload=/bin/kill -HUP $MAINPID
 ```
 
 ## Compatibility with Resque

data/docs/signals.md

@@ -8,6 +8,7 @@ Here is a summary of how signals get passed between resqued's processes:
                   master    listener    worker
                   ------    --------    ------
 restart            HUP   -> QUIT     -> QUIT
+reexec master     USR1   -> QUIT     -> QUIT
 exit now           INT   ->  INT     ->  INT
 exit now          TERM   -> TERM     -> TERM
 exit when ready   QUIT   -> QUIT     -> QUIT
@@ -23,6 +24,7 @@ Read on for more information about what the signals mean.
 The Master process handles several signals.
 
 * `HUP`: Start a new listener. After it boots, kill the previous listener with `SIGQUIT`.
+* `USR1`: Re-exec the master process, preserving the current state. Also exec a new listener.
 * `USR2`: Pause processing. Kills the current listener, and does not start a replacement.
 * `CONT`: Resume processing. If there is no listener, start one. If there is a listener, send it `SIGCONT`.
 * `QUIT`, `INT`, or `TERM`: Kill the listener with the same signal and wait for it to exit. If `--fast-exit` was specified, the master exits immediately without waiting for the listener to exit.

data/exe/resqued

@@ -13,7 +13,7 @@ end
 
 require "optparse"
 
-options = { exec_on_hup: true }
+options = {}
 daemonize = false
 test = false
 
@@ -52,10 +52,6 @@ opts = OptionParser.new do |opts| # rubocop: disable Lint/ShadowingOuterLocalVar
     daemonize = true
   end
 
-  opts.on "--no-exec-on-hup", "Do not re-exec the master process on SIGHUP" do
-    options[:exec_on_hup] = false
-  end
-
   opts.on "--replace FILE", "(internal)" do |v|
     options[:master_state] = v
   end
@@ -84,7 +80,7 @@ else
   state = Resqued::MasterState.new
   if options[:master_state]
     Resqued::Logging.logger.info "Resuming master from #{options[:master_state]}"
-    Resqued::ExecOnHUP.restore_state(state, options[:master_state])
+    Resqued::ReplaceMaster.restore_state(state, options[:master_state])
   else
     require_config_paths! options, opts
     Resqued.capture_start_ctx!

data/lib/resqued/master.rb

@@ -1,10 +1,10 @@
 require "resqued/backoff"
-require "resqued/exec_on_hup"
 require "resqued/listener_pool"
 require "resqued/logging"
 require "resqued/master_state"
 require "resqued/pidfile"
 require "resqued/procline_version"
+require "resqued/replace_master"
 require "resqued/sleepy"
 
 module Resqued
@@ -55,13 +55,12 @@ module Resqued
         when :INFO
           dump_object_counts
         when :HUP
-          if @state.exec_on_hup
-            log "Execing a new master"
-            ExecOnHUP.exec!(@state)
-          end
           reopen_logs
           log "Restarting listener with new configuration and application."
           prepare_new_listener
+        when :USR1
+          log "Execing a new master"
+          ReplaceMaster.exec!(@state)
         when :USR2
           log "Pause job processing"
           @state.paused = true
@@ -215,7 +214,7 @@ module Resqued
       end
     end
 
-    SIGNALS = [:HUP, :INT, :USR2, :CONT, :TERM, :QUIT].freeze
+    SIGNALS = [:HUP, :INT, :USR1, :USR2, :CONT, :TERM, :QUIT].freeze
     OPTIONAL_SIGNALS = [:INFO].freeze
     OTHER_SIGNALS = [:CHLD, "EXIT"].freeze
     TRAPS = SIGNALS + OPTIONAL_SIGNALS + OTHER_SIGNALS

data/lib/resqued/master_state.rb

@@ -1,11 +1,14 @@
 module Resqued
+  # Tracks state from the master process. On re-exec, this object will
+  # be serialized and passed to the new master.
   class MasterState
     def initialize
       @listeners_created = 0
       @listener_states = {}
     end
 
-    # Public: When starting fresh, from command-line options, assign the initial values.
+    # Public: When starting fresh, from command-line options, assign the
+    # initial values.
     def init(options)
       @config_paths = options.fetch(:config_paths)
       @exec_on_hup  = options.fetch(:exec_on_hup, false)
@@ -55,19 +58,45 @@ module Resqued
       }
     end
 
-    # Public: Return an array of open sockets or other file handles that should be forwarded to a new master.
+    # Public: Return an array of open sockets or other file handles that
+    # should be forwarded to a new master.
     def sockets
       @listener_states.values.map { |l| l.master_socket }.compact
     end
 
+    # Paths of app's resqued configuration files. The paths are passed
+    # to the listener, and the listener reads the config so that it
+    # knows which workers to create.
     attr_reader :config_paths
+
+    # The PID of the newest listener. This is the one listener that
+    # should continue running.
     attr_accessor :current_listener_pid
+
+    # (Deprecated.) If true, on SIGHUP re-exec the master process before
+    # starting a new listener.
     attr_reader :exec_on_hup
+
+    # If true, on SIGTERM/SIGQUIT/SIGINT, don't wait for listeners to
+    # exit before the master exits.
     attr_reader :fast_exit
+
+    # The PID of the newest listener that booted successfully. This
+    # listener won't be stopped until a newer listener boots
+    # successfully.
     attr_accessor :last_good_listener_pid
+
+    # The number of listeners that have been created by this master.
     attr_accessor :listeners_created
+
+    # A hash of pid => ListenerState for all running listeners.
     attr_reader :listener_states
+
+    # Set to true when this master is paused and should not be running
+    # any listeners.
     attr_accessor :paused
+
+    # If set, the master's PID will be written to this file.
     attr_reader :pidfile
   end
 end

data/lib/resqued/{exec_on_hup.rb → replace_master.rb}

@@ -2,7 +2,7 @@ require "tempfile"
 require "yaml"
 
 module Resqued
-  class ExecOnHUP
+  class ReplaceMaster
     # Public: Replace the current master process with a new one, while preserving state.
     def self.exec!(state)
       exec Resqued::START_CTX["$0"], "--replace", store_state(state), exec_opts(state)

data/lib/resqued/version.rb

@@ -1,3 +1,3 @@
 module Resqued
-  VERSION = "0.12.4".freeze
+  VERSION = "0.13.0".freeze
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: resqued
 version: !ruby/object:Gem::Version
-  version: 0.12.4
+  version: 0.13.0
 platform: ruby
 authors:
 - Matt Burke
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2023-01-05 00:00:00.000000000 Z
+date: 2023-12-01 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: kgio
@@ -118,7 +118,6 @@ files:
 - lib/resqued/config/dsl.rb
 - lib/resqued/config/worker.rb
 - lib/resqued/daemon.rb
-- lib/resqued/exec_on_hup.rb
 - lib/resqued/listener.rb
 - lib/resqued/listener_pool.rb
 - lib/resqued/listener_proxy.rb
@@ -129,6 +128,7 @@ files:
 - lib/resqued/pidfile.rb
 - lib/resqued/procline_version.rb
 - lib/resqued/quit-and-wait.rb
+- lib/resqued/replace_master.rb
 - lib/resqued/runtime_info.rb
 - lib/resqued/sleepy.rb
 - lib/resqued/test_case.rb