solid_queue 0.3.4 → 0.4.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b4a85fb424e127543352846ee1b0b4aa958d4c77a7a9e1c26a8082464ae7bebc
-  data.tar.gz: f15b4db5c77c14af4b989fc8abe2670fd49006853ca9bfce0080e8b15df733c3
+  metadata.gz: cb47bf3ee9dffa1300c093abb260394d088f100b3c16a026d755f706d9f7a852
+  data.tar.gz: 10cc1b6f866d148d0c1cefce7587614aaca3e938f3f04370c15318e5b8d3509f
 SHA512:
-  metadata.gz: ba913d760e6f1ac8bac01f68b3ffeb61811952d13a7862731c517dea0ec3e8938b8fb5194113e356fe8a35a02f4d95b306f327772e21f8f6098349bd1235dc53
-  data.tar.gz: 7418c2db5be34b59e123395ee3edf519d703b6e0488a564a85c24dc22344d67e894ad8bb378f88a3cd6e0b9085602be1850dab60c168816e92b081364b720594
+  metadata.gz: 71054017fcd26421f25140db9c929518ee011210fd6fea51e77352abf2a128289e965843dc1cb1fe97b62fa003d571e1de3631fcab329914b069ef9e0621806b
+  data.tar.gz: d08497fc98f9498aeaa3449182fc13cb90f71c859c67d55e0e5c14803bb1d5445e7717e1e3aca578fcb22a09c47126d465306e0220f54d6af4a8f8027fef0072

data/README.md

@@ -66,7 +66,7 @@ $ bundle exec rake solid_queue:start
 
 This will start processing jobs in all queues using the default configuration. See [below](#configuration) to learn more about configuring Solid Queue.
 
-For small projects, you can run Solid Queue on the same machine as your webserver. When you're ready to scale, Solid Queue supports horizontal scaling out-of-the-box. You can run Solid Queue on a separate server from your webserver, or even run `bundle exec rake solid_queue:start` on multiple machines at the same time. If you'd like to designate some machines to be only dispatchers or only workers, use `bundle exec rake solid_queue:dispatch` or `bundle exec rake solid_queue:work`, respectively.
+For small projects, you can run Solid Queue on the same machine as your webserver. When you're ready to scale, Solid Queue supports horizontal scaling out-of-the-box. You can run Solid Queue on a separate server from your webserver, or even run `bundle exec rake solid_queue:start` on multiple machines at the same time. Depending on the configuration, you can designate some machines to run only dispatchers or only workers. See the [configuration](#configuration) section for more details on this.
 
 ## Requirements
 Besides Rails 7.1, Solid Queue works best with MySQL 8+ or PostgreSQL 9.5+, as they support `FOR UPDATE SKIP LOCKED`. You can use it with older versions, but in that case, you might run into lock waits if you run multiple workers for the same queue.
@@ -75,10 +75,12 @@ Besides Rails 7.1, Solid Queue works best with MySQL 8+ or PostgreSQL 9.5+, as t
 
 ### Workers and dispatchers
 
-We have three types of processes in Solid Queue:
+We have three types of actors in Solid Queue:
 - _Workers_ are in charge of picking jobs ready to run from queues and processing them. They work off the `solid_queue_ready_executions` table.
 - _Dispatchers_ are in charge of selecting jobs scheduled to run in the future that are due and _dispatching_ them, which is simply moving them from the `solid_queue_scheduled_executions` table over to the `solid_queue_ready_executions` table so that workers can pick them up. They're also in charge of managing [recurring tasks](#recurring-tasks), dispatching jobs to process them according to their schedule. On top of that, they do some maintenance work related to [concurrency controls](#concurrency-controls).
-- The _supervisor_ forks workers and dispatchers according to the configuration, controls their heartbeats, and sends them signals to stop and start them when needed.
+- The _supervisor_ runs workers and dispatchers according to the configuration, controls their heartbeats, and stops and starts them when needed.
+
+By default, Solid Queue runs in `fork` mode. This means the supervisor will fork a separate process for each supervised worker/dispatcher. There's also an `async` mode where each worker and dispatcher will be run as a thread of the supervisor process. This can be used with [the provided Puma plugin](#puma-plugin)
 
 By default, Solid Queue will try to find your configuration under `config/solid_queue.yml`, but you can set a different path using the environment variable `SOLID_QUEUE_CONFIG`. This is what this configuration looks like:
 
@@ -98,7 +100,18 @@ production:
       processes: 3
 ```
 
-Everything is optional. If no configuration is provided, Solid Queue will run with one dispatcher and one worker with default settings.
+Everything is optional. If no configuration at all is provided, Solid Queue will run with one dispatcher and one worker with default settings. If you want to run only dispatchers or workers, you just need to include that section alone in the configuration. For example, with the following configuration:
+
+```yml
+production:
+  dispatchers:
+    - polling_interval: 1
+      batch_size: 500
+      concurrency_maintenance_interval: 300
+```
+the supervisor will run 1 dispatcher and no workers.
+
+Here's an overview of the different options:
 
 - `polling_interval`: the time interval in seconds that workers and dispatchers will wait before checking for more jobs. This time defaults to `1` second for dispatchers and `0.1` seconds for workers.
 - `batch_size`: the dispatcher will dispatch jobs in batches of this size. The default is 500.
@@ -118,7 +131,7 @@ Everything is optional. If no configuration is provided, Solid Queue will run wi
 
   Finally, you can combine prefixes with exact names, like `[ staging*, background ]`, and the behaviour with respect to order will be the same as with only exact names.
 - `threads`: this is the max size of the thread pool that each worker will have to run jobs. Each worker will fetch this number of jobs from their queue(s), at most and will post them to the thread pool to be run. By default, this is `3`. Only workers have this setting.
-- `processes`: this is the number of worker processes that will be forked by the supervisor with the settings given. By default, this is `1`, just a single process. This setting is useful if you want to dedicate more than one CPU core to a queue or queues with the same configuration. Only workers have this setting.
+- `processes`: this is the number of worker processes that will be forked by the supervisor with the settings given. By default, this is `1`, just a single process. This setting is useful if you want to dedicate more than one CPU core to a queue or queues with the same configuration. Only workers have this setting. **Note**: this option will be ignored if [running in `async` mode](#running-as-a-fork-or-asynchronously).
 - `concurrency_maintenance`: whether the dispatcher will perform the concurrency maintenance work. This is `true` by default, and it's useful if you don't use any [concurrency controls](#concurrency-controls) and want to disable it or if you run multiple dispatchers and want some of them to just dispatch jobs without doing anything else.
 - `recurring_tasks`: a list of recurring tasks the dispatcher will manage. Read more details about this one in the [Recurring tasks](#recurring-tasks) section.
 
@@ -232,7 +245,8 @@ class MyJob < ApplicationJob
   # ...
 ```
 - `key` is the only required parameter, and it can be a symbol, a string or a proc that receives the job arguments as parameters and will be used to identify the jobs that need to be limited together. If the proc returns an Active Record record, the key will be built from its class name and `id`.
-- `to` is `1` by default, and `duration` is set to `SolidQueue.default_concurrency_control_period` by default, which itself defaults to `3 minutes`, but that you can configure as well.
+- `to` is `1` by default.
+- `duration` is set to `SolidQueue.default_concurrency_control_period` by default, which itself defaults to `3 minutes`, but that you can configure as well.
 - `group` is used to control the concurrency of different job classes together. It defaults to the job class name.
 
 When a job includes these controls, we'll ensure that, at most, the number of jobs (indicated as `to`) that yield the same `key` will be performed concurrently, and this guarantee will last for `duration` for each job enqueued. Note that there's no guarantee about _the order of execution_, only about jobs being performed at the same time (overlapping).
@@ -291,6 +305,18 @@ plugin :solid_queue
 ```
 to your `puma.rb` configuration.
 
+### Running as a fork or asynchronously
+
+By default, the Puma plugin will fork additional processes for each worker and dispatcher so that they run in different processes. This provides the best isolation and performance, but can have additional memory usage.
+
+Alternatively, workers and dispatchers can be run within the same Puma process(s). To do so just configure the plugin as:
+
+```ruby
+plugin :solid_queue
+solid_queue_mode :async
+```
+
+Note that in this case, the `processes` configuration option will be ignored.
 
 ## Jobs and transactional integrity
 :warning: Having your jobs in the same ACID-compliant database as your application data enables a powerful yet sharp tool: taking advantage of transactional integrity to ensure some action in your app is not committed unless your job is also committed. This can be very powerful and useful, but it can also backfire if you base some of your logic on this behaviour, and in the future, you move to another active job backend, or if you simply move Solid Queue to its own database, and suddenly the behaviour changes under you.

data/app/models/solid_queue/claimed_execution.rb

@@ -15,9 +15,14 @@ class SolidQueue::ClaimedExecution < SolidQueue::Execution
     def claiming(job_ids, process_id, &block)
       job_data = Array(job_ids).collect { |job_id| { job_id: job_id, process_id: process_id } }
 
-      insert_all!(job_data)
-      where(job_id: job_ids, process_id: process_id).load.tap do |claimed|
-        block.call(claimed)
+      SolidQueue.instrument(:claim, process_id: process_id, job_ids: job_ids) do |payload|
+        insert_all!(job_data)
+        where(job_id: job_ids, process_id: process_id).load.tap do |claimed|
+          block.call(claimed)
+
+          payload[:size] = claimed.size
+          payload[:claimed_job_ids] = claimed.map(&:job_id)
+        end
       end
     end
 

data/app/models/solid_queue/process/executor.rb

@@ -0,0 +1,20 @@
+# frozen_string_literal: true
+
+module SolidQueue
+  class Process
+    module Executor
+      extend ActiveSupport::Concern
+
+      included do
+        has_many :claimed_executions
+
+        after_destroy -> { claimed_executions.release_all }, if: :claims_executions?
+      end
+
+      private
+        def claims_executions?
+          kind == "Worker"
+        end
+    end
+  end
+end

data/app/models/solid_queue/process/prunable.rb

@@ -1,19 +1,23 @@
 # frozen_string_literal: true
 
-module SolidQueue::Process::Prunable
-  extend ActiveSupport::Concern
+module SolidQueue
+  class Process
+    module Prunable
+      extend ActiveSupport::Concern
 
-  included do
-    scope :prunable, -> { where(last_heartbeat_at: ..SolidQueue.process_alive_threshold.ago) }
-  end
+      included do
+        scope :prunable, -> { where(last_heartbeat_at: ..SolidQueue.process_alive_threshold.ago) }
+      end
 
-  class_methods do
-    def prune
-      SolidQueue.instrument :prune_processes, size: 0 do |payload|
-        prunable.non_blocking_lock.find_in_batches(batch_size: 50) do |batch|
-          payload[:size] += batch.size
+      class_methods do
+        def prune
+          SolidQueue.instrument :prune_processes, size: 0 do |payload|
+            prunable.non_blocking_lock.find_in_batches(batch_size: 50) do |batch|
+              payload[:size] += batch.size
 
-          batch.each { |process| process.deregister(pruned: true) }
+              batch.each { |process| process.deregister(pruned: true) }
+            end
+          end
         end
       end
     end

data/app/models/solid_queue/process.rb

@@ -1,19 +1,18 @@
 # frozen_string_literal: true
 
 class SolidQueue::Process < SolidQueue::Record
-  include Prunable
+  include Executor, Prunable
 
-  belongs_to :supervisor, class_name: "SolidQueue::Process", optional: true, inverse_of: :forks
-  has_many :forks, class_name: "SolidQueue::Process", inverse_of: :supervisor, foreign_key: :supervisor_id, dependent: :destroy
-  has_many :claimed_executions
+  belongs_to :supervisor, class_name: "SolidQueue::Process", optional: true, inverse_of: :supervisees
+  has_many :supervisees, class_name: "SolidQueue::Process", inverse_of: :supervisor, foreign_key: :supervisor_id, dependent: :destroy
 
   store :metadata, coder: JSON
 
-  after_destroy -> { claimed_executions.release_all }
-
   def self.register(**attributes)
-    SolidQueue.instrument :register_process, **attributes do
-      create!(attributes.merge(last_heartbeat_at: Time.current))
+    SolidQueue.instrument :register_process, **attributes do |payload|
+      create!(attributes.merge(last_heartbeat_at: Time.current)).tap do |process|
+        payload[:process_id] = process.id
+      end
     end
   rescue Exception => error
     SolidQueue.instrument :register_process, **attributes.merge(error: error)
@@ -25,7 +24,9 @@ class SolidQueue::Process < SolidQueue::Record
   end
 
   def deregister(pruned: false)
-    SolidQueue.instrument :deregister_process, process: self, pruned: pruned, claimed_size: claimed_executions.size do |payload|
+    SolidQueue.instrument :deregister_process, process: self, pruned: pruned do |payload|
+      payload[:claimed_size] = claimed_executions.size if claims_executions?
+
       destroy!
     rescue Exception => error
       payload[:error] = error

data/lib/puma/plugin/solid_queue.rb

@@ -1,28 +1,50 @@
 require "puma/plugin"
 
+module Puma
+  class DSL
+    def solid_queue_mode(mode = :fork)
+      @options[:solid_queue_mode] = mode.to_sym
+    end
+  end
+end
+
 Puma::Plugin.create do
-  attr_reader :puma_pid, :solid_queue_pid, :log_writer
+  attr_reader :puma_pid, :solid_queue_pid, :log_writer, :solid_queue_supervisor
 
   def start(launcher)
     @log_writer = launcher.log_writer
     @puma_pid = $$
 
-    in_background do
-      monitor_solid_queue
+    if launcher.options[:solid_queue_mode] == :async
+      start_async(launcher)
+    else
+      start_forked(launcher)
     end
+  end
 
-    launcher.events.on_booted do
-      @solid_queue_pid = fork do
-        Thread.new { monitor_puma }
-        SolidQueue::Supervisor.start(mode: :all)
+  private
+    def start_forked(launcher)
+      in_background do
+        monitor_solid_queue
       end
+
+      launcher.events.on_booted do
+        @solid_queue_pid = fork do
+          Thread.new { monitor_puma }
+          SolidQueue::Supervisor.start(mode: :fork)
+        end
+      end
+
+      launcher.events.on_stopped { stop_solid_queue }
+      launcher.events.on_restart { stop_solid_queue }
     end
 
-    launcher.events.on_stopped { stop_solid_queue }
-    launcher.events.on_restart { stop_solid_queue }
-  end
+    def start_async(launcher)
+      launcher.events.on_booted { @solid_queue_supervisor = SolidQueue::Supervisor.start(mode: :async) }
+      launcher.events.on_stopped { solid_queue_supervisor.stop }
+      launcher.events.on_restart { solid_queue_supervisor.stop; solid_queue_supervisor.start }
+    end
 
-  private
     def stop_solid_queue
       Process.waitpid(solid_queue_pid, Process::WNOHANG)
       log "Stopping Solid Queue..."

data/lib/solid_queue/configuration.rb

@@ -17,38 +17,30 @@ module SolidQueue
       recurring_tasks: []
     }
 
-    def initialize(mode: :work, load_from: nil)
-      @mode = mode
+    def initialize(mode: :fork, load_from: nil)
+      @mode = mode.to_s.inquiry
       @raw_config = config_from(load_from)
     end
 
     def processes
-      case mode
-      when :dispatch then dispatchers
-      when :work     then workers
-      when :all      then dispatchers + workers
-      else           raise "Invalid mode #{mode}"
-      end
+      dispatchers + workers
     end
 
     def workers
-      if mode.in? %i[ work all]
-        workers_options.flat_map do |worker_options|
-          processes = worker_options.fetch(:processes, WORKER_DEFAULTS[:processes])
-          processes.times.map { Worker.new(**worker_options.with_defaults(WORKER_DEFAULTS)) }
+      workers_options.flat_map do |worker_options|
+        processes = if mode.fork?
+          worker_options.fetch(:processes, WORKER_DEFAULTS[:processes])
+        else
+          WORKER_DEFAULTS[:processes]
         end
-      else
-        []
+        processes.times.map { Worker.new(**worker_options.with_defaults(WORKER_DEFAULTS)) }
       end
     end
 
     def dispatchers
-      if mode.in? %i[ dispatch all]
-        dispatchers_options.map do |dispatcher_options|
-          recurring_tasks = parse_recurring_tasks dispatcher_options[:recurring_tasks]
-
-          Dispatcher.new **dispatcher_options.merge(recurring_tasks: recurring_tasks).with_defaults(DISPATCHER_DEFAULTS)
-        end
+      dispatchers_options.map do |dispatcher_options|
+        recurring_tasks = parse_recurring_tasks dispatcher_options[:recurring_tasks]
+        Dispatcher.new **dispatcher_options.merge(recurring_tasks: recurring_tasks).with_defaults(DISPATCHER_DEFAULTS)
       end
     end
 
@@ -68,15 +60,19 @@ module SolidQueue
       end
 
       def workers_options
-        @workers_options ||= (raw_config[:workers] || [ WORKER_DEFAULTS ])
+        @workers_options ||= options_from_raw_config(:workers, WORKER_DEFAULTS)
           .map { |options| options.dup.symbolize_keys }
       end
 
       def dispatchers_options
-        @dispatchers_options ||= (raw_config[:dispatchers] || [ DISPATCHER_DEFAULTS ])
+        @dispatchers_options ||= options_from_raw_config(:dispatchers, DISPATCHER_DEFAULTS)
           .map { |options| options.dup.symbolize_keys }
       end
 
+      def options_from_raw_config(key, defaults)
+        raw_config.empty? ? [ defaults ] : Array(raw_config[key])
+      end
+
       def parse_recurring_tasks(tasks)
         Array(tasks).map do |id, options|
           Dispatcher::RecurringTask.from_configuration(id, **options)

data/lib/solid_queue/dispatcher/concurrency_maintenance.rb

@@ -25,7 +25,7 @@ module SolidQueue
     end
 
     def stop
-      @concurrency_maintenance_task.shutdown
+      @concurrency_maintenance_task&.shutdown
     end
 
     private

data/lib/solid_queue/dispatcher.rb

@@ -1,9 +1,7 @@
 # frozen_string_literal: true
 
 module SolidQueue
-  class Dispatcher < Processes::Base
-    include Processes::Poller
-
+  class Dispatcher < Processes::Poller
     attr_accessor :batch_size, :concurrency_maintenance, :recurring_schedule
 
     after_boot :start_concurrency_maintenance, :load_recurring_schedule
@@ -13,10 +11,11 @@ module SolidQueue
       options = options.dup.with_defaults(SolidQueue::Configuration::DISPATCHER_DEFAULTS)
 
       @batch_size = options[:batch_size]
-      @polling_interval = options[:polling_interval]
 
       @concurrency_maintenance = ConcurrencyMaintenance.new(options[:concurrency_maintenance_interval], options[:batch_size]) if options[:concurrency_maintenance]
       @recurring_schedule = RecurringSchedule.new(options[:recurring_tasks])
+
+      super(**options)
     end
 
     def metadata

data/lib/solid_queue/log_subscriber.rb

@@ -7,6 +7,10 @@ class SolidQueue::LogSubscriber < ActiveSupport::LogSubscriber
     debug formatted_event(event, action: "Dispatch scheduled jobs", **event.payload.slice(:batch_size, :size))
   end
 
+  def claim(event)
+    debug formatted_event(event, action: "Claim jobs", **event.payload.slice(:process_id, :job_ids, :claimed_job_ids, :size))
+  end
+
   def release_many_claimed(event)
     debug formatted_event(event, action: "Release claimed jobs", **event.payload.slice(:size))
   end
@@ -40,19 +44,16 @@ class SolidQueue::LogSubscriber < ActiveSupport::LogSubscriber
   end
 
   def enqueue_recurring_task(event)
-    attributes = event.payload.slice(:task, :active_job_id, :enqueue_error, :at)
+    attributes = event.payload.slice(:task, :active_job_id, :enqueue_error)
+    attributes[:at] = event.payload[:at]&.iso8601
 
-    if event.payload[:other_adapter]
-      action = attributes[:active_job_id].present? ? "Enqueued recurring task outside Solid Queue" : "Error enqueuing recurring task"
-      info formatted_event(event, action: action, **attributes)
+    if attributes[:active_job_id].nil? && event.payload[:skipped].nil?
+      error formatted_event(event, action: "Error enqueuing recurring task", **attributes)
+    elsif event.payload[:other_adapter]
+      debug formatted_event(event, action: "Enqueued recurring task outside Solid Queue", **attributes)
     else
-      action = case
-      when event.payload[:skipped].present? then "Skipped recurring task – already dispatched"
-      when attributes[:active_job_id].nil?  then "Error enqueuing recurring task"
-      else                                       "Enqueued recurring task"
-      end
-
-      info formatted_event(event, action: action, **attributes)
+      action = event.payload[:skipped].present? ? "Skipped recurring task – already dispatched" : "Enqueued recurring task"
+      debug formatted_event(event, action: action, **attributes)
     end
   end
 
@@ -61,7 +62,8 @@ class SolidQueue::LogSubscriber < ActiveSupport::LogSubscriber
 
     attributes = {
       pid: process.pid,
-      hostname: process.hostname
+      hostname: process.hostname,
+      process_id: process.process_id
     }.merge(process.metadata)
 
     info formatted_event(event, action: "Started #{process.kind}", **attributes)
@@ -72,15 +74,16 @@ class SolidQueue::LogSubscriber < ActiveSupport::LogSubscriber
 
     attributes = {
       pid: process.pid,
-      hostname: process.hostname
+      hostname: process.hostname,
+      process_id: process.process_id
     }.merge(process.metadata)
 
-    info formatted_event(event, action: "Shut down #{process.kind}", **attributes)
+    info formatted_event(event, action: "Shutdown #{process.kind}", **attributes)
   end
 
   def register_process(event)
     process_kind = event.payload[:kind]
-    attributes = event.payload.slice(:pid, :hostname)
+    attributes = event.payload.slice(:pid, :hostname, :process_id)
 
     if error = event.payload[:error]
       warn formatted_event(event, action: "Error registering #{process_kind}", **attributes.merge(error: formatted_error(error)))
@@ -96,9 +99,9 @@ class SolidQueue::LogSubscriber < ActiveSupport::LogSubscriber
       process_id: process.id,
       pid: process.pid,
       hostname: process.hostname,
-      last_heartbeat_at: process.last_heartbeat_at,
-      claimed_size: process.claimed_executions.size,
-      pruned: event.payload
+      last_heartbeat_at: process.last_heartbeat_at.iso8601,
+      claimed_size: event.payload[:claimed_size],
+      pruned: event.payload[:pruned]
     }
 
     if error = event.payload[:error]
@@ -117,7 +120,7 @@ class SolidQueue::LogSubscriber < ActiveSupport::LogSubscriber
   end
 
   def graceful_termination(event)
-    attributes = event.payload.slice(:supervisor_pid, :supervised_pids)
+    attributes = event.payload.slice(:process_id, :supervisor_pid, :supervised_processes)
 
     if event.payload[:shutdown_timeout_exceeded]
       warn formatted_event(event, action: "Supervisor wasn't terminated gracefully - shutdown timeout exceeded", **attributes)
@@ -127,7 +130,7 @@ class SolidQueue::LogSubscriber < ActiveSupport::LogSubscriber
   end
 
   def immediate_termination(event)
-    info formatted_event(event, action: "Supervisor terminated immediately", **event.payload.slice(:supervisor_pid, :supervised_pids))
+    info formatted_event(event, action: "Supervisor terminated immediately", **event.payload.slice(:process_id, :supervisor_pid, :supervised_processes))
   end
 
   def unhandled_signal_error(event)

data/lib/solid_queue/processes/callbacks.rb

@@ -8,12 +8,5 @@ module SolidQueue::Processes
       extend ActiveModel::Callbacks
       define_model_callbacks :boot, :shutdown
     end
-
-    private
-      def boot
-      end
-
-      def shutdown
-      end
   end
 end

data/lib/solid_queue/processes/poller.rb

@@ -1,13 +1,13 @@
 # frozen_string_literal: true
 
 module SolidQueue::Processes
-  module Poller
-    extend ActiveSupport::Concern
-
+  class Poller < Base
     include Runnable
 
-    included do
-      attr_accessor :polling_interval
+    attr_accessor :polling_interval
+
+    def initialize(polling_interval:, **options)
+      @polling_interval = polling_interval
     end
 
     def metadata
@@ -16,11 +16,7 @@ module SolidQueue::Processes
 
     private
       def run
-        if mode.async?
-          @thread = Thread.new { start_loop }
-        else
-          start_loop
-        end
+        start_loop
       end
 
       def start_loop
@@ -43,6 +39,9 @@ module SolidQueue::Processes
         raise NotImplementedError
       end
 
+      def shutdown
+      end
+
       def with_polling_volume
         if SolidQueue.silence_polling? && ActiveRecord::Base.logger
           ActiveRecord::Base.logger.silence { yield }

data/lib/solid_queue/processes/procline.rb

@@ -5,7 +5,7 @@ module SolidQueue::Processes
     # Sets the procline ($0)
     # solid-queue-supervisor(0.1.0): <string>
     def procline(string)
-      $0 = "solid-queue-#{self.class.name.demodulize.downcase}(#{SolidQueue::VERSION}): #{string}"
+      $0 = "solid-queue-#{self.class.name.demodulize.underscore.dasherize}(#{SolidQueue::VERSION}): #{string}"
     end
   end
 end

data/lib/solid_queue/processes/registrable.rb

@@ -11,12 +11,16 @@ module SolidQueue::Processes
       after_shutdown :deregister
     end
 
+    def process_id
+      process&.id
+    end
+
     private
       attr_accessor :process
 
       def register
         @process = SolidQueue::Process.register \
-          kind: self.class.name.demodulize,
+          kind: kind,
           pid: pid,
           hostname: hostname,
           supervisor: try(:supervisor),
@@ -36,6 +40,10 @@ module SolidQueue::Processes
           wrap_in_app_executor { heartbeat }
         end
 
+        @heartbeat_task.add_observer do |_, _, error|
+          handle_thread_error(error) if error
+        end
+
         @heartbeat_task.execute
       end
 

data/lib/solid_queue/processes/runnable.rb

@@ -7,20 +7,32 @@ module SolidQueue::Processes
     attr_writer :mode
 
     def start
-      @stopping = false
+      @stopped = false
 
       SolidQueue.instrument(:start_process, process: self) do
         run_callbacks(:boot) { boot }
       end
 
-      run
+      if running_async?
+        @thread = create_thread { run }
+      else
+        run
+      end
     end
 
     def stop
-      @stopping = true
+      @stopped = true
       @thread&.join
     end
 
+    def name
+      @name ||= [ kind.downcase, SecureRandom.hex(6) ].join("-")
+    end
+
+    def alive?
+      !running_async? || @thread.alive?
+    end
+
     private
       DEFAULT_MODE = :async
 
@@ -29,22 +41,22 @@ module SolidQueue::Processes
       end
 
       def boot
-        if supervised?
+        if running_as_fork?
           register_signal_handlers
           set_procline
         end
       end
 
       def shutting_down?
-        stopping? || supervisor_went_away? || finished?
+        stopped? || (running_as_fork? && supervisor_went_away?) || finished?
       end
 
       def run
         raise NotImplementedError
       end
 
-      def stopping?
-        @stopping
+      def stopped?
+        @stopped
       end
 
       def finished?
@@ -61,5 +73,24 @@ module SolidQueue::Processes
       def running_inline?
         mode.inline?
       end
+
+      def running_async?
+        mode.async?
+      end
+
+      def running_as_fork?
+        mode.fork?
+      end
+
+
+      def create_thread(&block)
+        Thread.new do
+          Thread.current.name = name
+          block.call
+        rescue Exception => exception
+          handle_thread_error(exception)
+          raise
+        end
+      end
   end
 end