solid_queue 0.3.3 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 50d118a41d39dff4da741a1e7fb105b47571dd719b883fb53843e7b57f7aff66
-  data.tar.gz: a1dc7ff9b71a07f41fab851ea0dd2f16170240e2a465098f98e75a96c33edb4b
+  metadata.gz: e12cccf2a1485f92c30925675e6a91bb36b063057390e57250a582440a046c8a
+  data.tar.gz: a1c857b509ff15124eed754cb8ed8613d95764f97e2f06758baf4031ea10d387
 SHA512:
-  metadata.gz: f344e6ba1a395f014fc08437c1322eac9b3f5218874f4b824cacab042d20053ee9a5504601e37864e04ccc956fc1f08fe35979c9d4a7b19dc42cc53ae5502c56
-  data.tar.gz: 27e1d8c00e2c68330d52e938bb00ecadaf246678e16b12b0e1269f8a4e8e3328a972b5a007a04d2839baf2edb480e9f0cae3b6099cd6009a75b11e85858ee6e2
+  metadata.gz: bf861243ec274d583b29222275303a3cb8383803291e911f8d2872ba110d92705978d31ceada27e246368410668fc7f2ba8a9bb368c19cdfff666ed74636ea3c
+  data.tar.gz: a6ccfc868515c105f67bbcbdcb9cda4b4ea790e77a61d16e8dd310e6e68244791f1fb2a0e693cec5c2af0c2ba93056b078269d97673884c4e80ade876d48fcc5

data/README.md

@@ -66,7 +66,7 @@ $ bundle exec rake solid_queue:start
 
 This will start processing jobs in all queues using the default configuration. See [below](#configuration) to learn more about configuring Solid Queue.
 
-For small projects, you can run Solid Queue on the same machine as your webserver. When you're ready to scale, Solid Queue supports horizontal scaling out-of-the-box. You can run Solid Queue on a separate server from your webserver, or even run `bundle exec rake solid_queue:start` on multiple machines at the same time. If you'd like to designate some machines to be only dispatchers or only workers, use `bundle exec rake solid_queue:dispatch` or `bundle exec rake solid_queue:work`, respectively.
+For small projects, you can run Solid Queue on the same machine as your webserver. When you're ready to scale, Solid Queue supports horizontal scaling out-of-the-box. You can run Solid Queue on a separate server from your webserver, or even run `bundle exec rake solid_queue:start` on multiple machines at the same time. Depending on the configuration, you can designate some machines to run only dispatchers or only workers. See the [configuration](#configuration) section for more details on this.
 
 ## Requirements
 Besides Rails 7.1, Solid Queue works best with MySQL 8+ or PostgreSQL 9.5+, as they support `FOR UPDATE SKIP LOCKED`. You can use it with older versions, but in that case, you might run into lock waits if you run multiple workers for the same queue.
@@ -75,10 +75,12 @@ Besides Rails 7.1, Solid Queue works best with MySQL 8+ or PostgreSQL 9.5+, as t
 
 ### Workers and dispatchers
 
-We have three types of processes in Solid Queue:
+We have three types of actors in Solid Queue:
 - _Workers_ are in charge of picking jobs ready to run from queues and processing them. They work off the `solid_queue_ready_executions` table.
 - _Dispatchers_ are in charge of selecting jobs scheduled to run in the future that are due and _dispatching_ them, which is simply moving them from the `solid_queue_scheduled_executions` table over to the `solid_queue_ready_executions` table so that workers can pick them up. They're also in charge of managing [recurring tasks](#recurring-tasks), dispatching jobs to process them according to their schedule. On top of that, they do some maintenance work related to [concurrency controls](#concurrency-controls).
-- The _supervisor_ forks workers and dispatchers according to the configuration, controls their heartbeats, and sends them signals to stop and start them when needed.
+- The _supervisor_ runs workers and dispatchers according to the configuration, controls their heartbeats, and stops and starts them when needed.
+
+By default, Solid Queue runs in `fork` mode. This means the supervisor will fork a separate process for each supervised worker/dispatcher. There's also an `async` mode where each worker and dispatcher will be run as a thread of the supervisor process. This can be used with [the provided Puma plugin](#puma-plugin)
 
 By default, Solid Queue will try to find your configuration under `config/solid_queue.yml`, but you can set a different path using the environment variable `SOLID_QUEUE_CONFIG`. This is what this configuration looks like:
 
@@ -98,7 +100,18 @@ production:
       processes: 3
 ```
 
-Everything is optional. If no configuration is provided, Solid Queue will run with one dispatcher and one worker with default settings.
+Everything is optional. If no configuration at all is provided, Solid Queue will run with one dispatcher and one worker with default settings. If you want to run only dispatchers or workers, you just need to include that section alone in the configuration. For example, with the following configuration:
+
+```yml
+production:
+  dispatchers:
+    - polling_interval: 1
+      batch_size: 500
+      concurrency_maintenance_interval: 300
+```
+the supervisor will run 1 dispatcher and no workers.
+
+Here's an overview of the different options:
 
 - `polling_interval`: the time interval in seconds that workers and dispatchers will wait before checking for more jobs. This time defaults to `1` second for dispatchers and `0.1` seconds for workers.
 - `batch_size`: the dispatcher will dispatch jobs in batches of this size. The default is 500.
@@ -118,7 +131,7 @@ Everything is optional. If no configuration is provided, Solid Queue will run wi
 
   Finally, you can combine prefixes with exact names, like `[ staging*, background ]`, and the behaviour with respect to order will be the same as with only exact names.
 - `threads`: this is the max size of the thread pool that each worker will have to run jobs. Each worker will fetch this number of jobs from their queue(s), at most and will post them to the thread pool to be run. By default, this is `3`. Only workers have this setting.
-- `processes`: this is the number of worker processes that will be forked by the supervisor with the settings given. By default, this is `1`, just a single process. This setting is useful if you want to dedicate more than one CPU core to a queue or queues with the same configuration. Only workers have this setting.
+- `processes`: this is the number of worker processes that will be forked by the supervisor with the settings given. By default, this is `1`, just a single process. This setting is useful if you want to dedicate more than one CPU core to a queue or queues with the same configuration. Only workers have this setting. **Note**: this option will be ignored if [running in `async` mode](#running-as-a-fork-or-asynchronously).
 - `concurrency_maintenance`: whether the dispatcher will perform the concurrency maintenance work. This is `true` by default, and it's useful if you don't use any [concurrency controls](#concurrency-controls) and want to disable it or if you run multiple dispatchers and want some of them to just dispatch jobs without doing anything else.
 - `recurring_tasks`: a list of recurring tasks the dispatcher will manage. Read more details about this one in the [Recurring tasks](#recurring-tasks) section.
 
@@ -145,6 +158,56 @@ When receiving a `QUIT` signal, if workers still have jobs in-flight, these will
 
 If processes have no chance of cleaning up before exiting (e.g. if someone pulls a cable somewhere), in-flight jobs might remain claimed by the processes executing them. Processes send heartbeats, and the supervisor checks and prunes processes with expired heartbeats, which will release any claimed jobs back to their queues. You can configure both the frequency of heartbeats and the threshold to consider a process dead. See the section below for this.
 
+
+### Dedicated database configuration
+
+Solid Queue can be configured to run on a different database than the main application.
+
+Configure the `connects_to` option in `config/application.rb` or your environment config, with the custom database configuration that will be used in the abstract `SolidQueue::Record` Active Record model.
+
+```ruby
+  # Use a separate DB for Solid Queue
+  config.solid_queue.connects_to = { database: { writing: :solid_queue_primary, reading: :solid_queue_replica } }
+  ```
+
+Add the dedicated database configuration to `config/database.yml`, differentiating between the main app's database and the dedicated `solid_queue` database. Make sure to include the `migrations_paths` for the solid queue database. This is where migration files for Solid Queue tables will reside.
+
+```yml
+default: &default
+  adapter: sqlite3
+  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
+  timeout: 5000
+
+solid_queue: &solid_queue
+  <<: *default
+  migrations_paths: db/solid_queue_migrate
+
+development:
+  primary:
+    <<: *default
+    # ...
+  solid_queue_primary:
+    <<: *solid_queue
+    # ...
+  solid_queue_replica:
+    <<: *solid_queue
+    # ...
+```
+
+Install migrations and specify the dedicated database name with the `DATABASE` option. This will create the Solid Queue migration files in a separate directory, matching the value provided in `migrations_paths` in `config/database.yml`.
+
+```bash
+$ bin/rails solid_queue:install:migrations DATABASE=solid_queue
+```
+
+Note: If you've already run the solid queue install command (`bin/rails generate solid_queue:install`), the migration files will have already been generated under the primary database's `db/migrate/` directory. You can remove these files and keep the ones generated by the database-specific migration installation above.
+
+Finally, run the migrations:
+
+```bash
+$ bin/rails db:migrate
+```
+
 ### Other configuration settings
 _Note_: The settings in this section should be set in your `config/application.rb` or your environment config like this: `config.solid_queue.silence_polling = true`
 
@@ -156,12 +219,6 @@ There are several settings that control how Solid Queue works that you can set a
   ```ruby
   -> (exception) { Rails.error.report(exception, handled: false) }
   ```
-- `connects_to`: a custom database configuration that will be used in the abstract `SolidQueue::Record` Active Record model. This is required to use a different database than the main app. For example:
-
-  ```ruby
-  # Use a separate DB for Solid Queue
-  config.solid_queue.connects_to = { database: { writing: :solid_queue_primary, reading: :solid_queue_replica } }
-  ```
 - `use_skip_locked`: whether to use `FOR UPDATE SKIP LOCKED` when performing locking reads. This will be automatically detected in the future, and for now, you'd only need to set this to `false` if your database doesn't support it. For MySQL, that'd be versions < 8, and for PostgreSQL, versions < 9.5. If you use SQLite, this has no effect, as writes are sequential.
 - `process_heartbeat_interval`: the heartbeat interval that all processes will follow—defaults to 60 seconds.
 - `process_alive_threshold`: how long to wait until a process is considered dead after its last heartbeat—defaults to 5 minutes.
@@ -173,6 +230,10 @@ There are several settings that control how Solid Queue works that you can set a
 - `default_concurrency_control_period`: the value to be used as the default for the `duration` parameter in [concurrency controls](#concurrency-controls). It defaults to 3 minutes.
 - `enqueue_after_transaction_commit`: whether the job queuing is deferred to after the current Active Record transaction is committed. The default is `false`. [Read more](https://github.com/rails/rails/pull/51426).
 
+## Errors when enqueuing
+Solid Queue will raise a `SolidQueue::Job::EnqueueError` for any Active Record errors that happen when enqueuing a job. The reason for not raising `ActiveJob::EnqueueError` is that this one gets handled by Active Job, causing `perform_later` to return `false` and set `job.enqueue_error`, yielding the job to a block that you need to pass to `perform_later`. This works very well for your own jobs, but makes failure very hard to handle for jobs enqueued by Rails or other gems, such as `Turbo::Streams::BroadcastJob` or `ActiveStorage::AnalyzeJob`, because you don't control the call to `perform_later` in that cases.
+
+In the case of recurring tasks, if such error is raised when enqueuing the job corresponding to the task, it'll be handled and logged but it won't bubble up.
 
 ## Concurrency controls
 Solid Queue extends Active Job with concurrency controls, that allows you to limit how many jobs of a certain type or with certain arguments can run at the same time. When limited in this way, jobs will be blocked from running, and they'll stay blocked until another job finishes and unblocks them, or after the set expiry time (concurrency limit's _duration_) elapses. Jobs are never discarded or lost, only blocked.
@@ -184,7 +245,8 @@ class MyJob < ApplicationJob
   # ...
 ```
 - `key` is the only required parameter, and it can be a symbol, a string or a proc that receives the job arguments as parameters and will be used to identify the jobs that need to be limited together. If the proc returns an Active Record record, the key will be built from its class name and `id`.
-- `to` is `1` by default, and `duration` is set to `SolidQueue.default_concurrency_control_period` by default, which itself defaults to `3 minutes`, but that you can configure as well.
+- `to` is `1` by default.
+- `duration` is set to `SolidQueue.default_concurrency_control_period` by default, which itself defaults to `3 minutes`, but that you can configure as well.
 - `group` is used to control the concurrency of different job classes together. It defaults to the job class name.
 
 When a job includes these controls, we'll ensure that, at most, the number of jobs (indicated as `to`) that yield the same `key` will be performed concurrently, and this guarantee will last for `duration` for each job enqueued. Note that there's no guarantee about _the order of execution_, only about jobs being performed at the same time (overlapping).
@@ -243,6 +305,18 @@ plugin :solid_queue
 ```
 to your `puma.rb` configuration.
 
+### Running as a fork or asynchronously
+
+By default, the Puma plugin will fork additional processes for each worker and dispatcher so that they run in different processes. This provides the best isolation and performance, but can have additional memory usage.
+
+Alternatively, workers and dispatchers can be run within the same Puma process(s). To do so just configure the plugin as:
+
+```ruby
+plugin :solid_queue
+solid_queue_mode :async
+```
+
+Note that in this case, the `processes` configuration option will be ignored.
 
 ## Jobs and transactional integrity
 :warning: Having your jobs in the same ACID-compliant database as your application data enables a powerful yet sharp tool: taking advantage of transactional integrity to ensure some action in your app is not committed unless your job is also committed. This can be very powerful and useful, but it can also backfire if you base some of your logic on this behaviour, and in the future, you move to another active job backend, or if you simply move Solid Queue to its own database, and suddenly the behaviour changes under you.
@@ -252,7 +326,7 @@ By default, Solid Queue runs in the same DB as your app, and job enqueuing is _n
 If you prefer not to rely on this, or avoid relying on it unintentionally, you should make sure that:
 - You set [`config.active_job.enqueue_after_transaction_commit`](https://edgeguides.rubyonrails.org/configuring.html#config-active-job-enqueue-after-transaction-commit) to `always`, if you're using Rails 7.2+.
 - Or, your jobs relying on specific records are always enqueued on [`after_commit` callbacks](https://guides.rubyonrails.org/active_record_callbacks.html#after-commit-and-after-rollback) or otherwise from a place where you're certain that whatever data the job will use has been committed to the database before the job is enqueued.
-- Or, you configure a database for Solid Queue, even if it's the same as your app, ensuring that a different connection on the thread handling requests or running jobs for your app will be used to enqueue jobs. For example:
+- Or, you configure a different database for Solid Queue, even if it's the same as your app, ensuring that a different connection on the thread handling requests or running jobs for your app will be used to enqueue jobs. For example:
 
   ```ruby
   class ApplicationRecord < ActiveRecord::Base
@@ -288,7 +362,7 @@ Tasks are enqueued at their corresponding times by the dispatcher that owns them
 
 It's possible to run multiple dispatchers with the same `recurring_tasks` configuration. To avoid enqueuing duplicate tasks at the same time, an entry in a new `solid_queue_recurring_executions` table is created in the same transaction as the job is enqueued. This table has a unique index on `task_key` and `run_at`, ensuring only one entry per task per time will be created. This only works if you have `preserve_finished_jobs` set to `true` (the default), and the guarantee applies as long as you keep the jobs around.
 
-Finally, it's possible to configure jobs that aren't handled by Solid Queue. That's it, you can a have a job like this in your app:
+Finally, it's possible to configure jobs that aren't handled by Solid Queue. That is, you can have a job like this in your app:
 ```ruby
 class MyResqueJob < ApplicationJob
   self.queue_adapter = :resque

data/app/models/solid_queue/claimed_execution.rb

@@ -3,6 +3,8 @@
 class SolidQueue::ClaimedExecution < SolidQueue::Execution
   belongs_to :process
 
+  scope :orphaned, -> { where.missing(:process) }
+
   class Result < Struct.new(:success, :error)
     def success?
       success
@@ -13,9 +15,14 @@ class SolidQueue::ClaimedExecution < SolidQueue::Execution
     def claiming(job_ids, process_id, &block)
       job_data = Array(job_ids).collect { |job_id| { job_id: job_id, process_id: process_id } }
 
-      insert_all!(job_data)
-      where(job_id: job_ids, process_id: process_id).load.tap do |claimed|
-        block.call(claimed)
+      SolidQueue.instrument(:claim, process_id: process_id, job_ids: job_ids) do |payload|
+        insert_all!(job_data)
+        where(job_id: job_ids, process_id: process_id).load.tap do |claimed|
+          block.call(claimed)
+
+          payload[:size] = claimed.size
+          payload[:claimed_job_ids] = claimed.map(&:job_id)
+        end
       end
     end
 

data/app/models/solid_queue/failed_execution.rb

@@ -33,9 +33,45 @@ module SolidQueue
     end
 
     private
+      JSON_OVERHEAD = 256
+
       def expand_error_details_from_exception
         if exception
-          self.error = { exception_class: exception.class.name, message: exception.message, backtrace: exception.backtrace }
+          self.error = { exception_class: exception_class_name, message: exception_message, backtrace: exception_backtrace }
+        end
+      end
+
+      def exception_class_name
+        exception.class.name
+      end
+
+      def exception_message
+        exception.message
+      end
+
+      def exception_backtrace
+        if (limit = determine_backtrace_size_limit) && exception.backtrace.to_json.bytesize > limit
+          truncate_backtrace(exception.backtrace, limit)
+        else
+          exception.backtrace
+        end
+      end
+
+      def determine_backtrace_size_limit
+        column = self.class.connection.schema_cache.columns_hash(self.class.table_name)["error"]
+        if column.limit.present?
+          column.limit - exception_class_name.bytesize - exception_message.bytesize - JSON_OVERHEAD
+        end
+      end
+
+      def truncate_backtrace(lines, limit)
+        [].tap do |truncated_backtrace|
+          lines.each do |line|
+            if (truncated_backtrace << line).to_json.bytesize > limit
+              truncated_backtrace.pop
+              break
+            end
+          end
         end
       end
   end

data/app/models/solid_queue/job.rb

@@ -2,6 +2,8 @@
 
 module SolidQueue
   class Job < Record
+    class EnqueueError < StandardError; end
+
     include Executable, Clearable, Recurrable
 
     serialize :arguments, coder: JSON
@@ -37,6 +39,11 @@ module SolidQueue
 
         def create_from_active_job(active_job)
           create!(**attributes_from_active_job(active_job))
+        rescue ActiveRecord::ActiveRecordError => e
+          enqueue_error = EnqueueError.new("#{e.class.name}: #{e.message}").tap do |error|
+            error.set_backtrace e.backtrace
+          end
+          raise enqueue_error
         end
 
         def create_all_from_active_jobs(active_jobs)

data/app/models/solid_queue/process/executor.rb

@@ -0,0 +1,20 @@
+# frozen_string_literal: true
+
+module SolidQueue
+  class Process
+    module Executor
+      extend ActiveSupport::Concern
+
+      included do
+        has_many :claimed_executions
+
+        after_destroy -> { claimed_executions.release_all }, if: :claims_executions?
+      end
+
+      private
+        def claims_executions?
+          kind == "Worker"
+        end
+    end
+  end
+end

data/app/models/solid_queue/process/prunable.rb

@@ -1,19 +1,23 @@
 # frozen_string_literal: true
 
-module SolidQueue::Process::Prunable
-  extend ActiveSupport::Concern
+module SolidQueue
+  class Process
+    module Prunable
+      extend ActiveSupport::Concern
 
-  included do
-    scope :prunable, -> { where(last_heartbeat_at: ..SolidQueue.process_alive_threshold.ago) }
-  end
+      included do
+        scope :prunable, -> { where(last_heartbeat_at: ..SolidQueue.process_alive_threshold.ago) }
+      end
 
-  class_methods do
-    def prune
-      SolidQueue.instrument :prune_processes, size: 0 do |payload|
-        prunable.non_blocking_lock.find_in_batches(batch_size: 50) do |batch|
-          payload[:size] += batch.size
+      class_methods do
+        def prune
+          SolidQueue.instrument :prune_processes, size: 0 do |payload|
+            prunable.non_blocking_lock.find_in_batches(batch_size: 50) do |batch|
+              payload[:size] += batch.size
 
-          batch.each { |process| process.deregister(pruned: true) }
+              batch.each { |process| process.deregister(pruned: true) }
+            end
+          end
         end
       end
     end

data/app/models/solid_queue/process.rb

@@ -1,19 +1,18 @@
 # frozen_string_literal: true
 
 class SolidQueue::Process < SolidQueue::Record
-  include Prunable
+  include Executor, Prunable
 
-  belongs_to :supervisor, class_name: "SolidQueue::Process", optional: true, inverse_of: :forks
-  has_many :forks, class_name: "SolidQueue::Process", inverse_of: :supervisor, foreign_key: :supervisor_id, dependent: :destroy
-  has_many :claimed_executions
+  belongs_to :supervisor, class_name: "SolidQueue::Process", optional: true, inverse_of: :supervisees
+  has_many :supervisees, class_name: "SolidQueue::Process", inverse_of: :supervisor, foreign_key: :supervisor_id, dependent: :destroy
 
   store :metadata, coder: JSON
 
-  after_destroy -> { claimed_executions.release_all }
-
   def self.register(**attributes)
-    SolidQueue.instrument :register_process, **attributes do
-      create!(attributes.merge(last_heartbeat_at: Time.current))
+    SolidQueue.instrument :register_process, **attributes do |payload|
+      create!(attributes.merge(last_heartbeat_at: Time.current)).tap do |process|
+        payload[:process_id] = process.id
+      end
     end
   rescue Exception => error
     SolidQueue.instrument :register_process, **attributes.merge(error: error)
@@ -25,7 +24,9 @@ class SolidQueue::Process < SolidQueue::Record
   end
 
   def deregister(pruned: false)
-    SolidQueue.instrument :deregister_process, process: self, pruned: pruned, claimed_size: claimed_executions.size do |payload|
+    SolidQueue.instrument :deregister_process, process: self, pruned: pruned do |payload|
+      payload[:claimed_size] = claimed_executions.size if claims_executions?
+
       destroy!
     rescue Exception => error
       payload[:error] = error

data/app/models/solid_queue/recurring_execution.rb

@@ -2,17 +2,21 @@
 
 module SolidQueue
   class RecurringExecution < Execution
+    class AlreadyRecorded < StandardError; end
+
     scope :clearable, -> { where.missing(:job) }
 
     class << self
       def record(task_key, run_at, &block)
         transaction do
           block.call.tap do |active_job|
-            create!(job_id: active_job.provider_job_id, task_key: task_key, run_at: run_at)
+            if active_job
+              create!(job_id: active_job.provider_job_id, task_key: task_key, run_at: run_at)
+            end
           end
         end
-      rescue ActiveRecord::RecordNotUnique
-        # Task already dispatched
+      rescue ActiveRecord::RecordNotUnique => e
+        raise AlreadyRecorded
       end
 
       def clear_in_batches(batch_size: 500)

data/lib/puma/plugin/solid_queue.rb

@@ -1,28 +1,50 @@
 require "puma/plugin"
 
+module Puma
+  class DSL
+    def solid_queue_mode(mode = :fork)
+      @options[:solid_queue_mode] = mode.to_sym
+    end
+  end
+end
+
 Puma::Plugin.create do
-  attr_reader :puma_pid, :solid_queue_pid, :log_writer
+  attr_reader :puma_pid, :solid_queue_pid, :log_writer, :solid_queue_supervisor
 
   def start(launcher)
     @log_writer = launcher.log_writer
     @puma_pid = $$
 
-    launcher.events.on_booted do
-      @solid_queue_pid = fork do
-        Thread.new { monitor_puma }
-        SolidQueue::Supervisor.start(mode: :all)
-      end
+    if launcher.options[:solid_queue_mode] == :async
+      start_async(launcher)
+    else
+      start_forked(launcher)
+    end
+  end
 
+  private
+    def start_forked(launcher)
       in_background do
         monitor_solid_queue
       end
+
+      launcher.events.on_booted do
+        @solid_queue_pid = fork do
+          Thread.new { monitor_puma }
+          SolidQueue::Supervisor.start(mode: :fork)
+        end
+      end
+
+      launcher.events.on_stopped { stop_solid_queue }
+      launcher.events.on_restart { stop_solid_queue }
     end
 
-    launcher.events.on_stopped { stop_solid_queue }
-    launcher.events.on_restart { stop_solid_queue }
-  end
+    def start_async(launcher)
+      launcher.events.on_booted { @solid_queue_supervisor = SolidQueue::Supervisor.start(mode: :async) }
+      launcher.events.on_stopped { solid_queue_supervisor.stop }
+      launcher.events.on_restart { solid_queue_supervisor.stop; solid_queue_supervisor.start }
+    end
 
-  private
     def stop_solid_queue
       Process.waitpid(solid_queue_pid, Process::WNOHANG)
       log "Stopping Solid Queue..."
@@ -51,12 +73,18 @@ Puma::Plugin.create do
     end
 
     def solid_queue_dead?
-      Process.waitpid(solid_queue_pid, Process::WNOHANG)
+      if solid_queue_started?
+        Process.waitpid(solid_queue_pid, Process::WNOHANG)
+      end
       false
     rescue Errno::ECHILD, Errno::ESRCH
       true
     end
 
+    def solid_queue_started?
+      solid_queue_pid.present?
+    end
+
     def puma_dead?
       Process.ppid != puma_pid
     end

data/lib/solid_queue/configuration.rb

@@ -17,38 +17,30 @@ module SolidQueue
       recurring_tasks: []
     }
 
-    def initialize(mode: :work, load_from: nil)
-      @mode = mode
+    def initialize(mode: :fork, load_from: nil)
+      @mode = mode.to_s.inquiry
       @raw_config = config_from(load_from)
     end
 
     def processes
-      case mode
-      when :dispatch then dispatchers
-      when :work     then workers
-      when :all      then dispatchers + workers
-      else           raise "Invalid mode #{mode}"
-      end
+      dispatchers + workers
     end
 
     def workers
-      if mode.in? %i[ work all]
-        workers_options.flat_map do |worker_options|
-          processes = worker_options.fetch(:processes, WORKER_DEFAULTS[:processes])
-          processes.times.map { Worker.new(**worker_options.with_defaults(WORKER_DEFAULTS)) }
+      workers_options.flat_map do |worker_options|
+        processes = if mode.fork?
+          worker_options.fetch(:processes, WORKER_DEFAULTS[:processes])
+        else
+          WORKER_DEFAULTS[:processes]
         end
-      else
-        []
+        processes.times.map { Worker.new(**worker_options.with_defaults(WORKER_DEFAULTS)) }
       end
     end
 
     def dispatchers
-      if mode.in? %i[ dispatch all]
-        dispatchers_options.map do |dispatcher_options|
-          recurring_tasks = parse_recurring_tasks dispatcher_options[:recurring_tasks]
-
-          Dispatcher.new **dispatcher_options.merge(recurring_tasks: recurring_tasks).with_defaults(DISPATCHER_DEFAULTS)
-        end
+      dispatchers_options.map do |dispatcher_options|
+        recurring_tasks = parse_recurring_tasks dispatcher_options[:recurring_tasks]
+        Dispatcher.new **dispatcher_options.merge(recurring_tasks: recurring_tasks).with_defaults(DISPATCHER_DEFAULTS)
       end
     end
 
@@ -68,15 +60,19 @@ module SolidQueue
       end
 
       def workers_options
-        @workers_options ||= (raw_config[:workers] || [ WORKER_DEFAULTS ])
+        @workers_options ||= options_from_raw_config(:workers, WORKER_DEFAULTS)
           .map { |options| options.dup.symbolize_keys }
       end
 
       def dispatchers_options
-        @dispatchers_options ||= (raw_config[:dispatchers] || [ DISPATCHER_DEFAULTS ])
+        @dispatchers_options ||= options_from_raw_config(:dispatchers, DISPATCHER_DEFAULTS)
           .map { |options| options.dup.symbolize_keys }
       end
 
+      def options_from_raw_config(key, defaults)
+        raw_config.empty? ? [ defaults ] : Array(raw_config[key])
+      end
+
       def parse_recurring_tasks(tasks)
         Array(tasks).map do |id, options|
           Dispatcher::RecurringTask.from_configuration(id, **options)

data/lib/solid_queue/dispatcher/concurrency_maintenance.rb

@@ -25,7 +25,7 @@ module SolidQueue
     end
 
     def stop
-      @concurrency_maintenance_task.shutdown
+      @concurrency_maintenance_task&.shutdown
     end
 
     private

data/lib/solid_queue/dispatcher/recurring_schedule.rb

@@ -11,6 +11,10 @@ module SolidQueue
       @scheduled_tasks = Concurrent::Hash.new
     end
 
+    def empty?
+      configured_tasks.empty?
+    end
+
     def load_tasks
       configured_tasks.each do |task|
         load_task(task)

data/lib/solid_queue/dispatcher/recurring_task.rb

@@ -31,15 +31,23 @@ module SolidQueue
 
     def enqueue(at:)
       SolidQueue.instrument(:enqueue_recurring_task, task: key, at: at) do |payload|
-        if using_solid_queue_adapter?
+        active_job = if using_solid_queue_adapter?
           perform_later_and_record(run_at: at)
         else
           payload[:other_adapter] = true
 
-          perform_later
-        end.tap do |active_job|
-          payload[:active_job_id] = active_job&.job_id
+          perform_later do |job|
+            unless job.successfully_enqueued?
+              payload[:enqueue_error] = job.enqueue_error&.message
+            end
+          end
         end
+
+        payload[:active_job_id] = active_job.job_id if active_job
+      rescue RecurringExecution::AlreadyRecorded
+        payload[:skipped] = true
+      rescue Job::EnqueueError => error
+        payload[:enqueue_error] = error.message
       end
     end
 
@@ -68,8 +76,8 @@ module SolidQueue
         RecurringExecution.record(key, run_at) { perform_later }
       end
 
-      def perform_later
-        job_class.perform_later(*arguments_with_kwargs)
+      def perform_later(&block)
+        job_class.perform_later(*arguments_with_kwargs, &block)
       end
 
       def arguments_with_kwargs