solid_queue 1.4.0 → 1.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: fd0590f46160c60f3a496158cf8dc2412803025cd06d94ac423c32f0b688dd77
-  data.tar.gz: 8c892f457280b1974908d2de0c3e5be5229ff6bcb448ed61c2937ff4566da796
+  metadata.gz: d6cda6edb92a7805d6c2c0fc13b2649c2c2df26a4d65e555049efde022cb8d14
+  data.tar.gz: c6d0036cbeb743e56faa4ed9fe6756f1a8e3ae3705a402f55440bb14cb3174dc
 SHA512:
-  metadata.gz: 0a86b46d35b0cc8aef4a235e401b4470a6f6e64ff8c44ebdefc06f597d6cbe67ea76c786fc1e85caee1c3fc4dd492180530348078890ef74af90461705150a60
-  data.tar.gz: efeadbeb7dc0d044c801915d6ba4d8c249b249ef0dde8726454a359d3f1a4ce9ad4e2edb6cd19f8c371967059081c0ef2904ea630a0ae620123939ce881f0fdb
+  metadata.gz: 5f7420684f8aa122d12e2314aa2445fb31a3f18f2a7bc53b4ceffccccd131a7f6d9408ef776da33dbbfe437be4c835437524279ac6150e57d134cf85698caf7a
+  data.tar.gz: cd02de9753ce345ec599d72230f634df0e633f278871df5da4330318fcb92b8820d99612f43fb8d89be348460d1cb198a5bfab34fdce5c6f639709ab8dd426a2

data/README.md

@@ -23,6 +23,7 @@ Solid Queue can be used with SQL databases such as MySQL, PostgreSQL, or SQLite,
   - [Threads, processes, and signals](#threads-processes-and-signals)
   - [Database configuration](#database-configuration)
   - [Other configuration settings](#other-configuration-settings)
+  - [Validating the configuration](#validating-the-configuration)
 - [Lifecycle hooks](#lifecycle-hooks)
 - [Errors when enqueuing](#errors-when-enqueuing)
 - [Concurrency controls](#concurrency-controls)
@@ -44,7 +45,7 @@ Solid Queue is configured by default in new Rails 8 applications. If you're runn
 1. `bundle add solid_queue`
 2. `bin/rails solid_queue:install`
 
-(Note: The minimum supported version of Rails is 7.1 and Ruby is 3.1.6.)
+(Note: The minimum supported version of Rails is 7.1 and Ruby is 3.2.)
 
 This will configure Solid Queue as the production Active Job backend, create the configuration files `config/queue.yml` and `config/recurring.yml`, and create the `db/queue_schema.rb`. It'll also create a `bin/jobs` executable wrapper that you can use to start Solid Queue.
 
@@ -203,6 +204,10 @@ Or you can also set the environment variable `SOLID_QUEUE_SUPERVISOR_MODE` to `a
 
 **The recommended and default mode is `fork`. Only use `async` if you know what you're doing and have strong reasons to**
 
+This supervisor mode is separate from a worker's concurrency model. Supervisor mode decides whether supervised processes live in forks or threads. Worker configuration decides whether claimed jobs run in a thread pool (`threads: N`) or as fibers on a single fiber reactor thread (`fibers: N`).
+
+Because these are separate concerns, you can combine the default `fork` supervisor mode with fiber workers. In that setup, each worker process gets its own fiber reactor and bounded fiber count.
+
 ## Configuration
 
 By default, Solid Queue will try to find your configuration under `config/queue.yml`, but you can set a different path using the environment variable `SOLID_QUEUE_CONFIG` or by using the `-c/--config_file` option with `bin/jobs`, like this:
@@ -213,6 +218,8 @@ bin/jobs -c config/calendar.yml
 
 You can also skip the scheduler process by setting the environment variable `SOLID_QUEUE_SKIP_RECURRING=true`. This is useful for environments like staging, review apps, or development where you don't want any recurring jobs to run. This is equivalent to using the `--skip-recurring` option with `bin/jobs`.
 
+To run **only** the scheduler (no workers or dispatchers)—for example to isolate recurring tasks on a dedicated process—set `SOLID_QUEUE_ONLY_RECURRING=true` or use the `--only-recurring` option with `bin/jobs`.
+
 This is what this configuration looks like:
 
 ```yml
@@ -229,6 +236,9 @@ production:
       threads: 5
       polling_interval: 0.1
       processes: 3
+    - queues: "api*"
+      fibers: 100
+      polling_interval: 0.05
   scheduler:
     dynamic_tasks_enabled: true
     polling_interval: 5
@@ -271,9 +281,11 @@ Here's an overview of the different options:
 
   Check the sections below on [how queue order behaves combined with priorities](#queue-order-and-priorities), and [how the way you specify the queues per worker might affect performance](#queues-specification-and-performance).
 
-- `threads`: this is the max size of the thread pool that each worker will have to run jobs. Each worker will fetch this number of jobs from their queue(s), at most and will post them to the thread pool to be run. By default, this is `3`. Only workers have this setting.
-It is recommended to set this value less than or equal to the queue database's connection pool size minus 2, as each worker thread uses one connection, and two additional connections are reserved for polling and heartbeat.
-- `processes`: this is the number of worker processes that will be forked by the supervisor with the settings given. By default, this is `1`, just a single process. This setting is useful if you want to dedicate more than one CPU core to a queue or queues with the same configuration. Only workers have this setting. **Note**: this option will be ignored if [running in `async` mode](#fork-vs-async-mode).
+- `threads`: configures a worker to execute jobs in a thread pool of this size. By default, workers use `threads: 3`. Only workers have this setting, and it can't be combined with `fibers`.
+It is recommended to set this value less than or equal to the queue database's connection pool size minus 2, as each worker uses connections for polling and heartbeat and thread mode may use additional connections for job execution.
+- `fibers`: configures a worker to execute jobs as fibers on a single fiber reactor thread, with this value as the maximum number of in-flight jobs. It can't be combined with `threads`.
+  Fiber workers require fiber-scoped isolated execution state. In Rails apps, set `config.active_support.isolation_level = :fiber` before using `fibers`. Solid Queue refuses to boot fiber workers when isolation remains thread-scoped. On Rails 7.2 and later, a practical starting point is usually `3-5` queue database connections per worker process rather than matching the `fibers` value, because ordinary Active Record query paths can release connections between non-blocking waits. On Rails 7.1, size the queue database pool more conservatively, as in-flight fiber jobs may still retain connections roughly in proportion to `fibers`.
+- `processes`: this is the number of worker processes that will be forked by the supervisor with the settings given. By default, this is `1`, just a single process. This setting is useful if you want to dedicate more than one CPU core to a queue or queues with the same configuration. Only workers have this setting. This works with both `threads` and `fibers` workers as long as the supervisor is running in the default `fork` mode. **Note**: this option is ignored only when the supervisor itself is [running in `async` mode](#fork-vs-async-mode).
 - `concurrency_maintenance`: whether the dispatcher will perform the concurrency maintenance work. This is `true` by default, and it's useful if you don't use any [concurrency controls](#concurrency-controls) and want to disable it or if you run multiple dispatchers and want some of them to just dispatch jobs without doing anything else.
 
 
@@ -338,7 +350,7 @@ FROM solid_queue_ready_executions
 WHERE queue_name LIKE 'beta%';
 ```
 
-This type of `DISTINCT` query on a column that's the leftmost column in an index can be performed very fast in MySQL thanks to a technique called [Loose Index Scan](https://dev.mysql.com/doc/refman/8.0/en/group-by-optimization.html#loose-index-scan). PostgreSQL and SQLite, however, don't implement this technique, which means that if your `solid_queue_ready_executions` table is very big because your queues get very deep, this query will get slow. Normally your `solid_queue_ready_executions` table will be small, but it can happen.
+This type of `DISTINCT` query on a column that's the leftmost column in an index can be performed very fast in MySQL thanks to a technique called [Loose Index Scan](https://dev.mysql.com/doc/refman/8.0/en/group-by-optimization.html#loose-index-scan). PostgreSQL doesn't implement this technique natively, so Solid Queue uses a [recursive CTE](https://www.postgresql.org/docs/current/queries-with.html#QUERIES-WITH-RECURSIVE) to emulate it, achieving similar performance by walking the B-tree index and jumping between distinct values. SQLite doesn't implement loose index scan either, but this is unlikely to be a problem since SQLite is typically used in development with small datasets.
 
 Similarly to using prefixes, the same will happen if you have paused queues, because we need to get a list of all queues with a query like
 ```sql
@@ -364,7 +376,17 @@ queues: back*
 
 ### Threads, processes, and signals
 
-Workers in Solid Queue use a thread pool to run work in multiple threads, configurable via the `threads` parameter above. Besides this, parallelism can be achieved via multiple processes on one machine (configurable via different workers or the `processes` parameter above) or by horizontal scaling.
+By default, workers in Solid Queue use a thread pool to run work in multiple threads, configurable via the `threads` parameter above. Workers can also be configured with `fibers`, in which case claimed jobs are executed as fibers on a single reactor thread and bounded by the worker's fiber count. Besides this, parallelism can be achieved via multiple processes on one machine (configurable via different workers or the `processes` parameter above) or by horizontal scaling.
+
+Fiber worker execution is best suited for cooperative, mostly I/O-bound jobs. Blocking or CPU-heavy work still blocks the single reactor thread, so it should not be expected to outperform thread mode for every workload.
+
+Because fiber workers run multiple fibers on a single thread, Rails must also isolate execution state per fiber rather than per thread. If your app keeps the default thread-scoped isolation level, Solid Queue will raise a boot-time error instead of running fiber workers with shared Active Record state.
+
+Keep in mind that `config.active_support.isolation_level = :fiber` applies to your whole application, not just to Solid Queue: if you run Solid Queue inside Puma via [the plugin](#puma-plugin), or combine fiber workers with thread workers in the same process using the supervisor's `async` mode, everything in that process will use fiber-scoped execution state. This is fully supported by Rails, but it's a global setting worth being deliberate about.
+
+On Rails 7.2 and later, fiber workers can often use a much smaller queue database pool than an equivalent thread pool. A practical starting point is `3-5` queue database connections per worker process: one for job execution, one for polling, one for heartbeats, plus some headroom. In the default `fork` supervisor mode, that guidance applies per worker process. In supervisor `async` mode, all workers share one process, so add together the requirements for the workers running there.
+
+That lower-pool guidance depends on job code not holding connections open across non-blocking waits. APIs such as `ActiveRecord::Base.connection`, `lease_connection`, `connection_pool.checkout`, or long-lived `with_connection` / transaction blocks can pin connections and push fiber workers back toward thread-like pool usage. On Rails 7.1, plan conservatively and assume the configured fiber count can still grow queue database connection usage.
 
 The supervisor is in charge of managing these processes, and it responds to the following signals when running in its own process via `bin/jobs` or with [the Puma plugin](#puma-plugin) with the default `fork` mode:
 - `TERM`, `INT`: starts graceful termination. The supervisor will send a `TERM` signal to its supervised processes, and it'll wait up to `SolidQueue.shutdown_timeout` time until they're done. If any supervised processes are still around by then, it'll send a `QUIT` signal to them to indicate they must exit.
@@ -372,8 +394,14 @@ The supervisor is in charge of managing these processes, and it responds to the
 
 When receiving a `QUIT` signal, if workers still have jobs in-flight, these will be returned to the queue when the processes are deregistered.
 
+On Windows, the `QUIT` signal can't be trapped, so the supervisor only responds to `TERM` and `INT` there.
+
 If processes have no chance of cleaning up before exiting (e.g. if someone pulls a cable somewhere), in-flight jobs might remain claimed by the processes executing them. Processes send heartbeats, and the supervisor checks and prunes processes with expired heartbeats. Jobs that were claimed by processes with an expired heartbeat will be marked as failed with a `SolidQueue::Processes::ProcessPrunedError`. You can configure both the frequency of heartbeats and the threshold to consider a process dead. See the section below for this.
 
+Worker heartbeats are driven by a separate timer task, not by the worker execution backend itself. This means fiber workers do not rely on the reactor loop to prove liveness. However, liveness is still tracked at the worker-process level, not at the individual thread or fiber level.
+
+This means finished and failed jobs still follow the normal Solid Queue lifecycle, but a single stuck job can remain claimed if the worker process itself is still alive. If you need stronger stuck-job detection, that requires an explicit timeout or watchdog mechanism on top of process heartbeats.
+
 In a similar way, if a worker is terminated in any other way not initiated by the above signals (e.g. a worker is sent a `KILL` signal), jobs in progress will be marked as failed so that they can be inspected, with a `SolidQueue::Processes::ProcessExitError`. Sometimes a job in particular is responsible for this, for example, if it has a memory leak and you have a mechanism to kill processes over a certain memory threshold, so this will help identifying this kind of situation.
 
 
@@ -389,7 +417,7 @@ _Note_: The settings in this section should be set in your `config/application.r
 
 There are several settings that control how Solid Queue works that you can set as well:
 - `logger`: the logger you want Solid Queue to use. Defaults to the app logger.
-- `app_executor`: the [Rails executor](https://guides.rubyonrails.org/threading_and_code_execution.html#executor) used to wrap asynchronous operations, defaults to the app executor
+- `app_executor`: the [Rails executor](https://guides.rubyonrails.org/threading_and_code_execution.html#executor) used to wrap background operations, defaults to the app executor
 - `on_thread_error`: custom lambda/Proc to call when there's an error within a Solid Queue thread that takes the exception raised as argument. Defaults to
 
   ```ruby
@@ -401,6 +429,7 @@ There are several settings that control how Solid Queue works that you can set a
 - `use_skip_locked`: whether to use `FOR UPDATE SKIP LOCKED` when performing locking reads. This will be automatically detected in the future, and for now, you only need to set this to `false` if your database doesn't support it. For MySQL, that'd be versions < 8; for MariaDB, versions < 10.6; and for PostgreSQL, versions < 9.5. If you use SQLite, this has no effect, as writes are sequential.
 - `process_heartbeat_interval`: the heartbeat interval that all processes will follow—defaults to 60 seconds.
 - `process_alive_threshold`: how long to wait until a process is considered dead after its last heartbeat—defaults to 5 minutes.
+- `fork_boot_timeout`: how long a forked process can take to finish booting before the supervisor replaces it—defaults to 5 minutes. It only applies in the default `fork` mode.
 - `shutdown_timeout`: time the supervisor will wait since it sent the `TERM` signal to its supervised processes before sending a `QUIT` version to them requesting immediate termination—defaults to 5 seconds.
 - `silence_polling`: whether to silence Active Record logs emitted when polling for both workers and dispatchers—defaults to `true`.
 - `supervisor_pidfile`: path to a pidfile that the supervisor will create when booting to prevent running more than one supervisor in the same host, or in case you want to use it for a health check. It's `nil` by default.
@@ -408,6 +437,22 @@ There are several settings that control how Solid Queue works that you can set a
 - `clear_finished_jobs_after`: period to keep finished jobs around, in case `preserve_finished_jobs` is true — defaults to 1 day. When installing Solid Queue, [a recurring job](#recurring-tasks) is automatically configured to clear finished jobs every hour on the 12th minute in batches. You can edit the `recurring.yml` configuration to change this as you see fit.
 - `default_concurrency_control_period`: the value to be used as the default for the `duration` parameter in [concurrency controls](#concurrency-controls). It defaults to 3 minutes.
 
+### Validating the configuration
+
+You can validate the Solid Queue configuration ahead of time, without starting any process. This is handy in deploy scripts or CI to catch mistakes—a typo in `recurring.yml`, no processes configured, and so on—before they cause a supervisor to boot into a broken state:
+
+```bash
+# Using the bin/jobs binstub
+bin/jobs check
+
+# Or via rake
+bin/rails solid_queue:check
+```
+
+Both commands validate the configuration for the current Rails environment. On success they print `Solid Queue configuration is valid.` and exit `0`; otherwise they print the errors and exit non-zero. When the number of threads is larger than the [database connection pool](#database-configuration), they also print an advisory warning about it—the same one the supervisor logs on boot. They're tolerant of a missing database connection, so they can run on CI or deploy hosts without database credentials.
+
+`bin/jobs check` accepts the same options as `bin/jobs start` (e.g. `--config_file`, `--recurring_schedule_file`, `--skip-recurring`). The rake task honors the same environment variables Solid Queue already uses: `SOLID_QUEUE_CONFIG`, `SOLID_QUEUE_RECURRING_SCHEDULE`, and `SOLID_QUEUE_SKIP_RECURRING`. To validate a specific environment's configuration, set `RAILS_ENV`, for example `RAILS_ENV=production bin/jobs check`.
+
 
 ## Lifecycle hooks
 
@@ -471,7 +516,7 @@ Solid Queue extends Active Job with concurrency controls, that allows you to lim
 
 ```ruby
 class MyJob < ApplicationJob
-  limits_concurrency to: max_concurrent_executions, key: ->(arg1, arg2, **) { ... }, duration: max_interval_to_guarantee_concurrency_limit, group: concurrency_group, on_conflict: on_conflict_behaviour
+  limits_concurrency to: max_concurrent_executions, key: ->(arg1, arg2, *) { ... }, duration: max_interval_to_guarantee_concurrency_limit, group: concurrency_group, on_conflict: on_conflict_behaviour
 
   # ...
 ```
@@ -690,6 +735,8 @@ bin/jobs --recurring_schedule_file=config/schedule.yml
 
 You can completely disable recurring tasks by setting the environment variable `SOLID_QUEUE_SKIP_RECURRING=true` or by using the `--skip-recurring` option with `bin/jobs`.
 
+To run only the scheduler (no workers or dispatchers), set `SOLID_QUEUE_ONLY_RECURRING=true` or use `--only-recurring` with `bin/jobs`.
+
 The configuration itself looks like this:
 
 ```yml
@@ -705,7 +752,9 @@ production:
 
 Tasks are specified as a hash/dictionary, where the key will be the task's key internally. Each task needs to either have a `class`, which will be the job class to enqueue, or a `command`, which will be eval'ed in the context of a job (`SolidQueue::RecurringJob`) that will be enqueued according to its schedule, in the `solid_queue_recurring` queue.
 
-Each task needs to have also a schedule, which is parsed using [Fugit](https://github.com/floraison/fugit), so it accepts anything [that Fugit accepts as a cron](https://github.com/floraison/fugit?tab=readme-ov-file#fugitcron). You can optionally supply the following for each task:
+Each task needs to have also a schedule, which is parsed using [Fugit](https://github.com/floraison/fugit), so it accepts anything [that Fugit accepts as a cron](https://github.com/floraison/fugit?tab=readme-ov-file#fugitcron). Schedules can include a time zone (e.g. `0 9 * * * America/New_York` or `every day at 9am America/New_York`). When a schedule doesn't specify one, it's interpreted in the application's configured time zone (`config.time_zone`) by default. You can change or disable this default with `config.solid_queue.time_zone`; setting it to `nil` falls back to the system's local time.
+
+You can optionally supply the following for each task:
 - `args`: the arguments to be passed to the job, as a single argument, a hash, or an array of arguments that can also include kwargs as the last element in the array.
 
 The job in the example configuration above will be enqueued every second as:
@@ -781,6 +830,8 @@ SolidQueue.unschedule_recurring_task("my_dynamic_task")
 
 Only dynamic tasks can be unscheduled at runtime. Attempting to unschedule a static task (defined in `config/recurring.yml`) will raise an `ActiveRecord::RecordNotFound` error.
 
+To update an existing dynamic task, unschedule it and then schedule it again with the new options. A running scheduler only detects dynamic tasks being created and deleted, so updating a `SolidQueue::RecurringTask` record in place (for example, changing its `schedule` with `update!`) won't be picked up until the scheduler restarts.
+
 Tasks scheduled like this persist between Solid Queue's restarts and won't stop running until you manually unschedule them. 
 
 ## Inspiration

data/UPGRADING.md

@@ -1,3 +1,14 @@
+# Upgrading to version 1.5.x
+Ruby 3.1 is no longer supported, as it reached end-of-life in March 2025. Solid Queue now requires Ruby 3.2 or newer. If you're still on Ruby 3.1, Bundler will continue to resolve solid_queue 1.4.x for you, but you won't receive any new versions until you upgrade Ruby.
+
+Recurring schedules that don't specify a time zone are now interpreted in your application's configured time zone (`config.time_zone`) by default, instead of the system's local time. This only affects schedules without an explicit time zone (e.g. `every day at 9am`); schedules that already include one (e.g. `0 9 * * * America/New_York`) are unaffected.
+
+If your `config.time_zone` differs from the system time where your processes run, recurring jobs may fire at a different wall-clock time than before. To keep the previous behavior, set:
+
+```ruby
+config.solid_queue.time_zone = nil
+```
+
 # Upgrading to version 1.x
 The value returned for `enqueue_after_transaction_commit?` has changed to `true`, and it's no longer configurable. If you want to change this, you need to use Active Job's configuration options.
 

data/app/models/solid_queue/claimed_execution.rb

@@ -43,7 +43,6 @@ class SolidQueue::ClaimedExecution < SolidQueue::Execution
         SolidQueue.instrument(:fail_many_claimed) do |payload|
           executions.each do |execution|
             execution.failed_with(error)
-            execution.unblock_next_job
           end
 
           payload[:process_ids] = executions.map(&:process_id).uniq
@@ -71,13 +70,11 @@ class SolidQueue::ClaimedExecution < SolidQueue::Execution
       failed_with(result.error)
       raise result.error
     end
-  ensure
-    unblock_next_job
   end
 
   def release
     SolidQueue.instrument(:release_claimed, job_id: job.id, process_id: process_id) do
-      transaction do
+      unless_already_finalized do
         job.dispatch_bypassing_concurrency_limits
         destroy!
       end
@@ -89,14 +86,7 @@ class SolidQueue::ClaimedExecution < SolidQueue::Execution
   end
 
   def failed_with(error)
-    transaction do
-      job.failed_with(error)
-      destroy!
-    end
-  end
-
-  def unblock_next_job
-    job.unblock_next_blocked_job
+    finalize { job.failed_with(error) }
   end
 
   private
@@ -108,9 +98,28 @@ class SolidQueue::ClaimedExecution < SolidQueue::Execution
     end
 
     def finished
-      transaction do
-        job.finished!
+      finalize { job.finished! }
+    end
+
+    def finalize
+      finalized = unless_already_finalized do
+        yield
         destroy!
+        true
+      end
+
+      # Unblock the next job outside the finalize transaction so a failure while
+      # releasing the concurrency lock or dispatching the next job can't roll back
+      # a job that already finished or failed. Only the actor that owned and
+      # finalized the claim gets here, so the lock is released exactly once.
+      job.unblock_next_blocked_job if finalized
+    end
+
+    def unless_already_finalized
+      transaction do
+        return false unless self.class.unscoped.lock.find_by(id: id)
+
+        yield
       end
     end
 end

data/app/models/solid_queue/job/executable.rb

@@ -41,15 +41,16 @@ module SolidQueue
           end
 
           def successfully_dispatched(jobs)
-            dispatched_and_ready(jobs) + dispatched_and_blocked(jobs)
+            jobs_by_id = jobs.index_by(&:id)
+            dispatched_and_ready(jobs_by_id) + dispatched_and_blocked(jobs_by_id)
           end
 
-          def dispatched_and_ready(jobs)
-            where(id: ReadyExecution.where(job_id: jobs.map(&:id)).pluck(:job_id))
+          def dispatched_and_ready(jobs_by_id)
+            ReadyExecution.where(job_id: jobs_by_id.keys).pluck(:job_id).map { |id| jobs_by_id[id] }
           end
 
-          def dispatched_and_blocked(jobs)
-            where(id: BlockedExecution.where(job_id: jobs.map(&:id)).pluck(:job_id))
+          def dispatched_and_blocked(jobs_by_id)
+            BlockedExecution.where(job_id: jobs_by_id.keys).pluck(:job_id).map { |id| jobs_by_id[id] }
           end
       end
 

data/app/models/solid_queue/job/schedulable.rb

@@ -23,7 +23,8 @@ module SolidQueue
           end
 
           def successfully_scheduled(jobs)
-            where(id: ScheduledExecution.where(job_id: jobs.map(&:id)).pluck(:job_id))
+            jobs_by_id = jobs.index_by(&:id)
+            ScheduledExecution.where(job_id: jobs_by_id.keys).pluck(:job_id).map { |id| jobs_by_id[id] }
           end
       end
 

data/app/models/solid_queue/queue.rb

@@ -6,9 +6,7 @@ module SolidQueue
 
     class << self
       def all
-        Job.select(:queue_name).distinct.collect do |job|
-          new(job.queue_name)
-        end
+        Job.distinct_values_of(:queue_name).map { |name| new(name) }
       end
 
       def find_by_name(name)

data/app/models/solid_queue/queue_selector.rb

@@ -43,7 +43,7 @@ module SolidQueue
       end
 
       def all_queues
-        relation.distinct(:queue_name).pluck(:queue_name)
+        relation.distinct_values_of(:queue_name)
       end
 
       def exact_names
@@ -53,7 +53,7 @@ module SolidQueue
       def prefixed_names
         if prefixes.empty? then []
         else
-          relation.where(([ "queue_name LIKE ?" ] * prefixes.count).join(" OR "), *prefixes).distinct(:queue_name).pluck(:queue_name)
+          relation.where(([ "queue_name LIKE ?" ] * prefixes.count).join(" OR "), *prefixes).distinct_values_of(:queue_name)
         end
       end
 

data/app/models/solid_queue/record/distinct_values.rb

@@ -0,0 +1,50 @@
+# frozen_string_literal: true
+
+module SolidQueue
+  class Record
+    module DistinctValues
+      extend ActiveSupport::Concern
+
+      # PostgreSQL has no native loose index scan, so a plain DISTINCT on a leading
+      # index column degrades to a full index scan on large tables. We emulate one
+      # with a recursive CTE that walks the index jumping between distinct values.
+      class_methods do
+        def distinct_values_of(column)
+          if loose_index_scan_emulation_needed?
+            loose_distinct_via_recursive_cte(column)
+          else
+            distinct.pluck(column)
+          end
+        end
+
+        private
+          def loose_index_scan_emulation_needed?
+            connection_pool.with_connection { |connection| connection.adapter_name == "PostgreSQL" }
+          end
+
+          # Emulates a loose index scan, honoring the current scope (e.g. LIKE prefixes)
+          # by building the anchor and the recursive step as scoped relations, whose
+          # #to_sql inlines any bind parameters so they can be embedded in the raw CTE.
+          def loose_distinct_via_recursive_cte(column)
+            connection_pool.with_connection do |connection|
+              col = connection.quote_column_name(column)
+
+              connection.select_values(<<~SQL.squish)
+                WITH RECURSIVE t AS (
+                  (#{next_distinct_value(col, "#{col} IS NOT NULL")})
+                  UNION ALL
+                  SELECT (#{next_distinct_value(col, "#{col} > t.#{col}")}) FROM t WHERE t.#{col} IS NOT NULL
+                )
+                SELECT #{col} FROM t WHERE #{col} IS NOT NULL
+              SQL
+            end
+          end
+
+          # Smallest value of `col` within the current scope that matches `condition`.
+          def next_distinct_value(col, condition)
+            all.where(Arel.sql(condition)).reorder(Arel.sql(col)).limit(1).select(Arel.sql(col)).to_sql
+          end
+      end
+    end
+  end
+end

data/app/models/solid_queue/record.rb

@@ -3,6 +3,9 @@
 module SolidQueue
   class Record < ActiveRecord::Base
     self.abstract_class = true
+    self.strict_loading_by_default = false
+
+    include DistinctValues
 
     connects_to(**SolidQueue.connects_to) if SolidQueue.connects_to
 
@@ -21,6 +24,16 @@ module SolidQueue
         end
       end
 
+      def warn_about_pending_migrations
+        SolidQueue.deprecator.warn(<<~DEPRECATION)
+          Solid Queue has pending database migrations. To get the new migration files, run:
+            rails solid_queue:update
+          And then:
+            rails db:migrate
+          These migrations will be required after version #{SolidQueue.next_major_version}.0
+        DEPRECATION
+      end
+
       # Pass index hints to the query optimizer using SQL comment hints.
       # Uses MySQL 8 optimizer hint query comments, which SQLite and
       # PostgreSQL ignore.

data/app/models/solid_queue/recurring_task.rb

@@ -56,16 +56,17 @@ module SolidQueue
       end
     end
 
-    def delay_from_now
-      [ (next_time - Time.current).to_f, 0.1 ].max
+
+    def next_time_after(time)
+      parsed_schedule_with_time_zone.next_time(time).utc
     end
 
     def next_time
-      parsed_schedule.next_time.utc
+      parsed_schedule_with_time_zone.next_time.utc
     end
 
     def previous_time
-      parsed_schedule.previous_time.utc
+      parsed_schedule_with_time_zone.previous_time.utc
     end
 
     def last_enqueued_time
@@ -85,6 +86,7 @@ module SolidQueue
 
           perform_later.tap do |job|
             unless job.successfully_enqueued?
+              report_enqueue_error(job.enqueue_error, at: at)
               payload[:enqueue_error] = job.enqueue_error&.message
             end
           end
@@ -97,6 +99,7 @@ module SolidQueue
         payload[:skipped] = true
         false
       rescue Job::EnqueueError => error
+        report_enqueue_error(error, at: at)
         payload[:enqueue_error] = error.message
         false
       end
@@ -168,11 +171,24 @@ module SolidQueue
         end
       end
 
+      def parsed_schedule_with_time_zone
+        @parsed_schedule_with_time_zone ||= apply_default_time_zone_to(parsed_schedule)
+      end
 
       def parsed_schedule
         @parsed_schedule ||= Fugit.parse(schedule, multi: :fail)
       end
 
+      def apply_default_time_zone_to(schedule)
+        if schedule.respond_to?(:zone) && schedule.zone.nil? && default_time_zone.present?
+          Fugit.parse("#{schedule.to_cron_s} #{default_time_zone}", multi: :fail)
+        else
+          schedule
+        end
+      rescue ArgumentError
+        schedule
+      end
+
       def job_class
         @job_class ||= class_name.present? ? class_name.safe_constantize : self.class.default_job_class
       end
@@ -180,5 +196,15 @@ module SolidQueue
       def enqueue_options
         { queue: queue_name, priority: priority }.compact
       end
+
+      def default_time_zone
+        SolidQueue.time_zone
+      end
+
+      def report_enqueue_error(error, at:)
+        if error
+          Rails.error.report(error, handled: true, source: "application.solid_queue", context: { task: key, at: at })
+        end
+      end
   end
 end

data/app/models/solid_queue/semaphore.rb

@@ -32,7 +32,13 @@ module SolidQueue
 
     class Proxy
       def self.signal_all(jobs)
-        Semaphore.where(key: jobs.map(&:concurrency_key)).update_all("value = value + 1")
+        # Guard against incrementing a semaphore's value beyond its limit. Jobs can
+        # have different limits, so group them and cap each group with `value < limit`.
+        jobs.group_by { |job| job.concurrency_limit || 1 }.each do |limit, grouped_jobs|
+          Semaphore.where(key: grouped_jobs.map(&:concurrency_key))
+            .where(value: ...limit)
+            .update_all("value = value + 1")
+        end
       end
 
       def initialize(job)

data/lib/active_job/queue_adapters/solid_queue_adapter.rb

@@ -8,9 +8,17 @@ module ActiveJob
     #
     #   Rails.application.config.active_job.queue_adapter = :solid_queue
     class SolidQueueAdapter < (Rails::VERSION::MAJOR == 7 && Rails::VERSION::MINOR == 1 ? Object : AbstractAdapter)
-      class_attribute :stopping, default: false, instance_writer: false
+      class_attribute :stopping, default: false, instance_writer: false, instance_predicate: false
       SolidQueue.on_worker_stop { self.stopping = true }
 
+      # Accept an optional job argument for compatibility with Rails main, which
+      # began passing the running job to +queue_adapter.stopping?+ so adapters can
+      # decide whether to checkpoint based on it. We rely solely on the worker
+      # shutdown flag, so the argument is ignored.
+      def stopping?(_job = nil)
+        self.class.stopping
+      end
+
       def enqueue_after_transaction_commit?
         true
       end

data/lib/generators/solid_queue/install/templates/config/queue.yml

@@ -6,7 +6,7 @@ default: &default
     - queues: "*"
       threads: 3
       processes: <%%= ENV.fetch("JOB_CONCURRENCY", 1) %>
-      polling_interval: 0.1
+      polling_interval: 1
 
 development:
   <<: *default

data/lib/generators/solid_queue/update/update_generator.rb

@@ -0,0 +1,19 @@
+# frozen_string_literal: true
+
+require "rails/generators/active_record"
+
+class SolidQueue::UpdateGenerator < Rails::Generators::Base
+  include ActiveRecord::Generators::Migration
+
+  source_root File.expand_path("templates", __dir__)
+
+  class_option :database, type: :string, aliases: %i[ --db ], default: "queue",
+    desc: "The database that Solid Queue uses. Defaults to `queue`"
+
+  def copy_new_migrations
+    Dir.glob(File.join(self.class.source_root, "db", "*.rb")).each do |migration_file|
+      name = File.basename(migration_file)
+      migration_template File.join("db", name), File.join(db_migrate_path, name), skip: true
+    end
+  end
+end

data/lib/solid_queue/cli.rb

@@ -20,6 +20,10 @@ module SolidQueue
       desc: "Whether to skip recurring tasks scheduling",
       banner: "SOLID_QUEUE_SKIP_RECURRING"
 
+    class_option :only_recurring, type: :boolean,
+      desc: "Whether to run only the scheduler process for recurring tasks",
+      banner: "SOLID_QUEUE_ONLY_RECURRING"
+
     def self.exit_on_failure?
       true
     end
@@ -30,5 +34,11 @@ module SolidQueue
     def start
       SolidQueue::Supervisor.start(**options.symbolize_keys)
     end
+
+    desc :check, "Validates the Solid Queue configuration for the current Rails env without starting anything. Exits non-zero on errors."
+    def check
+      configuration = SolidQueue::Configuration.new(**options.symbolize_keys)
+      exit 1 unless configuration.check
+    end
   end
 end