solid_queue 1.2.0 → 1.2.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1d0b2a7e3a5ad6577d52d3528f42ff8862c087f6f406d0f6fd96f12f36db6795
-  data.tar.gz: 4db8708635736bd5ccb32ad619577494c039574fd452a40c3fecc55d0f57c27b
+  metadata.gz: ee016dde69aa11302fc20335e657d8f65f31049ad1fad9bd56c363c8e8a6ee4d
+  data.tar.gz: fd54a73d694f7dba45bd8a86daf426cceb4f351c87492d75130517b23929d884
 SHA512:
-  metadata.gz: 2ba5bb04f334bcc56bb0504d0baac497287058c242aaf8fccd85c72e1c72c5107ad38eff211d9c16641a3cc7ad366673a316cf357daf70bdf97009d68b7c5646
-  data.tar.gz: bf7770477e8dd290bd9461bcc5202b5d0832639ed31e7df7e96d192562fe5c0574522f1f689504c6d98c44c14ba389597bd7627fe3ec76c1e76937c0f0eaaac7
+  metadata.gz: 4a30c8abce9f22bd28901387bbb7c0a1da41671926033f48c2f46339f3b447ae5677763f52f0b1b46801a9f3bf206834d1eae712462989bbe6a6d8daa2fd43e5
+  data.tar.gz: 286fafe7fc7202053989993bbfc51a207b530a538f2898685482c328cfe6bf28fc3b514d4ea15c2c5fcb57cadbf010c43823fba12dd92dfc54b0b105a45f340c

data/README.md

@@ -1,12 +1,12 @@
 # Solid Queue
 
-Solid Queue is a DB-based queuing backend for [Active Job](https://edgeguides.rubyonrails.org/active_job_basics.html), designed with simplicity and performance in mind.
+Solid Queue is a database-based queuing backend for [Active Job](https://edgeguides.rubyonrails.org/active_job_basics.html), designed with simplicity and performance in mind.
 
-Besides regular job enqueuing and processing, Solid Queue supports delayed jobs, concurrency controls, recurring jobs, pausing queues, numeric priorities per job, priorities by queue order, and bulk enqueuing (`enqueue_all` for Active Job's `perform_all_later`).
+In addition to regular job enqueuing and processing, Solid Queue supports delayed jobs, concurrency controls, recurring jobs, pausing queues, numeric priorities per job, priorities by queue order, and bulk enqueuing (`enqueue_all` for Active Job's `perform_all_later`).
 
-Solid Queue can be used with SQL databases such as MySQL, PostgreSQL or SQLite, and it leverages the `FOR UPDATE SKIP LOCKED` clause, if available, to avoid blocking and waiting on locks when polling jobs. It relies on Active Job for retries, discarding, error handling, serialization, or delays, and it's compatible with Ruby on Rails's multi-threading.
+Solid Queue can be used with SQL databases such as MySQL, PostgreSQL, or SQLite, and it leverages the `FOR UPDATE SKIP LOCKED` clause, if available, to avoid blocking and waiting on locks when polling jobs. It relies on Active Job for retries, discarding, error handling, serialization, and delays, and it's compatible with Ruby on Rails's multi-threading.
 
-## Table of contents
+## Table of Contents
 
 - [Installation](#installation)
   - [Usage in development and other non-production environments](#usage-in-development-and-other-non-production-environments)
@@ -15,10 +15,10 @@ Solid Queue can be used with SQL databases such as MySQL, PostgreSQL or SQLite,
   - [Incremental adoption](#incremental-adoption)
   - [High performance requirements](#high-performance-requirements)
 - [Configuration](#configuration)
-  - [Workers, dispatchers and scheduler](#workers-dispatchers-and-scheduler)
+  - [Workers, dispatchers, and scheduler](#workers-dispatchers-and-scheduler)
   - [Queue order and priorities](#queue-order-and-priorities)
   - [Queues specification and performance](#queues-specification-and-performance)
-  - [Threads, processes and signals](#threads-processes-and-signals)
+  - [Threads, processes, and signals](#threads-processes-and-signals)
   - [Database configuration](#database-configuration)
   - [Other configuration settings](#other-configuration-settings)
 - [Lifecycle hooks](#lifecycle-hooks)
@@ -36,7 +36,7 @@ Solid Queue can be used with SQL databases such as MySQL, PostgreSQL or SQLite,
 
 ## Installation
 
-Solid Queue is configured by default in new Rails 8 applications. But if you're running an earlier version, you can add it manually following these steps:
+Solid Queue is configured by default in new Rails 8 applications. If you're running an earlier version, you can add it manually following these steps:
 
 1. `bundle add solid_queue`
 2. `bin/rails solid_queue:install`
@@ -45,7 +45,7 @@ Solid Queue is configured by default in new Rails 8 applications. But if you're
 
 This will configure Solid Queue as the production Active Job backend, create the configuration files `config/queue.yml` and `config/recurring.yml`, and create the `db/queue_schema.rb`. It'll also create a `bin/jobs` executable wrapper that you can use to start Solid Queue.
 
-Once you've done that, you will then have to add the configuration for the queue database in `config/database.yml`. If you're using SQLite, it'll look like this:
+Once you've done that, you will have to add the configuration for the queue database in `config/database.yml`. If you're using SQLite, it'll look like this:
 
 ```yaml
 production:
@@ -79,7 +79,7 @@ Now you're ready to start processing jobs by running `bin/jobs` on the server th
 
 For small projects, you can run Solid Queue on the same machine as your webserver. When you're ready to scale, Solid Queue supports horizontal scaling out-of-the-box. You can run Solid Queue on a separate server from your webserver, or even run `bin/jobs` on multiple machines at the same time. Depending on the configuration, you can designate some machines to run only dispatchers or only workers. See the [configuration](#configuration) section for more details on this.
 
-**Note**: future changes to the schema will come in the form of regular migrations.
+**Note**: Future changes to the schema will come in the form of regular migrations.
 
 ### Usage in development and other non-production environments
 
@@ -150,7 +150,7 @@ development:
 
 ### Single database configuration
 
-Running Solid Queue in a separate database is recommended, but it's also possible to use one single database for both the app and the queue. Just follow these steps:
+Running Solid Queue in a separate database is recommended, but it's also possible to use one single database for both the app and the queue. Follow these steps:
 
 1. Copy the contents of `db/queue_schema.rb` into a normal migration and delete `db/queue_schema.rb`
 2. Remove `config.solid_queue.connects_to` from `production.rb`
@@ -158,7 +158,7 @@ Running Solid Queue in a separate database is recommended, but it's also possibl
 
 You won't have multiple databases, so `database.yml` doesn't need to have primary and queue database.
 
-### Dashboard ui setup
+### Dashboard UI Setup
 
 For viewing information about your jobs via a UI, we recommend taking a look at [mission_control-jobs](https://github.com/rails/mission_control-jobs), a dashboard where, among other things, you can examine and retry/discard failed jobs.
 
@@ -181,7 +181,7 @@ Solid Queue was designed for the highest throughput when used with MySQL 8+ or P
 
 ## Configuration
 
-### Workers, dispatchers and scheduler
+### Workers, dispatchers, and scheduler
 
 We have several types of actors in Solid Queue:
 
@@ -330,7 +330,7 @@ queues: back*
 ```
 
 
-### Threads, processes and signals
+### Threads, processes, and signals
 
 Workers in Solid Queue use a thread pool to run work in multiple threads, configurable via the `threads` parameter above. Besides this, parallelism can be achieved via multiple processes on one machine (configurable via different workers or the `processes` parameter above) or by horizontal scaling.
 
@@ -366,7 +366,7 @@ There are several settings that control how Solid Queue works that you can set a
 
   **This is not used for errors raised within a job execution**. Errors happening in jobs are handled by Active Job's `retry_on` or `discard_on`, and ultimately will result in [failed jobs](#failed-jobs-and-retries). This is for errors happening within Solid Queue itself.
 
-- `use_skip_locked`: whether to use `FOR UPDATE SKIP LOCKED` when performing locking reads. This will be automatically detected in the future, and for now, you'd only need to set this to `false` if your database doesn't support it. For MySQL, that'd be versions < 8, and for PostgreSQL, versions < 9.5. If you use SQLite, this has no effect, as writes are sequential.
+- `use_skip_locked`: whether to use `FOR UPDATE SKIP LOCKED` when performing locking reads. This will be automatically detected in the future, and for now, you only need to set this to `false` if your database doesn't support it. For MySQL, that'd be versions < 8, and for PostgreSQL, versions < 9.5. If you use SQLite, this has no effect, as writes are sequential.
 - `process_heartbeat_interval`: the heartbeat interval that all processes will follow—defaults to 60 seconds.
 - `process_alive_threshold`: how long to wait until a process is considered dead after its last heartbeat—defaults to 5 minutes.
 - `shutdown_timeout`: time the supervisor will wait since it sent the `TERM` signal to its supervised processes before sending a `QUIT` version to them requesting immediate termination—defaults to 5 seconds.
@@ -433,7 +433,9 @@ In the case of recurring tasks, if such error is raised when enqueuing the job c
 
 ## Concurrency controls
 
-Solid Queue extends Active Job with concurrency controls, that allows you to limit how many jobs of a certain type or with certain arguments can run at the same time. When limited in this way, by default, jobs will be blocked from running, and they'll stay blocked until another job finishes and unblocks them, or after the set expiry time (concurrency limit's _duration_) elapses. Alternatively, jobs can be configured to be discarded instead of blocked. This means that if a job with certain arguments has already been enqueued, other jobs with the same characteristics (in the same concurrency _class_) won't be enqueued.
+Solid Queue extends Active Job with concurrency controls, that allows you to limit how many jobs of a certain type or with certain arguments can run at the same time. When limited in this way, **by default, jobs will be blocked from running**, and they'll stay blocked until another job finishes and unblocks them, or after the set expiry time (concurrency limit's _duration_) elapses.
+
+**Alternatively, jobs can be configured to be discarded instead of blocked**. This means that if a job with certain arguments has already been enqueued, other jobs with the same characteristics (in the same concurrency _class_) won't be enqueued.
 
 ```ruby
 class MyJob < ApplicationJob
@@ -496,6 +498,29 @@ Jobs are unblocked in order of priority but **queue order is not taken into acco
 
 Finally, failed jobs that are automatically or manually retried work in the same way as new jobs that get enqueued: they get in the queue for getting an open semaphore, and whenever they get it, they'll be run. It doesn't matter if they had already gotten an open semaphore in the past.
 
+### Scheduled jobs
+
+Jobs set to run in the future (via Active Job's `wait` or `wait_until` options) have concurrency limits enforced when they're due, not when they're scheduled. For example, consider this job:
+```ruby
+class DeliverAnnouncementToContactJob < ApplicationJob
+  limits_concurrency to: 1, key: ->(contact) { contact.account }, duration: 5.minutes
+
+  def perform(contact)
+    # ...
+```
+
+If several jobs are enqueued like this:
+
+```ruby
+DeliverAnnouncementToContactJob.set(wait: 10.minutes).perform_later(contact)
+DeliverAnnouncementToContactJob.set(wait: 10.minutes).perform_later(contact)
+DeliverAnnouncementToContactJob.set(wait: 30.minutes).perform_later(contact)
+```
+
+The 3 jobs will go into the scheduled queue and will wait there until they're due. Then, 10 minutes after, the first two jobs will be enqueued and the second one most likely will be blocked because the first one will be running first. Then, assuming the jobs are fast and finish in a few seconds, when the third job is due, it'll be enqueued normally.
+
+Normally scheduled jobs are enqueued in batches, but with concurrency controls, jobs need to be enqueued one by one. This has an impact on performance, similarly to the impact of concurrency controls in bulk enqueuing. Read below for more details. I'd generally advise against mixing concurrency controls with waiting/scheduling in the future.
+
 ### Performance considerations
 
 Concurrency controls introduce significant overhead (blocked executions need to be created and promoted to ready, semaphores need to be created and updated) so you should consider carefully whether you need them. For throttling purposes, where you plan to have `limit` significantly larger than 1, I'd encourage relying on a limited number of workers per queue instead. For example:
@@ -591,7 +616,7 @@ end
 
 Using this option, you can also use Solid Queue in the same database as your app but not rely on transactional integrity.
 
-If you don't set this option but still want to make sure you're not inadvertently on transactional integrity, you can make sure that:
+If you don't set this option but still want to make sure you're not inadvertently relying on transactional integrity, you can make sure that:
 - Your jobs relying on specific data are always enqueued on [`after_commit` callbacks](https://guides.rubyonrails.org/active_record_callbacks.html#after-commit-and-after-rollback) or otherwise from a place where you're certain that whatever data the job will use has been committed to the database before the job is enqueued.
 - Or, you configure a different database for Solid Queue, even if it's the same as your app, ensuring that a different connection on the thread handling requests or running jobs for your app will be used to enqueue jobs. For example:
 

data/app/models/solid_queue/claimed_execution.rb

@@ -37,8 +37,10 @@ class SolidQueue::ClaimedExecution < SolidQueue::Execution
     end
 
     def fail_all_with(error)
-      SolidQueue.instrument(:fail_many_claimed) do |payload|
-        includes(:job).tap do |executions|
+      includes(:job).tap do |executions|
+        return if executions.empty?
+
+        SolidQueue.instrument(:fail_many_claimed) do |payload|
           executions.each do |execution|
             execution.failed_with(error)
             execution.unblock_next_job

data/app/models/solid_queue/failed_execution.rb

@@ -58,8 +58,11 @@ module SolidQueue
       end
 
       def determine_backtrace_size_limit
-        column = self.class.connection.schema_cache.columns_hash(self.class.table_name)["error"]
-        if column.limit.present?
+        column = self.class.connection_pool.with_connection do |connection|
+          connection.schema_cache.columns_hash(self.class.table_name)["error"]
+        end
+
+        if column && column.limit.present?
           column.limit - exception_class_name.bytesize - exception_message.bytesize - JSON_OVERHEAD
         end
       end

data/app/models/solid_queue/job.rb

@@ -10,6 +10,7 @@ module SolidQueue
 
     class << self
       def enqueue_all(active_jobs)
+        active_jobs.each { |job| job.scheduled_at ||= Time.current }
         active_jobs_by_job_id = active_jobs.index_by(&:job_id)
 
         transaction do
@@ -30,6 +31,7 @@ module SolidQueue
 
         create_from_active_job(active_job).tap do |enqueued_job|
           active_job.provider_job_id = enqueued_job.id if enqueued_job.persisted?
+          active_job.successfully_enqueued = enqueued_job.persisted?
         end
       end
 

data/app/models/solid_queue/record.rb

@@ -6,11 +6,19 @@ module SolidQueue
 
     connects_to(**SolidQueue.connects_to) if SolidQueue.connects_to
 
-    def self.non_blocking_lock
-      if SolidQueue.use_skip_locked
-        lock(Arel.sql("FOR UPDATE SKIP LOCKED"))
-      else
-        lock
+    class << self
+      def non_blocking_lock
+        if SolidQueue.use_skip_locked
+          lock(Arel.sql("FOR UPDATE SKIP LOCKED"))
+        else
+          lock
+        end
+      end
+
+      def supports_insert_conflict_target?
+        connection_pool.with_connection do |connection|
+          connection.supports_insert_conflict_target?
+        end
       end
     end
   end

data/app/models/solid_queue/recurring_execution.rb

@@ -8,7 +8,7 @@ module SolidQueue
 
     class << self
       def create_or_insert!(**attributes)
-        if connection.supports_insert_conflict_target?
+        if supports_insert_conflict_target?
           # PostgreSQL fails and aborts the current transaction when it hits a duplicate key conflict
           # during two concurrent INSERTs for the same value of an unique index. We need to explicitly
           # indicate unique_by to ignore duplicate rows by this value when inserting

data/app/models/solid_queue/recurring_task.rb

@@ -36,7 +36,7 @@ module SolidQueue
       end
 
       def create_or_update_all(tasks)
-        if connection.supports_insert_conflict_target?
+        if supports_insert_conflict_target?
           # PostgreSQL fails and aborts the current transaction when it hits a duplicate key conflict
           # during two concurrent INSERTs for the same value of an unique index. We need to explicitly
           # indicate unique_by to ignore duplicate rows by this value when inserting
@@ -48,7 +48,7 @@ module SolidQueue
     end
 
     def delay_from_now
-      [ (next_time - Time.current).to_f, 0 ].max
+      [ (next_time - Time.current).to_f, 0.1 ].max
     end
 
     def next_time
@@ -130,7 +130,6 @@ module SolidQueue
             active_job.run_callbacks(:enqueue) do
               Job.enqueue(active_job)
             end
-            active_job.successfully_enqueued = true
           end
         end
       end

data/app/models/solid_queue/semaphore.rb

@@ -20,7 +20,7 @@ module SolidQueue
 
       # Requires a unique index on key
       def create_unique_by(attributes)
-        if connection.supports_insert_conflict_target?
+        if supports_insert_conflict_target?
           insert({ **attributes }, unique_by: :key).any?
         else
           create!(**attributes)

data/lib/puma/plugin/solid_queue.rb

@@ -11,15 +11,27 @@ Puma::Plugin.create do
       monitor_solid_queue
     end
 
-    launcher.events.on_booted do
-      @solid_queue_pid = fork do
-        Thread.new { monitor_puma }
-        SolidQueue::Supervisor.start
+    if Gem::Version.new(Puma::Const::VERSION) < Gem::Version.new("7")
+      launcher.events.on_booted do
+        @solid_queue_pid = fork do
+          Thread.new { monitor_puma }
+          SolidQueue::Supervisor.start
+        end
+      end
+
+      launcher.events.on_stopped { stop_solid_queue }
+      launcher.events.on_restart { stop_solid_queue }
+    else
+      launcher.events.after_booted do
+        @solid_queue_pid = fork do
+          Thread.new { monitor_puma }
+          SolidQueue::Supervisor.start
+        end
       end
-    end
 
-    launcher.events.on_stopped { stop_solid_queue }
-    launcher.events.on_restart { stop_solid_queue }
+      launcher.events.after_stopped { stop_solid_queue }
+      launcher.events.before_restart { stop_solid_queue }
+    end
   end
 
   private

data/lib/solid_queue/cli.rb

@@ -12,7 +12,7 @@ module SolidQueue
       desc: "Path to recurring schedule definition (default: #{Configuration::DEFAULT_RECURRING_SCHEDULE_FILE_PATH}).",
       banner: "SOLID_QUEUE_RECURRING_SCHEDULE"
 
-    class_option :skip_recurring, type: :boolean, default: false,
+    class_option :skip_recurring, type: :boolean,
       desc: "Whether to skip recurring tasks scheduling",
       banner: "SOLID_QUEUE_SKIP_RECURRING"
 

data/lib/solid_queue/supervisor.rb

@@ -29,18 +29,22 @@ module SolidQueue
     end
 
     def start
-      boot
-      run_start_hooks
+      wrap_in_app_executor do
+        boot
+        run_start_hooks
 
-      start_processes
-      launch_maintenance_task
+        start_processes
+        launch_maintenance_task
 
-      supervise
+        supervise
+      end
     end
 
     def stop
-      super
-      run_stop_hooks
+      wrap_in_app_executor do
+        super
+        run_stop_hooks
+      end
     end
 
     private

data/lib/solid_queue/version.rb

@@ -1,3 +1,3 @@
 module SolidQueue
-  VERSION = "1.2.0"
+  VERSION = "1.2.2"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: solid_queue
 version: !ruby/object:Gem::Version
-  version: 1.2.0
+  version: 1.2.2
 platform: ruby
 authors:
 - Rosa Gutierrez
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2025-07-11 00:00:00.000000000 Z
+date: 2025-10-21 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activerecord
@@ -72,26 +72,26 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 1.11.0
+        version: '1.11'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 1.11.0
+        version: '1.11'
 - !ruby/object:Gem::Dependency
   name: thor
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
         version: 1.3.1
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
         version: 1.3.1
 - !ruby/object:Gem::Dependency
@@ -140,16 +140,16 @@ dependencies:
   name: puma
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
+    - - "~>"
       - !ruby/object:Gem::Version
-        version: '0'
+        version: '7.0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
+    - - "~>"
       - !ruby/object:Gem::Version
-        version: '0'
+        version: '7.0'
 - !ruby/object:Gem::Dependency
   name: mysql2
   requirement: !ruby/object:Gem::Requirement
@@ -257,7 +257,6 @@ extra_rdoc_files: []
 files:
 - MIT-LICENSE
 - README.md
-- Rakefile
 - UPGRADING.md
 - app/jobs/solid_queue/recurring_job.rb
 - app/models/solid_queue/blocked_execution.rb

data/Rakefile

@@ -1,21 +0,0 @@
-# frozen_string_literal: true
-
-require "bundler/setup"
-
-APP_RAKEFILE = File.expand_path("test/dummy/Rakefile", __dir__)
-load "rails/tasks/engine.rake"
-
-load "rails/tasks/statistics.rake"
-
-require "bundler/gem_tasks"
-
-def databases
-  %w[ mysql postgres sqlite ]
-end
-
-task :test do
-  databases.each do |database|
-    sh("TARGET_DB=#{database} bin/setup")
-    sh("TARGET_DB=#{database} bin/rails test")
-  end
-end