solid_queue 1.6.0 → 1.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d6cda6edb92a7805d6c2c0fc13b2649c2c2df26a4d65e555049efde022cb8d14
-  data.tar.gz: c6d0036cbeb743e56faa4ed9fe6756f1a8e3ae3705a402f55440bb14cb3174dc
+  metadata.gz: 15044fdadc1f5170f909ebfeeb3f0d31e948bb51edb336aa040ad477b9d12545
+  data.tar.gz: 566ab7ad221caa7c96ea1f755c8e5537ac4e37304afbb7ced6697eb1a120033b
 SHA512:
-  metadata.gz: 5f7420684f8aa122d12e2314aa2445fb31a3f18f2a7bc53b4ceffccccd131a7f6d9408ef776da33dbbfe437be4c835437524279ac6150e57d134cf85698caf7a
-  data.tar.gz: cd02de9753ce345ec599d72230f634df0e633f278871df5da4330318fcb92b8820d99612f43fb8d89be348460d1cb198a5bfab34fdce5c6f639709ab8dd426a2
+  metadata.gz: 0744c1361eddc92f7cc24f26d3ca6194c7a06474cc2e3a287b3935e86b6e4dbb88c595b70ef63327431f687b8578770805a68cb33729aecb1ef7ca895b3d8e07
+  data.tar.gz: 2a40a741c8269c3a31baec31970addfc268d65f9d3bdc6130b75a1e908f22ecce2a0786b34c8dfd6c88cb92a024ca3fb84709f6d1c79e4f00022880ebd55056f

data/README.md

@@ -30,6 +30,12 @@ Solid Queue can be used with SQL databases such as MySQL, PostgreSQL, or SQLite,
   - [Performance considerations](#performance-considerations)
 - [Failed jobs and retries](#failed-jobs-and-retries)
   - [Error reporting on jobs](#error-reporting-on-jobs)
+  - [Jobs interrupted by non-graceful process death](#jobs-interrupted-by-non-graceful-process-death)
+- [Batch jobs](#batch-jobs)
+  - [Batch progress and counters](#batch-progress-and-counters)
+  - [Batch maintenance](#batch-maintenance)
+  - [Clearing batches](#clearing-batches)
+  - [Upgrading existing installations](#upgrading-existing-installations)
 - [Puma plugin](#puma-plugin)
 - [Jobs and transactional integrity](#jobs-and-transactional-integrity)
 - [Recurring tasks](#recurring-tasks)
@@ -287,6 +293,7 @@ It is recommended to set this value less than or equal to the queue database's c
   Fiber workers require fiber-scoped isolated execution state. In Rails apps, set `config.active_support.isolation_level = :fiber` before using `fibers`. Solid Queue refuses to boot fiber workers when isolation remains thread-scoped. On Rails 7.2 and later, a practical starting point is usually `3-5` queue database connections per worker process rather than matching the `fibers` value, because ordinary Active Record query paths can release connections between non-blocking waits. On Rails 7.1, size the queue database pool more conservatively, as in-flight fiber jobs may still retain connections roughly in proportion to `fibers`.
 - `processes`: this is the number of worker processes that will be forked by the supervisor with the settings given. By default, this is `1`, just a single process. This setting is useful if you want to dedicate more than one CPU core to a queue or queues with the same configuration. Only workers have this setting. This works with both `threads` and `fibers` workers as long as the supervisor is running in the default `fork` mode. **Note**: this option is ignored only when the supervisor itself is [running in `async` mode](#fork-vs-async-mode).
 - `concurrency_maintenance`: whether the dispatcher will perform the concurrency maintenance work. This is `true` by default, and it's useful if you don't use any [concurrency controls](#concurrency-controls) and want to disable it or if you run multiple dispatchers and want some of them to just dispatch jobs without doing anything else.
+- `batch_maintenance`: whether the dispatcher will sweep stalled [batches](#batch-jobs) as part of its maintenance work, on the same timer as concurrency maintenance (see [batch maintenance](#batch-maintenance)). This is `true` by default; disable it if you don't use batches, or if you run multiple dispatchers and want only some of them doing maintenance work.
 
 
 ### Optional scheduler configuration
@@ -622,7 +629,7 @@ production:
 Or something similar to that depending on your setup. You can also assign a different queue to a job on the moment of enqueuing so you can decide whether to enqueue a job in the throttled queue or another queue depending on the arguments, or pass a block to `queue_as` as explained [here](https://guides.rubyonrails.org/active_job_basics.html#queues).
 
 
-In addition, mixing concurrency controls with **bulk enqueuing** (Active Job's `perform_all_later`) is not a good idea because concurrency controlled job needs to be enqueued one by one to ensure concurrency limits are respected, so you lose all the benefits of bulk enqueuing.
+In addition, mixing concurrency controls with **bulk enqueuing** (Active Job's `perform_all_later`) has no benefit because concurrency-controlled jobs need to be enqueued one by one to ensure concurrency limits are respected, so you lose all the benefits of bulk enqueuing.
 
 When jobs that have concurrency controls and `on_conflict: :discard` are enqueued in bulk, the ones that fail to be enqueued and are discarded would have `successfully_enqueued` set to `false`. The total count of jobs enqueued returned by `perform_all_later` will exclude these jobs as expected.
 
@@ -661,9 +668,134 @@ class ApplicationMailer < ActionMailer::Base
     Rails.error.report(exception)
     raise exception
   end
+```
+
+### Jobs interrupted by non-graceful process death
+
+When a process dies without a clean shutdown (for example, `SIGKILL`ed by the OS or the container runtime because of memory limits), the jobs it was running can't be released back to their queues. Once another process notices the missing heartbeats and prunes the dead process's registration, its in-flight jobs are marked as failed with `SolidQueue::Processes::ProcessPrunedError`. Solid Queue deliberately doesn't retry these automatically: the job itself might be what's killing the process (for example, a job that exhausts the container's memory), and retrying it blindly would just kill the next worker too.
+
+If you know your jobs are idempotent and want to implement your own recovery policy, you can subscribe to the `fail_many_claimed.solid_queue` event, which includes the error and the affected job IDs in its payload:
+
+```ruby
+# config/initializers/solid_queue_recovery.rb
+ActiveSupport::Notifications.subscribe("fail_many_claimed.solid_queue") do |event|
+  if event.payload[:error].is_a?(SolidQueue::Processes::ProcessPrunedError)
+    SolidQueue::FailedExecution.where(job_id: event.payload[:job_ids]).each do |failed_execution|
+      # Apply your own safeguard against retrying the same job in a loop,
+      # e.g. a counter in the job's arguments or a cap stored elsewhere.
+      failed_execution.retry
+    end
+  end
+end
+```
+
+The event is emitted in the process that performs the pruning (or the supervisor when it reaps a crashed fork, with `SolidQueue::Processes::ProcessExitError`), so make sure the subscription is set up in an initializer, where all Solid Queue processes will load it.
+
+## Batch jobs
+
+Solid Queue supports grouping jobs into batches, so you can track the progress of the set as a whole and optionally fire callbacks based on its status. Batches support the following:
+
+- Relating jobs to a batch, to track their status
+- Three available callbacks to fire:
+  - `on_finish`: fired when all jobs have finished, including retries, even when some jobs have failed.
+  - `on_success`: fired when all jobs have succeeded, including retries. It won't fire if any jobs have failed, but it will fire if jobs have been discarded using `discard_on`.
+  - `on_failure`: fired when all jobs have finished, including retries, and one or more of them have failed.
+- Enqueuing more jobs for a batch from inside one of its jobs, with `batch.enqueue`
+- Attaching a description and arbitrary metadata to a batch
+
+Callback jobs are regular jobs: the batch doesn't pass them any arguments (although you can configure your own), and they can access the batch they belong to through the `batch` accessor:
+
+```ruby
+class SleepyJob < ApplicationJob
+  def perform(seconds_to_sleep)
+    Rails.logger.info "Feeling #{seconds_to_sleep} seconds sleepy..."
+    sleep seconds_to_sleep
+  end
+end
+
+class BatchFinishJob < ApplicationJob
+  def perform
+    Rails.logger.info "Finished all #{batch.total_jobs} jobs"
+  end
+end
+
+class BatchSuccessJob < ApplicationJob
+  def perform
+    Rails.logger.info "All #{batch.completed_jobs} jobs worked!"
+  end
+end
+
+class BatchFailureJob < ApplicationJob
+  def perform
+    Rails.logger.info "#{batch.failed_jobs} jobs failed, sorry!"
+  end
+end
+
+SolidQueue::Batch.enqueue(
+  on_finish: BatchFinishJob,
+  on_success: BatchSuccessJob,
+  on_failure: BatchFailureJob,
+  user_id: 123
+) do
+  5.times { |i| SleepyJob.perform_later(i) }
 end
 ```
 
+A job joins the batch that's active *when its enqueue is requested*—this also works when Rails defers the actual enqueue until after the surrounding transaction commits. In particular:
+
+- A job created outside a batch and enqueued inside one joins that batch.
+- Creating a job inside a batch without enqueueing it doesn't keep the batch open: if the batch finishes before the job is finally enqueued, the enqueue raises `SolidQueue::Batch::AlreadyFinished`.
+- If a job already carries a batch ID but is enqueued inside another active batch, the active batch takes precedence.
+
+Besides the callbacks, `SolidQueue::Batch.enqueue` accepts a `description:`, to label the batch, and a `metadata:` hash; any other keyword arguments (like `user_id: 123` above) are merged into the batch's `metadata`.
+
+Callbacks can be given as a job class or as a configured job instance—for example, `on_finish: BatchFinishJob.new.set(queue: :batches)` or `on_success: BatchSuccessJob.new("some argument")`. Note that the job is serialized when the batch is created, so options resolved at that point (like `wait_until:` timestamps) are relative to batch creation, not to when the callback is eventually enqueued.
+
+Callback jobs always enqueue through Solid Queue, even when the job classes involved (or the application default) use a different Active Job adapter. And a batch that ends up with no jobs finishes as soon as it starts, firing its callbacks right away.
+
+### Batch progress and counters
+
+Batches track `total_jobs`, `completed_jobs`, `failed_jobs` and `pending_jobs`, plus a `progress_percentage` helper. A couple of accounting details to be aware of:
+
+- Counters track *logical* jobs, matching what you enqueued: a retry via `retry_on` keeps the job's Active Job ID, so a job that fails twice and then succeeds still contributes 1 to `total_jobs`. Each attempt does get its own row in the batch's `jobs` relation, though.
+- Jobs discarded via `discard_on`, concurrency's `on_conflict: :discard`, or manual discarding count as completed, not failed.
+- Manually retrying a failed job (via `SolidQueue::FailedExecution#retry`) doesn't re-add it to its batch: if the batch already finished as failed, a successful manual retry won't change the batch's status.
+
+### Batch maintenance
+
+Batch completion is normally detected as jobs finish, without ever locking the batch row outside a single once-per-batch moment. A few edge cases can't trigger that detection: jobs removed via bulk discards (which delete jobs without callbacks), a process that crashed after enqueueing jobs but before starting its batch, or a completion whose callback enqueueing failed and rolled back.
+
+The dispatcher sweeps these up automatically via `SolidQueue::Batch.sweep_stalled`, as part of its regular maintenance (every `concurrency_maintenance_interval` seconds, sharing a single maintenance timer and database connection). If you disable `batch_maintenance` (or don't run a dispatcher), you can run the sweep yourself, for example as a [recurring task](#recurring-tasks):
+
+```yml
+batch_maintenance:
+  command: "SolidQueue::Batch.sweep_stalled"
+  schedule: every 5 minutes
+```
+
+### Clearing batches
+
+Finished, non-failed batches are cleared with `SolidQueue::Batch.clear_finished_in_batches` after `config.solid_queue.clear_finished_jobs_after`, but only when you invoke it. Failed batches are kept, like failed jobs, so you can inspect them. Installing Solid Queue configures [a recurring task](#recurring-tasks) that clears finished jobs every hour; you can add a matching entry for batches to your `recurring.yml`:
+
+```yml
+clear_solid_queue_finished_batches:
+  command: "SolidQueue::Batch.clear_finished_in_batches(sleep_between_batches: 0.3)"
+  schedule: every hour at minute 12
+```
+
+### Upgrading existing installations
+
+If you installed Solid Queue before batches existed, copy the migration that adds the new tables to your app and run it:
+
+```bash
+bin/rails solid_queue:update
+bin/rails db:migrate
+```
+
+Until you do, Solid Queue works exactly as before—jobs enqueue and run without any batch bookkeeping, trying to start a batch raises, and the dispatcher logs a deprecation warning to remind you the migration is pending. It becomes part of the base schema in Solid Queue 2.0.
+
+The copied migration is yours to adapt: if you're on PostgreSQL with a large jobs table, consider building the jobs index concurrently—`algorithm: :concurrently` on its `add_index`, with `disable_ddl_transaction!` on the migration—so the build doesn't block enqueues while it runs. Everything in the migration skips what already exists, so it's safe to rerun after a failure; just drop the invalid index a failed concurrent build leaves behind first.
+
 ## Puma plugin
 
 We provide a Puma plugin if you want to run the Solid Queue's supervisor together with Puma and have Puma monitor and manage it. You just need to add

data/UPGRADING.md

@@ -1,3 +1,15 @@
+# Upgrading to version 1.7.x
+This version introduces support for grouping jobs into batches, which needs new tables. Fresh installs get them with the base schema; existing installations need to copy the migration that adds them and run it:
+
+```bash
+bin/rails solid_queue:update
+bin/rails db:migrate
+```
+
+The migration is optional for now: until you run it, everything works as before, batches aside. It will become part of the required schema in Solid Queue 2.0.
+
+The copied migration is yours to adapt—for example, on PostgreSQL with a large jobs table, you can build the jobs index concurrently (`algorithm: :concurrently` with `disable_ddl_transaction!`) so it doesn't block enqueues while it runs.
+
 # Upgrading to version 1.5.x
 Ruby 3.1 is no longer supported, as it reached end-of-life in March 2025. Solid Queue now requires Ruby 3.2 or newer. If you're still on Ruby 3.1, Bundler will continue to resolve solid_queue 1.4.x for you, but you won't receive any new versions until you upgrade Ruby.
 

data/app/models/solid_queue/batch/callbacks.rb

@@ -0,0 +1,50 @@
+# frozen_string_literal: true
+
+module SolidQueue
+  class Batch
+    module Callbacks
+      extend ActiveSupport::Concern
+
+      included do
+        %w[ finish success failure ].each do |callback_type|
+          serialize "on_#{callback_type}", coder: JSON
+
+          define_method("on_#{callback_type}=") do |callback|
+            super serialize_callback(callback)
+          end
+        end
+      end
+
+      private
+        def serialize_callback(value)
+          if value.present?
+            active_job = value.is_a?(ActiveJob::Base) ? value : value.new
+            # We can pick up batch ids from context, but callbacks should never be considered a part of the batch
+            active_job.batch_id = nil
+            active_job.serialize
+          end
+        end
+
+        def enqueue_callback_jobs
+          if failed? then enqueue_callback_job(:on_failure)
+          else
+            enqueue_callback_job(:on_success)
+          end
+
+          enqueue_callback_job(:on_finish)
+        end
+
+        def enqueue_callback_job(callback_name)
+          if callback = send(callback_name)
+            active_job = ActiveJob::Base.deserialize(callback)
+            active_job.callback_batch_id = id
+            # Bypass the job class's adapter so callbacks stay in Solid Queue and
+            # their enqueue stays in this transaction, while honoring enqueue callbacks.
+            active_job.run_callbacks(:enqueue) do
+              Job.enqueue(active_job, scheduled_at: active_job.scheduled_at || Time.current)
+            end
+          end
+        end
+    end
+  end
+end

data/app/models/solid_queue/batch/clearable.rb

@@ -0,0 +1,23 @@
+# frozen_string_literal: true
+
+module SolidQueue
+  class Batch
+    module Clearable
+      extend ActiveSupport::Concern
+
+      included do
+        scope :clearable, ->(finished_before: SolidQueue.clear_finished_jobs_after.ago) { succeeded.where(finished_at: ...finished_before) }
+      end
+
+      class_methods do
+        def clear_finished_in_batches(batch_size: 500, finished_before: SolidQueue.clear_finished_jobs_after.ago, sleep_between_batches: 0)
+          loop do
+            records_deleted = clearable(finished_before: finished_before).limit(batch_size).delete_all
+            sleep(sleep_between_batches) if sleep_between_batches > 0
+            break if records_deleted == 0
+          end
+        end
+      end
+    end
+  end
+end

data/app/models/solid_queue/batch/status.rb

@@ -0,0 +1,64 @@
+# frozen_string_literal: true
+
+module SolidQueue
+  class Batch
+    module Status
+      extend ActiveSupport::Concern
+
+      included do
+        scope :finished, -> { where.not(finished_at: nil) }
+        scope :succeeded, -> { finished.where(failed_at: nil) }
+        scope :unfinished, -> { where(finished_at: nil) }
+        scope :failed, -> { where.not(failed_at: nil) }
+        scope :enqueued, -> { where.not(enqueued_at: nil) }
+      end
+
+      def status
+        if finished?
+          failed? ? :failed : :completed
+        elsif enqueued?
+          :enqueued
+        else
+          :pending
+        end
+      end
+
+      def failed?
+        failed_at.present?
+      end
+
+      def succeeded?
+        finished? && !failed?
+      end
+
+      def finished?
+        finished_at.present?
+      end
+
+      def enqueued?
+        enqueued_at.present?
+      end
+
+      # Failed jobs no longer have tracking rows, so exclude them from the completed count.
+      def completed_jobs
+        finished? ? self[:completed_jobs] : [ total_jobs - pending_jobs - failed_jobs, 0 ].max
+      end
+
+      def failed_jobs
+        finished? ? self[:failed_jobs] : jobs.failed.count
+      end
+
+      # Pending counts attempts, not logical jobs: while a retry is enqueued
+      # and its previous attempt hasn't finished yet, both have tracking rows,
+      # so the counts derived from it clamp at the logical totals.
+      def pending_jobs
+        finished? ? 0 : batch_executions.count
+      end
+
+      def progress_percentage
+        return 0 if total_jobs == 0
+        ([ total_jobs - pending_jobs, 0 ].max * 100.0 / total_jobs).round(2)
+      end
+    end
+  end
+end

data/app/models/solid_queue/batch/sweepable.rb

@@ -0,0 +1,64 @@
+# frozen_string_literal: true
+
+module SolidQueue
+  class Batch
+    # Repairs batches that the regular completion detection can't finish on
+    # its own: jobs removed via bulk discards, processes that crashed after
+    # enqueueing jobs but before starting their batch, or completions whose
+    # callback enqueueing failed and rolled back.
+    module Sweepable
+      extend ActiveSupport::Concern
+
+      class_methods do
+        def sweep_stalled(stalled_for: 5.minutes, batch_size: 500)
+          SolidQueue.instrument(:sweep_stalled_batches, stalled_for: stalled_for, stale_executions: 0, finished_batches: 0, started_batches: 0) do |payload|
+            payload[:stale_executions] = sweep_stale_executions(batch_size:)
+            payload[:finished_batches] = finish_stalled_batches(batch_size:)
+            payload[:started_batches] = start_stalled_batches(stalled_for:, batch_size:)
+          end
+        end
+
+        private
+          # BatchExecution rows represent outstanding work. A row for a resolved
+          # job violates that invariant, so remove it immediately; destroy's
+          # after_commit callback retries the batch completion check.
+          def sweep_stale_executions(batch_size:)
+            swept = 0
+
+            [ BatchExecution.with_finished_jobs, BatchExecution.with_failed_jobs ].each do |stale|
+              stale.find_each(batch_size: batch_size) do |batch_execution|
+                swept += 1
+                batch_execution.destroy
+              end
+            end
+
+            swept
+          end
+
+          # A started batch with no tracking rows left can finish
+          def finish_stalled_batches(batch_size:)
+            finished = 0
+
+            unfinished.enqueued.without_executions.find_each(batch_size: batch_size) do |batch|
+              finished += 1
+              batch.finish
+            end
+
+            finished
+          end
+
+          # A batch that crashed between creation and start never got enqueued
+          def start_stalled_batches(stalled_for:, batch_size:)
+            started = 0
+
+            unfinished.where(enqueued_at: nil).where(created_at: ...stalled_for.ago).find_each(batch_size: batch_size) do |batch|
+              started += 1
+              batch.start
+            end
+
+            started
+          end
+      end
+    end
+  end
+end

data/app/models/solid_queue/batch.rb

@@ -0,0 +1,133 @@
+# frozen_string_literal: true
+
+module SolidQueue
+  class Batch < Record
+    class AlreadyFinished < StandardError; end
+
+    class PendingMigrations < StandardError
+      def initialize(message = "The batches schema hasn't been installed yet. Run `bin/rails solid_queue:update` to copy the pending migrations to your application, and then `bin/rails db:migrate` to run them")
+        super
+      end
+    end
+
+    include Callbacks, Status
+    include Clearable, Sweepable
+
+    has_many :jobs
+    has_many :batch_executions, dependent: :destroy
+
+    store :metadata, coder: JSON
+
+    # Join-free so update_all keeps this condition in the completion update's own WHERE
+    scope :without_executions, -> { where.not(id: BatchExecution.select(:batch_id)) }
+
+    # Provider-agnostic batch identifier, analogous to jobs.active_job_id.
+    before_create :set_active_job_batch_id
+    after_commit :start, on: :create, unless: -> { ActiveRecord.respond_to?(:after_all_transactions_commit) }
+
+    class << self
+      # The batches schema ships as an optional migration in Solid Queue 1.x
+      # and becomes part of the base schema in 2.0. Until the app has run the
+      # migration, jobs enqueue without any batch bookkeeping and batches
+      # themselves can't be used.
+      def migrated?
+        @migrated ||= table_exists? && BatchExecution.table_exists? && Job.column_names.include?("batch_id")
+      end
+
+      def enqueue(description: nil, on_success: nil, on_failure: nil, on_finish: nil, metadata: nil, **extra_metadata, &block)
+        raise PendingMigrations unless migrated?
+
+        new.tap do |batch|
+          batch.assign_attributes(description:, on_success:, on_failure:, on_finish:, metadata: (metadata || {}).merge(extra_metadata))
+          batch.enqueue(&block)
+        end
+      end
+
+      def current_batch_id
+        ActiveSupport::IsolatedExecutionState[:current_batch_id]
+      end
+
+      def wrap_in_batch_context(batch_id)
+        previous_batch_id = current_batch_id.presence
+        ActiveSupport::IsolatedExecutionState[:current_batch_id] = batch_id
+        yield
+      ensure
+        ActiveSupport::IsolatedExecutionState[:current_batch_id] = previous_batch_id
+      end
+    end
+
+    def enqueue(&block)
+      # Fast-fail for the common case. create_all_from_jobs atomically guards
+      # concurrent additions when it creates their tracking rows.
+      if finished?
+        raise AlreadyFinished, "Can't enqueue an already finished batch"
+      end
+
+      transaction do
+        save! if new_record?
+
+        self.class.wrap_in_batch_context(id) { block&.call(self) }
+
+        if ActiveRecord.respond_to?(:after_all_transactions_commit)
+          ActiveRecord.after_all_transactions_commit { start }
+        end
+      end
+    end
+
+    def metadata
+      (super || {}).with_indifferent_access
+    end
+
+    def start
+      mark_as_enqueued
+
+      # Refresh enqueued_at after marking as enqueued, and let a batch that started
+      # with no jobs finish right away
+      reload
+      finish
+    end
+
+    def finish
+      return if finished? || !enqueued?
+      return if batch_executions.exists?
+
+      transaction do
+        updated = Batch.where(id: id).unfinished.enqueued.without_executions.update_all(finished_at: Time.current)
+        finalize if updated > 0
+      end
+    end
+
+    private
+      def set_active_job_batch_id
+        self.active_job_batch_id ||= SecureRandom.uuid
+      end
+
+      def mark_as_enqueued
+        Batch.where(id: id, enqueued_at: nil).update_all(enqueued_at: Time.current)
+      end
+
+      def finalize
+        reload
+
+        # PostgreSQL can let a blocked CAS win from a stale NOT EXISTS snapshot:
+        # after a lock wait, READ COMMITTED re-checks the target row's conditions
+        # against the latest data but keeps the original snapshot for subqueries.
+        # Re-check in a new statement, which gets a fresh snapshot while this
+        # transaction's row lock keeps adders out, since they increment before
+        # inserting their executions. MySQL doesn't need this: it reads DML
+        # subqueries from the latest committed data, so its CAS can't win wrongly.
+        raise ActiveRecord::Rollback if batch_executions.exists?
+
+        SolidQueue.instrument(:finish_batch, batch_id: id) do |payload|
+          failed_jobs = jobs.failed.count
+          failed_at = Time.current if failed_jobs > 0
+          completed_jobs = total_jobs - failed_jobs
+
+          update_columns(failed_jobs:, failed_at:, completed_jobs:)
+          enqueue_callback_jobs
+
+          payload.merge!(total_jobs:, failed_jobs:, completed_jobs:)
+        end
+      end
+  end
+end

data/app/models/solid_queue/batch_execution.rb

@@ -0,0 +1,52 @@
+# frozen_string_literal: true
+
+module SolidQueue
+  class BatchExecution < Execution
+    self.assumable_attributes_from_job = [ :batch_id ]
+
+    belongs_to :batch
+
+    scope :with_finished_jobs, -> { joins(:job).merge(SolidQueue::Job.finished) }
+    scope :with_failed_jobs, -> { joins(job: :failed_execution) }
+
+    after_commit :finish_batch, on: :destroy
+
+    class << self
+      def create_all_from_jobs(jobs)
+        jobs.select(&:batched?).group_by(&:batch_id).each do |batch_id, jobs_in_batch|
+          # Update the counter first: inserting tracking rows takes a shared FK lock on
+          # the batch row, then incrementing can deadlock concurrent MySQL adders.
+          if attempt_to_update_total_jobs(batch_id, jobs_in_batch)
+            super jobs_in_batch
+          else
+            raise Batch::AlreadyFinished, "Can't add jobs into an already finished batch"
+          end
+        end
+      end
+
+      private
+        def attempt_to_update_total_jobs(batch_id, jobs)
+          new_jobs_count = count_new_jobs_among(jobs)
+          updated = SolidQueue::Batch.where(id: batch_id).unfinished.update_all([ "total_jobs = total_jobs + ?", new_jobs_count ])
+          updated > 0
+        end
+
+        # A job that has executed before was already counted when it first joined
+        # the batch: retries keep their active_job_id and batch across re-enqueues.
+        # This might undercount jobs whose retries switch to another batch, but that
+        # should be a rare enough case. The counter is used only for report/info, so
+        # we favour simplicity here
+        def count_new_jobs_among(jobs)
+          jobs.reject { |job| job.arguments["executions"].to_i > 0 }.map(&:active_job_id).uniq.size
+        end
+    end
+
+    private
+      def finish_batch
+        # Skip the serialized callback and metadata columns on this hot path
+        if batch = Batch.select(:id, :finished_at, :enqueued_at).find_by(id: batch_id)
+          batch.finish
+        end
+      end
+  end
+end

data/app/models/solid_queue/claimed_execution.rb

@@ -48,6 +48,7 @@ class SolidQueue::ClaimedExecution < SolidQueue::Execution
           payload[:process_ids] = executions.map(&:process_id).uniq
           payload[:job_ids] = executions.map(&:job_id).uniq
           payload[:size] = executions.size
+          payload[:error] = error
         end
       end
     end

data/app/models/solid_queue/failed_execution/batchable.rb

@@ -0,0 +1,22 @@
+# frozen_string_literal: true
+
+module SolidQueue
+  class FailedExecution
+    # A FailedExecution is created only after retries are exhausted, when the
+    # job stops counting as pending in its batch.
+    module Batchable
+      extend ActiveSupport::Concern
+
+      included do
+        after_create :destroy_job_batch_execution, if: -> { Batch.migrated? && job.batch_id? }
+      end
+
+      private
+        def destroy_job_batch_execution
+          job.batch_execution&.destroy!
+        rescue ActiveRecord::ActiveRecordError => e
+          SolidQueue.instrument(:batch_progress_error, batch_id: job.batch_id, job_id: job.id, error: e)
+        end
+    end
+  end
+end

data/app/models/solid_queue/failed_execution.rb

@@ -2,7 +2,7 @@
 
 module SolidQueue
   class FailedExecution < Execution
-    include Dispatching
+    include Dispatching, Batchable
 
     serialize :error, coder: JSON