solid_queue-coalesce 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 3f395a95881f8639b6abcee94b8e161395cf3026618f63c92ca9612540b30bf0
+  data.tar.gz: e5d16c7e02d2a5ae34d18826dfe5dabc42f8ccfcb1802a247fad8eebdd56926e
+SHA512:
+  metadata.gz: 6acaf7eca6a5915de50cbe9aca92b7c042635de9f32a44e0642927a957bdc2855a691c42d883758bceb8ef4cd6250525b9d3ac633b39b7150c2ecd1a207ac1d7
+  data.tar.gz: 11ec3d1ef332b093e1338c6e0dc24ad7ce2ea250c6cc0951981ee637f64908d586885d9dd244034d3abc5842426e6f57bcd30ca479063b4204e261b3065157ad

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) David Cristofaro
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,170 @@
+# Solid Queue Coalesce
+
+Coalescing for [Solid Queue](https://github.com/rails/solid_queue). Many `perform_later` calls with the same key share one pending job, and that key is held only until the job starts. A call that arrives after that opens a new job, so anything that happens after a run has started is always covered by another run.
+
+It's a separate gem that adds to Solid Queue from the outside, with its own tables. It only works with the Solid Queue adapter, because a foreign key to the job and a shared transaction are what let it work without a time window to configure.
+
+## Table of Contents
+
+- [Installation](#installation)
+- [Usage](#usage)
+- [Collecting calls](#collecting-calls)
+- [Retries and partial success](#retries-and-partial-success)
+- [Discarding and retention](#discarding-and-retention)
+- [Rules and limitations](#rules-and-limitations)
+- [Monitoring](#monitoring)
+- [Testing](#testing)
+- [License](#license)
+
+## Installation
+
+Add the gem to your application's Gemfile, alongside Solid Queue:
+
+```ruby
+gem "solid_queue-coalesce"
+```
+
+Then copy its migration to your queue database's migrations path and run it:
+
+```bash
+bundle install
+bin/rails generate solid_queue:coalesce:install
+bin/rails db:migrate
+```
+
+If Solid Queue doesn't use a database called `queue`, pass the right one with `--database`. For example, `bin/rails generate solid_queue:coalesce:install --database primary`.
+
+Until the migration has run, everything else works as before, and enqueuing a job that uses `coalesce` raises `SolidQueue::CoalesceKey::PendingMigrations`.
+
+The gem requires Solid Queue 1.7. It relies on private Solid Queue methods, so its gemspec allows one minor version at a time.
+
+A dispatcher with `concurrency_maintenance` on, which is the default, is what releases a coalesce key left on a job that finished or failed. If you run several dispatchers, at least one needs it.
+
+## Usage
+
+With coalescing, many `perform_later` calls with the same key share one pending job. The key has to be open for this to happen: it's taken when the first call creates a job, and it's free again the moment that job starts. A call that arrives after that opens a new job, so anything that happens after a run has started is always covered by another run.
+
+```ruby
+class RefreshAccountJob < ApplicationJob
+  coalesce key: ->(account) { account }
+
+  def perform(account)
+    account.refresh!
+  end
+end
+```
+
+`key` is the only required parameter, and it works like the one in `limits_concurrency`: a symbol, a string or a proc that receives the job arguments. If the proc returns an Active Record record, the key is built from its class name and `id`. Keys are always scoped to the job class name.
+
+By default a call that finds an open job for its key is dropped. It sets `coalesced?` to `true` on the Active Job instance and `provider_job_id` to the job it found, and emits a `coalesce.solid_queue` event with `key`, `job_id`, `active_job_id` and `action` (`:dropped` or `:collected`). Bear in mind that Active Job sets `successfully_enqueued?` to `true` after `perform_later` regardless, so `coalesced?` is what tells you the call was dropped. `ActiveJob.perform_all_later` counts dropped calls as not enqueued, as it does for `on_conflict: :discard`.
+
+The first call's schedule, queue and priority win. A later call's `wait:` doesn't move the open job.
+
+Coalescing doesn't change `limits_concurrency`, and you can use both in the same job. The concurrency key is the one computed for the call that created the job, and a job that's blocked still holds its coalesce key, since it hasn't started. If `on_conflict: :discard` destroys the job, that frees the key.
+
+### Collecting calls
+
+With `collect: true`, no call is dropped. Each one stores its arguments, and the job receives them all:
+
+```ruby
+class ReindexDocumentsJob < ApplicationJob
+  coalesce key: ->(document) { document.account_id }, collect: true, limit: 100
+
+  def perform(documents)
+    Search.index(documents)
+  end
+end
+```
+
+`perform` receives an array with one element per call, in the order they were stored. For a call with one argument the element is that argument, as above. For a call with several it's an array of them, and keyword arguments are a hash at the end of that array, so `perform_later(document, force: true)` gives `[ document, { force: true } ]`.
+
+`arguments` isn't part of this: it keeps holding the arguments of the call that created the job, which is what callbacks and the `limits_concurrency` key see.
+
+`limit` is 100 by default, and each run gets at most that many calls. If more are waiting when a job starts, it takes the oldest `limit` and a new job is created for the rest, in the same queue and with the same priority and concurrency key. It's due when the oldest call left asked to run, or right away if at least `limit` calls are left, unless you turn off `flush_when_full` (see below).
+
+Reaching `limit` also flushes a job that's scheduled: it's dispatched right away instead of waiting for its time. So you can use `wait:` to collect calls for a while, with `limit` as the point where you stop waiting. Concurrency limits apply to a flushed job as usual, and jobs already ready or blocked aren't affected. A call that arrives after a flush still has its own `wait:` stored, so if it ends up in the next job, that's the time the next job will run. Concurrent calls can leave slightly more than `limit` calls waiting, but that doesn't matter because the limit is enforced when the job starts.
+
+Use `flush_when_full: false` for a job that has to wait for its time, such as one that runs at a fixed time of day. Solid Queue can't work this out on its own, because `wait:` and `wait_until:` both end up as the same `scheduled_at`.
+
+```ruby
+class SendDigestJob < ApplicationJob
+  coalesce key: ->(entry) { entry.account_id }, collect: true, limit: 100, flush_when_full: false
+
+  def self.perform_tonight(entry)
+    set(wait_until: Time.current.tomorrow.change(hour: 2)).perform_later(entry)
+  end
+
+  def perform(entries)
+    DigestMailer.with(entries: entries).digest.deliver_now
+  end
+end
+```
+
+With this option, reaching `limit` doesn't dispatch the job, and it stays scheduled. When it starts with more than `limit` calls waiting, the new job for the rest is due when the oldest call left asked to run, however many are left. So a backlog that's already due drains in batches of `limit`, one after another, and calls dated in the future wait for their time.
+
+The first call since the previous job started sets the schedule, and later calls' times don't move an open job. A later call that joins a job still waiting to start runs with that job, even if it asked for a later time. A call that arrives after the last batch started opens a new job for its own time. This means any caller that might arrive first needs the right time: a plain `perform_later` would open a job that runs right away. That's why the example wraps `set(wait_until:)` in a class method.
+
+If you alert on the age of the oldest row in `solid_queue_coalesced_arguments`, allow for the wait you expect, because these calls sit there until their time comes.
+
+Each call is deserialized on its own. If a call refers to a record that has been deleted, that call is deleted and a `drop_coalesced_argument.solid_queue` event is emitted, while the others run. If every call refers to a deleted record, the events are emitted, `perform` is skipped and the job finishes normally. Those calls stay until the job is destroyed, so a re-run after a crash skips `perform` too. Any other deserialization error fails the job and keeps all of its calls.
+
+### Retries and partial success
+
+A retry is the same work continuing. It never coalesces: it can't be dropped because another job is open for its key, and later calls can't join it. A job that's retried or resumed keeps the exact calls it started with, including after `retry_on`, `rescue_from` with `retry_job`, and `ActiveJob::Continuable`. If the retry can't be enqueued, the job fails and keeps its calls. Retrying a failed job manually runs the same calls again, and so does a job that gets released after its worker was shut down.
+
+To avoid repeating calls that have already been processed, use `ActiveJob::Continuable` with a cursor. Use a record's `id` as the cursor instead of a position in the array, because calls for deleted records are dropped between runs and positions move:
+
+```ruby
+class ReindexDocumentsJob < ApplicationJob
+  include ActiveJob::Continuable
+
+  coalesce key: ->(document) { document.account_id }, collect: true
+
+  def perform(documents)
+    step :index do |step|
+      documents.sort_by(&:id).each do |document|
+        next if step.cursor && document.id <= step.cursor
+
+        Search.index(document)
+        step.set! document.id
+      end
+    end
+  end
+end
+```
+
+Coalescing has no per-call acknowledgement. Without a cursor a retry processes every call in the job again.
+
+### Discarding and retention
+
+Calls belong to their job, and are deleted with it. Discarding an open job deletes it, its key and every call waiting on it. If `preserve_finished_jobs` is on, a job's calls stay until the job is cleared, along with the finished job itself.
+
+A job that fails before starting, for example because its worker died or because its class no longer exists, takes care of its calls in the same way as if it had started. If the class is gone, there's no way to know the limit, so the failed job takes all the waiting calls. When you bring the class back and retry the job, it will run them all, even if that's more than `limit`.
+
+### Rules and limitations
+
+- `limit` and `flush_when_full` need `collect: true`, and `collect: true` can't be combined with `limits_concurrency` with `on_conflict: :discard`, because a conflict would destroy a job holding calls. Both raise `ArgumentError` when the class loads. Without `collect`, `on_conflict: :discard` is fine, and the job being discarded frees the key.
+- A coalesced job can't be enqueued inside `SolidQueue::Batch.enqueue`, either alone or in bulk. A call that joins an open job adds no job to the batch, which could finish early. Trying raises `SolidQueue::Job::CoalescedInBatchError`.
+- A recurring task can't use a job with `collect: true`: its configuration is invalid. Without `collect`, it works. A tick that finds an open job is dropped and isn't recorded.
+- Calling `perform_later` inside a transaction on the same database as Solid Queue is safe: a call for an open job doesn't abort the transaction. On MySQL, however, a deadlock rolls back the entire transaction, so we can only retry it automatically outside of one. Inside your own transaction it raises `SolidQueue::Job::EnqueueError`.
+- On MySQL, a deadlock while failing a job isn't retried, and it can stop the worker with the error Solid Queue raises for a job it can't finalize.
+- Keys longer than 255 characters fail on MySQL. `"a"` and `"a "` count as the same key there, because `utf8mb4_bin` pads spaces.
+- Concurrent calls that reach `limit` together can miss the early flush. The next call or the job's schedule covers it.
+- Removing `collect: true` from a job, or `coalesce` altogether, isn't safe while calls are waiting for it: they wouldn't be processed. Drain them first: stop enqueuing and wait for `SolidQueue::CoalescedArgument.waiting` to be empty before you deploy the change.
+
+### Monitoring
+
+Queue depth hides the backlog, because any number of calls waiting may be a single job. To see it, look at the calls instead:
+
+```ruby
+SolidQueue::CoalescedArgument.waiting.count
+SolidQueue::CoalescedArgument.waiting.minimum(:created_at) # nil when nothing is waiting
+```
+
+### Testing
+
+Everything above needs the Solid Queue adapter, so other adapters ignore `coalesce`. With the test adapter, the inline adapter or `perform_now`, each call runs on its own, and `perform` receives a batch of one built from that call's arguments. `assert_enqueued_with(args: [ document ])` still works, and the Solid Queue tables aren't touched.
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).

data/app/models/solid_queue/coalesce_key.rb

@@ -0,0 +1,132 @@
+# frozen_string_literal: true
+
+module SolidQueue
+  # A row means "this job is the open job for this key": it exists only while
+  # its job exists and hasn't started. job_id is null only between the insert and
+  # the end of the admission that inserted it, so it's never visible to others.
+  class CoalesceKey < Record
+    class PendingMigrations < StandardError
+      def initialize(message = "The coalescing schema hasn't been installed yet. Run `bin/rails generate solid_queue:coalesce:install` to copy the migration to your application, and then `bin/rails db:migrate` to run it")
+        super
+      end
+    end
+
+    belongs_to :job, optional: true
+    has_many :coalesced_arguments
+
+    scope :stranded, -> { left_joins(job: :failed_execution).merge(Job.finished.or(Job.where.not(failed_execution: { id: nil }))) }
+
+    class << self
+      # Until the app has run the gem's migration everything else works as before.
+      def migrated?
+        @migrated ||= table_exists? && CoalescedArgument.table_exists?
+      end
+
+      # Returns the row for this key, inserted if nobody holds it, and locked so
+      # a job start can't claim or delete it until this transaction commits. A row
+      # with a null job_id is ours: the caller has to create the job for it.
+      def admit(key)
+        loop do
+          insert_unless_held(key)
+
+          admitted = lock(share_lock).find_by(key: key)
+          return admitted if admitted
+        end
+      end
+
+      # Locks through the key's index, like admission. Going through the job_id
+      # index, even by deleting by it, takes InnoDB's locks on this row's indexes in
+      # the opposite order to admission, and that deadlocks.
+      def lock_for_release(job_id)
+        if key = where(job_id: job_id).pick(:key)
+          lock.find_by(key: key, job_id: job_id)
+        end
+      end
+
+      def release_stranded(batch_size)
+        stranded.limit(batch_size).pluck(:key).each do |key|
+          transaction do
+            coalesce_key = lock.find_by(key: key)
+            coalesce_key.release(claiming: !coalesce_key.job.finished?) if coalesce_key&.stranded?
+          end
+        end
+      end
+
+      private
+        # MariaDB and MySQL before 8 don't have FOR SHARE, and this means the same on MySQL 8
+        def share_lock
+          connection_pool.with_connection { |connection| connection.adapter_name == "PostgreSQL" } ? "FOR SHARE" : "LOCK IN SHARE MODE"
+        end
+
+        # Needs the unique index on key. Skips a conflict instead of raising, because
+        # a duplicate would abort an enclosing transaction on PostgreSQL. On MySQL it
+        # also avoids a deadlock: a plain INSERT that hits a duplicate takes a shared
+        # lock, and two of those on a row being deleted can't both go on to insert.
+        def insert_unless_held(key)
+          if supports_insert_conflict_target?
+            insert({ key: key }, unique_by: :key)
+          else
+            insert({ key: key })
+          end
+        end
+    end
+
+    def stranded?
+      job.finished? || job.failed?
+    end
+
+    def collect(active_jobs)
+      CoalescedArgument.insert_all(active_jobs.map { |active_job| argument_attributes_from(active_job) })
+    end
+
+    # Early flush. Bounded by the limit, however big the backlog is. A concurrent
+    # dispatcher holding the scheduled execution is dispatching it already, and if
+    # it's been dispatched before we got here there's nothing to lock.
+    def flush_if_full(limit)
+      if at_least?(limit)
+        scheduled = ScheduledExecution.where(job_id: job_id).non_blocking_lock.pluck(:job_id)
+        ScheduledExecution.dispatch_jobs(scheduled) if scheduled.any?
+      end
+    end
+
+    # Hands the waiting rows over as the job leaves the queue. Requires this row
+    # to be locked: that's what waits for in-flight appends, and it serializes this
+    # with other starts. Reads of the rows lock too, because on MySQL a plain read
+    # could use a snapshot older than the appends we waited for, and deleting this
+    # row with rows we didn't see would cascade to them.
+    #
+    # A finished job won't run again, so it claims nothing when there's a next job
+    # that can process the rows. Without a limit there can't be one: the class that
+    # would process them is gone, or stopped collecting.
+    def release(claiming: true)
+      limit = job.coalesce_limit
+      claim(claiming || limit.nil? ? limit : 0)
+
+      if oldest = coalesced_arguments.order(:id).lock.first
+        next_job = job.enqueue_next_coalesced(oldest, due_now: next_job_due_now?(limit))
+        update_columns(job_id: next_job.id)
+      else
+        destroy!
+      end
+    end
+
+    private
+      def next_job_due_now?(limit)
+        limit.present? && job.coalesce_flush_when_full? && at_least?(limit)
+      end
+
+      def argument_attributes_from(active_job)
+        { coalesce_key_id: id, arguments: active_job.serialize["arguments"], scheduled_at: active_job.scheduled_at }
+      end
+
+      def at_least?(count)
+        coalesced_arguments.order(:id).offset(count - 1).limit(1).pick(:id).present?
+      end
+
+      # A nil limit means we can't know it, so nothing is left behind
+      def claim(limit)
+        claimed = limit ? coalesced_arguments.where(id: coalesced_arguments.order(:id).limit(limit).lock.pluck(:id)) : coalesced_arguments
+        claimed.update_all(job_id: job_id, coalesce_key_id: nil)
+      end
+  end
+end

data/app/models/solid_queue/coalesced_argument.rb

@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+module SolidQueue
+  # The serialized arguments of one collected perform_later call. It waits on a
+  # CoalesceKey until a job starts and claims it, and from then on it belongs to
+  # that job, across retries, until the job is destroyed.
+  class CoalescedArgument < Record
+    belongs_to :coalesce_key, optional: true
+    belongs_to :job, optional: true
+
+    serialize :arguments, coder: JSON
+
+    scope :waiting, -> { where.not(coalesce_key_id: nil) }
+  end
+end

data/app/models/solid_queue/job/coalescible.rb

@@ -0,0 +1,205 @@
+# frozen_string_literal: true
+
+module SolidQueue
+  class Job
+    class CoalescedInBatchError < EnqueueError; end
+
+    module Coalescible
+      extend ActiveSupport::Concern
+
+      included do
+        has_many :coalesced_arguments
+      end
+
+      class_methods do
+        # Every call in active_jobs has the same key. Returns the job if the first
+        # call created one, nil if the calls joined the open job.
+        def enqueue_coalesced(key, active_jobs)
+          enqueued_job, open_job_id = raising_enqueue_errors do
+            retrying_deadlocks { transaction { admit_coalesced(key, active_jobs) } }
+          end
+
+          joined = enqueued_job ? active_jobs.drop(1) : active_jobs
+          record_coalesced(key, joined, open_job_id)
+
+          enqueued_job
+        end
+
+        def enqueue_all_coalesced(active_jobs)
+          active_jobs.group_by(&:coalesce_key).sort_by(&:first).each do |key, calls|
+            enqueue_coalesced(key, calls)
+          end
+        end
+
+        def ensure_coalescible(active_jobs)
+          raise CoalesceKey::PendingMigrations unless CoalesceKey.migrated?
+
+          if Batch.current_batch_id || active_jobs.any?(&:batch_id)
+            raise CoalescedInBatchError, "#{active_jobs.first.class.name} coalesces its calls, so it can't be enqueued as part of a batch: a call that joins an open job adds no job to the batch"
+          end
+        end
+
+        MAX_COALESCE_DEADLOCK_RETRIES = 5
+
+        # InnoDB can pick one of these transactions as a deadlock victim. Admissions
+        # waiting on a key that doesn't exist yet hold gap locks that block each other,
+        # creating a job can do the same on a new semaphore, and a job being destroyed
+        # cascades into the key's indexes in a different order. Nothing is lost by
+        # running ours again, unless it's part of a bigger transaction. In that case
+        # InnoDB has rolled all of it back, and only the caller can run it again.
+        def retrying_deadlocks
+          nested = connection_pool.with_connection(&:transaction_open?)
+          attempts = 0
+
+          begin
+            yield
+          rescue ActiveRecord::Deadlocked
+            raise if nested || (attempts += 1) > MAX_COALESCE_DEADLOCK_RETRIES
+            retry
+          end
+        end
+
+        private
+          # A retry of a collect job is a new job with the same active_job_id. The
+          # calls it owes move with it, so the old job finishes owning nothing. The
+          # block creates the new job, and sets provider_job_id to it.
+          def continuing_coalesced_run(active_job)
+            previous_job_id = active_job.provider_job_id
+            moving_calls = previous_job_id && active_job.class.coalesce_collect && CoalesceKey.migrated?
+
+            enqueued_job = raising_enqueue_errors do
+              transaction do
+                yield.tap do |job|
+                  CoalescedArgument.where(job_id: previous_job_id).update_all(job_id: job.id) if moving_calls && job.persisted?
+                end
+              end
+            end
+
+            # Active Job marks the call as enqueued once we return, so only raising
+            # stops on_conflict: :discard from silently dropping a coalesced job's retry
+            unless enqueued_job.persisted?
+              raise EnqueueError, "The retry of #{active_job.class.name} (#{active_job.job_id}) was discarded by its concurrency limit"
+            end
+
+            enqueued_job
+          end
+
+          def raising_enqueue_errors
+            yield
+          rescue ActiveRecord::ActiveRecordError => e
+            enqueue_error = EnqueueError.new("#{e.class.name}: #{e.message}").tap do |error|
+              error.set_backtrace e.backtrace
+            end
+            raise enqueue_error
+          end
+
+          def admit_coalesced(key, active_jobs)
+            job_class = active_jobs.first.class
+            coalesce_key = CoalesceKey.admit(key)
+            enqueued_job = create_for_coalesce_key(coalesce_key, active_jobs.first) if coalesce_key.job_id.nil?
+
+            if job_class.coalesce_collect && coalesce_key.job_id
+              coalesce_key.collect(active_jobs)
+              coalesce_key.flush_if_full(job_class.coalesce_limit) if job_class.coalesce_flush_when_full
+            end
+
+            [ enqueued_job, coalesce_key.job_id ]
+          end
+
+          def create_for_coalesce_key(coalesce_key, active_job)
+            create!(**attributes_from_active_job(active_job)).tap do |job|
+              # limits_concurrency with on_conflict: :discard destroys the job right away
+              if job.persisted?
+                coalesce_key.update_columns(job_id: job.id)
+              else
+                coalesce_key.destroy!
+              end
+
+              active_job.provider_job_id = job.id if job.persisted?
+              active_job.successfully_enqueued = job.persisted?
+            end
+          end
+
+          def record_coalesced(key, active_jobs, open_job_id)
+            active_jobs.each do |active_job|
+              collected = active_job.class.coalesce_collect
+
+              active_job.provider_job_id = open_job_id
+              active_job.successfully_enqueued = collected
+              active_job.coalesced = open_job_id.present?
+
+              if open_job_id
+                SolidQueue.instrument(:coalesce, key: key, job_id: open_job_id, active_job_id: active_job.job_id, action: collected ? :collected : :dropped)
+              end
+            end
+          end
+      end
+
+      def coalesced?
+        job_class.try(:coalesce_key).present?
+      end
+
+      def coalesce_limit
+        job_class.try(:coalesce_limit)
+      end
+
+      def coalesce_flush_when_full?
+        job_class.try(:coalesce_flush_when_full)
+      end
+
+      # The start step. Returns what the Active Job needs to know about this run,
+      # which stays in memory instead of going into its serialized data.
+      def start_coalesced_run
+        return unless coalesced? && CoalesceKey.migrated?
+
+        self.class.retrying_deadlocks do
+          transaction do
+            release_coalesce_key
+            { "calls" => coalesced_arguments.order(:id).pluck(:id, :arguments), "concurrency_key" => concurrency_key }
+          end
+        end
+      end
+
+      # The next job for the same key, picking up from the rows the job that's
+      # starting left waiting. Its arguments are the raw ones in the oldest row:
+      # deserializing them here would let a deleted record break the handover.
+      def enqueue_next_coalesced(oldest, due_now:)
+        due_at = (oldest.scheduled_at unless due_now) || Time.current
+        active_job_id = SecureRandom.uuid
+
+        self.class.create!(
+          queue_name: queue_name,
+          class_name: class_name,
+          priority: priority,
+          active_job_id: active_job_id,
+          scheduled_at: due_at,
+          concurrency_key: concurrency_key,
+          arguments: next_coalesced_arguments(active_job_id, oldest, due_at)
+        )
+      end
+
+      private
+        def release_coalesce_key
+          return unless CoalesceKey.migrated?
+
+          if job_class&.coalesce_key && !job_class.coalesce_collect
+            CoalesceKey.lock_for_release(id)&.destroy!
+          elsif job_class.nil? || coalesced?
+            CoalesceKey.lock_for_release(id)&.release
+          end
+        end
+
+        def next_coalesced_arguments(active_job_id, oldest, due_at)
+          arguments.except("continuation", "resumptions").merge(
+            "job_id" => active_job_id,
+            "provider_job_id" => nil,
+            "arguments" => oldest.arguments,
+            "executions" => 0,
+            "exception_executions" => {},
+            "enqueued_at" => Time.now.utc.iso8601(9),
+            "scheduled_at" => due_at.utc.iso8601(9)
+          )
+        end
+    end
+  end
+end

data/lib/active_job/coalescing.rb

@@ -0,0 +1,198 @@
+# frozen_string_literal: true
+
+module ActiveJob
+  module Coalescing
+    extend ActiveSupport::Concern
+
+    DEFAULT_COALESCE_LIMIT = 100
+
+    # Sits in front of the job's own #perform, whichever class defines it, so
+    # #arguments can keep holding this call's own arguments.
+    module CollectedPerform
+      def perform(*)
+        super(@coalesced_calls)
+      end
+    end
+
+    included do
+      class_attribute :coalesce_key, instance_accessor: false
+      class_attribute :coalesce_collect, default: false, instance_accessor: false
+      class_attribute :coalesce_limit, instance_accessor: false
+      class_attribute :coalesce_flush_when_full, instance_accessor: false
+
+      attr_writer :coalesced
+    end
+
+    class_methods do
+      def coalesce(key:, collect: false, limit: nil, flush_when_full: nil)
+        unless (key.is_a?(String) || key.is_a?(Symbol) || key.is_a?(Proc)) && key.present?
+          raise ArgumentError, "`key` must be a non-blank String or Symbol, or a Proc"
+        end
+
+        if limit && !collect
+          raise ArgumentError, "`limit` needs `collect: true`"
+        end
+
+        if !flush_when_full.nil? && !collect
+          raise ArgumentError, "`flush_when_full` needs `collect: true`"
+        end
+
+        limit ||= DEFAULT_COALESCE_LIMIT if collect
+
+        if limit && !(limit.is_a?(Integer) && limit.positive?)
+          raise ArgumentError, "`limit` must be a positive integer"
+        end
+
+        flush_when_full = true if collect && flush_when_full.nil?
+
+        unless flush_when_full.nil? || flush_when_full == true || flush_when_full == false
+          raise ArgumentError, "`flush_when_full` must be true or false"
+        end
+
+        ensure_coalesce_compatible_with_concurrency_controls(collect, concurrency_on_conflict)
+
+        self.coalesce_key = key
+        self.coalesce_collect = collect
+        self.coalesce_limit = limit
+        self.coalesce_flush_when_full = flush_when_full
+      end
+
+      def limits_concurrency(**)
+        super.tap { ensure_coalesce_compatible_with_concurrency_controls(coalesce_collect, concurrency_on_conflict) }
+      end
+
+      private
+        # A conflict destroys the job, and with it the calls it has collected
+        def ensure_coalesce_compatible_with_concurrency_controls(collect, on_conflict)
+          if collect && on_conflict == :discard
+            raise ArgumentError, "`coalesce` with `collect: true` can't be combined with `limits_concurrency` with `on_conflict: :discard`"
+          end
+        end
+    end
+
+    # Runs before the arguments are deserialized. Raising here reaches the rescue
+    # in ClaimedExecution#execute, which turns it into a failed execution.
+    def perform_now
+      start_coalesced_run
+      super
+    end
+
+    # Scoped to the job class name, and built like concurrency_key
+    def coalesce_key
+      if self.class.coalesce_key
+        param = compute_concurrency_parameter(self.class.coalesce_key)
+
+        case param
+        when ActiveRecord::Base
+          [ self.class.name, param.class.name, param.id ]
+        else
+          [ self.class.name, param ]
+        end.compact.join("/")
+      end
+    end
+
+    # Whether this call joined a job that was already open
+    def coalesced?
+      !!@coalesced
+    end
+
+    # A retry is the same work continuing, so it can't join or suppress anything
+    def coalesced_enqueue?
+      self.class.coalesce_key.present? && executions.to_i.zero?
+    end
+
+    # The stored key, in case the arguments it was computed from are gone
+    def concurrency_key
+      @coalesced_arguments_gone ? @coalesced_run["concurrency_key"] : super
+    end
+
+    # A retry that doesn't get enqueued has to fail the run: finishing would
+    # destroy the job and the calls it owns, with no job left to process them.
+    def retry_job(options = {})
+      super.tap do |enqueued|
+        if @coalesced_run && !enqueued
+          raise enqueue_error || SolidQueue::Job::EnqueueError.new("Couldn't enqueue the retry of #{self.class.name} (#{job_id})")
+        end
+      end
+    end
+
+    private
+      # Only the run of a job a worker has claimed. perform_now on an instance that
+      # went through perform_later also has a provider_job_id.
+      def start_coalesced_run
+        if self.class.coalesce_key && provider_job_id && self.class.queue_adapter_name == "solid_queue"
+          @coalesced_run = SolidQueue::Job.joins(:claimed_execution).find_by(id: provider_job_id)&.start_coalesced_run
+        end
+      end
+
+      def _perform_job
+        return super unless self.class.coalesce_collect
+
+        @coalesced_calls = coalesced_calls
+
+        if @coalesced_calls.any?
+          extend CollectedPerform
+          super
+        end
+      end
+
+      # The job's own arguments are a copy of its first call's, which may be for a
+      # record that's gone by now. That call's row decides what happens to it.
+      def deserialize_arguments_if_needed
+        super
+      rescue ActiveJob::DeserializationError => error
+        raise unless worker_coalesced_rows.any? && record_not_found?(error)
+
+        @coalesced_arguments_gone = true
+        @arguments = []
+      end
+
+      def worker_coalesced_rows
+        @coalesced_run&.fetch("calls") || []
+      end
+
+      def coalesced_calls
+        if worker_coalesced_rows.any?
+          calls_from_coalesced_rows
+        else
+          [ coalesced_call_from(arguments) ]
+        end
+      end
+
+      # Anything but a deleted record raises on the way, before any row has been
+      # dropped, and fails the run with its rows. When every record is gone the rows
+      # stay until the job is destroyed: if this run crashes before finishing, the
+      # re-run needs them to tell it has nothing to do, because a job without rows
+      # runs its own arguments.
+      def calls_from_coalesced_rows
+        loaded, missing = worker_coalesced_rows.map { |id, serialized| [ id, load_coalesced_row(serialized) ] }.partition(&:last)
+        drop_coalesced_rows(missing, delete: loaded.any?)
+
+        loaded.map { |_, call_arguments| coalesced_call_from(call_arguments) }
+      end
+
+      def load_coalesced_row(serialized)
+        ActiveJob::Arguments.deserialize(serialized)
+      rescue ActiveJob::DeserializationError => error
+        raise unless record_not_found?(error)
+      end
+
+      def drop_coalesced_rows(rows, delete:)
+        rows.each do |id, _|
+          SolidQueue.instrument(:drop_coalesced_argument, argument_id: id, job_id: provider_job_id, active_job_id: job_id) do
+            SolidQueue::CoalescedArgument.where(id: id).delete_all if delete
+          end
+        end
+      end
+
+      # Rails main has its own error for this, and its cause isn't Active Record's
+      def record_not_found?(error)
+        error.cause.is_a?(ActiveRecord::RecordNotFound) ||
+          (defined?(ActiveJob::DeserializationError::RecordNotFound) && error.is_a?(ActiveJob::DeserializationError::RecordNotFound))
+      end
+
+      def coalesced_call_from(call_arguments)
+        call_arguments.size == 1 ? call_arguments.first : call_arguments
+      end
+  end
+end

data/lib/generators/solid_queue/coalesce/install/install_generator.rb

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+require "rails/generators/active_record"
+
+class SolidQueue::Coalesce::InstallGenerator < Rails::Generators::Base
+  include ActiveRecord::Generators::Migration
+
+  source_root File.expand_path("templates", __dir__)
+
+  class_option :database, type: :string, aliases: %i[ --db ], default: "queue",
+    desc: "The database that Solid Queue uses. Defaults to `queue`"
+
+  def copy_migration
+    migration_template "db/create_solid_queue_coalesce_tables.rb", File.join(db_migrate_path, "create_solid_queue_coalesce_tables.rb"), skip: true
+  end
+end

data/lib/generators/solid_queue/coalesce/install/templates/db/create_solid_queue_coalesce_tables.rb

@@ -0,0 +1,28 @@
+class CreateSolidQueueCoalesceTables < ActiveRecord::Migration[7.1]
+  def change
+    create_table :solid_queue_coalesce_keys do |t|
+      # MySQL's default collations treat keys that differ by case or accent as the
+      # same key, which would drop a call that should open its own job
+      t.string :key, null: false, collation: ("utf8mb4_bin" if connection.adapter_name.match?(/mysql|trilogy/i))
+      t.bigint :job_id
+      t.datetime :created_at, null: false
+
+      t.index :key, unique: true
+      t.index :job_id, unique: true
+      t.foreign_key :solid_queue_jobs, column: :job_id, on_delete: :cascade
+    end
+
+    create_table :solid_queue_coalesced_arguments do |t|
+      t.bigint :coalesce_key_id
+      t.bigint :job_id
+      t.text :arguments, null: false
+      t.datetime :scheduled_at
+      t.datetime :created_at, null: false
+
+      t.index :coalesce_key_id
+      t.index :job_id
+      t.foreign_key :solid_queue_coalesce_keys, column: :coalesce_key_id, on_delete: :cascade
+      t.foreign_key :solid_queue_jobs, column: :job_id, on_delete: :cascade
+    end
+  end
+end

data/lib/solid_queue/coalesce/engine.rb

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+module SolidQueue
+  module Coalesce
+    class Engine < ::Rails::Engine
+      initializer "solid_queue.coalesce.active_job", after: "solid_queue.active_job.extensions" do
+        ActiveSupport.on_load :active_job do
+          include ActiveJob::Coalescing
+        end
+      end
+
+      # Job and RecurringTask are reloadable, so a reload needs the patches again
+      config.to_prepare { Patches.apply }
+    end
+  end
+end

data/lib/solid_queue/coalesce/patches.rb

@@ -0,0 +1,98 @@
+# frozen_string_literal: true
+
+# Every place this gem reaches into Solid Queue. All of it depends on private
+# details of the version pinned in the gemspec.
+module SolidQueue
+  module Coalesce
+    module Patches
+      # Prepended rather than added to the adapter, because RecurringTask calls
+      # Job.enqueue directly.
+      module Enqueuing
+        def enqueue_all(active_jobs)
+          coalesced_jobs, other_jobs = active_jobs.partition(&:coalesced_enqueue?)
+          return super if coalesced_jobs.empty?
+
+          coalesced_jobs.each { |active_job| active_job.scheduled_at ||= Time.current }
+          ensure_coalescible(coalesced_jobs)
+
+          super(other_jobs) if other_jobs.any?
+          enqueue_all_coalesced(coalesced_jobs)
+
+          active_jobs.count(&:successfully_enqueued?)
+        end
+
+        def enqueue(active_job, scheduled_at: Time.current)
+          if active_job.coalesced_enqueue?
+            active_job.scheduled_at = scheduled_at
+            ensure_coalescible([ active_job ])
+            enqueue_coalesced(active_job.coalesce_key, [ active_job ])
+          elsif active_job.class.coalesce_key
+            continuing_coalesced_run(active_job) { super }
+          else
+            super
+          end
+        end
+      end
+
+      module Failing
+        def failed_with(exception)
+          FailedExecution.transaction(requires_new: true) do
+            release_coalesce_key
+            super
+          end
+        end
+      end
+
+      # Maintenance#start runs its steps from an inline block, so this follows
+      # the only concurrency step it can reach.
+      module Maintaining
+        private
+          def expire_semaphores
+            super
+            release_stranded_coalesce_keys
+          end
+
+          def release_stranded_coalesce_keys
+            wrap_in_app_executor do
+              CoalesceKey.release_stranded(batch_size) if CoalesceKey.migrated?
+            end
+          end
+      end
+
+      module RecurringValidation
+        extend ActiveSupport::Concern
+
+        included do
+          validate :ensure_job_class_not_collecting
+        end
+
+        private
+          # A tick that joins an open job would record that job again, and the unique
+          # index on recurring_executions.job_id would roll back the call it collected.
+          def ensure_job_class_not_collecting
+            if class_name.present? && job_class&.try(:coalesce_collect)
+              errors.add :class_name, :collecting, message: "can't be a job that uses `coalesce` with `collect: true`"
+            end
+          end
+      end
+
+      def self.apply
+        SolidQueue::Job.include SolidQueue::Job::Coalescible
+        SolidQueue::Job.singleton_class.prepend Enqueuing
+        SolidQueue::Job.prepend Failing
+        SolidQueue::RecurringTask.include RecurringValidation
+        SolidQueue::Dispatcher::Maintenance.prepend Maintaining
+      end
+    end
+  end
+end
+
+class SolidQueue::LogSubscriber
+  def coalesce(event)
+    debug formatted_event(event, action: "Coalesce call", **event.payload.slice(:key, :job_id, :active_job_id), outcome: event.payload[:action])
+  end
+
+  def drop_coalesced_argument(event)
+    warn formatted_event(event, action: "Drop coalesced call for a deleted record", **event.payload.slice(:argument_id, :job_id, :active_job_id))
+  end
+end

data/lib/solid_queue/coalesce/version.rb

@@ -0,0 +1,5 @@
+module SolidQueue
+  module Coalesce
+    VERSION = "0.1.0"
+  end
+end

data/lib/solid_queue/coalesce.rb

@@ -0,0 +1,8 @@
+# frozen_string_literal: true
+
+require "solid_queue"
+
+require "active_job/coalescing"
+require "solid_queue/coalesce/version"
+require "solid_queue/coalesce/patches"
+require "solid_queue/coalesce/engine"

metadata

@@ -0,0 +1,209 @@
+--- !ruby/object:Gem::Specification
+name: solid_queue-coalesce
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- David Cristofaro
+bindir: bin
+cert_chain: []
+date: 1980-01-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: solid_queue
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 1.7.0
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 1.7.0
+- !ruby/object:Gem::Dependency
+  name: appraisal
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: debug
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.9'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.9'
+- !ruby/object:Gem::Dependency
+  name: minitest
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.0'
+- !ruby/object:Gem::Dependency
+  name: mocha
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: mysql2
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: pg
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: sqlite3
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: rubocop-rails-omakase
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: rdoc
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: logger
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+description: Lets many perform_later calls with the same key share one pending Solid
+  Queue job.
+email:
+- david@dtcristo.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- MIT-LICENSE
+- README.md
+- app/models/solid_queue/coalesce_key.rb
+- app/models/solid_queue/coalesced_argument.rb
+- app/models/solid_queue/job/coalescible.rb
+- lib/active_job/coalescing.rb
+- lib/generators/solid_queue/coalesce/install/install_generator.rb
+- lib/generators/solid_queue/coalesce/install/templates/db/create_solid_queue_coalesce_tables.rb
+- lib/solid_queue/coalesce.rb
+- lib/solid_queue/coalesce/engine.rb
+- lib/solid_queue/coalesce/patches.rb
+- lib/solid_queue/coalesce/version.rb
+homepage: https://github.com/dtcristo/solid_queue-coalesce
+licenses:
+- MIT
+metadata:
+  homepage_uri: https://github.com/dtcristo/solid_queue-coalesce
+  source_code_uri: https://github.com/dtcristo/solid_queue-coalesce
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '3.2'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 4.0.20
+specification_version: 4
+summary: Coalescing for Solid Queue.
+test_files: []