sidekiq-batch-jobs 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: e99dbc78b86da74277380f2ba6fe62c5433ab3815d23b0f756ff6173267fb168
+  data.tar.gz: 96f45bcea22e9cd2359190946e62b54d92c8c192ddfbc87dd4148ccb9abe53e1
+SHA512:
+  metadata.gz: 7d665aba6579df375e91325f433cecf574abb0310ad9efd142358232890b5ddabccdac3f6d3a3a9773080f7facd7617fc4d4c0ac4116285e938e6ec805a4c15e
+  data.tar.gz: dd13456bd144b5d0a104a78b7ddf8eee1469c06a28ac497add4b78a6f4c666fa2c2151fcb2824513292b65f13af16e237bcc9e7713dec07d1d14e79ab54a509e

data/README.md

@@ -0,0 +1,195 @@
+# sidekiq-batch-jobs
+
+Batch tracking and completion callbacks for Sidekiq, backed by ActiveRecord (PostgreSQL).
+
+A hand-rolled alternative to Sidekiq Pro batches. Group a set of `perform_async` calls into a batch, persist their state in the database, and fire a callback worker when the batch finishes — success or failure.
+
+## Installation
+
+Add to your Gemfile:
+
+```ruby
+gem "sidekiq-batch-jobs"
+```
+
+Then run the install generator and migrate:
+
+```bash
+bin/rails g sidekiq:batch:jobs:install
+bin/rails db:migrate
+```
+
+The gem registers its client middleware, server middleware, and death handler automatically at boot.
+
+To opt out (e.g. you need full control over middleware order), set this in `config/application.rb` before Rails boots:
+
+```ruby
+Sidekiq::Batch::Jobs.auto_install = false
+```
+
+Then either call `Sidekiq::Batch::Jobs.install!` from your own `config/initializers/sidekiq.rb`, or wire the three pieces by hand:
+
+```ruby
+# config/initializers/sidekiq.rb
+Sidekiq.configure_client do |config|
+  # Outermost on the client chain so any dedupe/suppression middleware
+  # registered later with `chain.add` gets the final say on whether a
+  # push actually happens. We only enroll jobs that the chain agrees to push.
+  config.client_middleware do |chain|
+    chain.prepend SidekiqBatch::ClientMiddleware
+  end
+end
+
+Sidekiq.configure_server do |config|
+  # Same prepend for the server's own client chain — covers `perform_async`
+  # calls made from inside a running worker.
+  config.client_middleware do |chain|
+    chain.prepend SidekiqBatch::ClientMiddleware
+  end
+
+  # `add` (not `prepend`) so this runs LAST in the server chain — outside
+  # Sidekiq's retry middleware. That way we see the terminal disposition
+  # of each attempt and only mark the row failed on the final retry.
+  config.server_middleware do |chain|
+    chain.add SidekiqBatch::Middleware
+  end
+
+  # Reconciles jobs that died without re-entering middleware (SIGKILL, OOM,
+  # pod eviction). Without this, a killed job's batch row stays `pending`
+  # forever and the batch never completes.
+  config.death_handlers << ->(job, ex) { SidekiqBatch::Middleware.handle_death(job, ex) }
+end
+```
+
+## Usage
+
+A batch is a group of Sidekiq jobs whose collective fate you care about. Once every job in the batch ends in a terminal state, a callback worker fires — exactly once. You typically use a batch when there's "fan-in" work to do after a bunch of independent jobs finish: rebuilding a derived dataset after rescoring, sending one summary email after a thousand individual notifications, marking an import as ready once all rows are processed.
+
+### The four steps
+
+```ruby
+# 1. Create the batch. Use `context` to stash any state the callback will need —
+#    the callback only receives the batch id, not the surrounding closure, so
+#    anything you'd otherwise close over goes here.
+batch = SidekiqBatch.create!(
+  description: "Rescore leaderboard #{leaderboard.id}",
+  context: {
+    "leaderboard_id" => leaderboard.id,
+    "triggered_by"   => current_user.id,
+    "reason"         => "manual rescore from admin panel",
+  },
+)
+
+# 2. Register what happens when the batch finishes
+batch.on(:complete, RebuildLeaderboardCacheWorker)
+batch.on(:failure, AlertOpsOfFailedRescoreWorker)
+
+# 3. Enqueue jobs *inside* the batch context — they get enrolled automatically
+batch.jobs do
+  leaderboard.entries.find_each do |entry|
+    ScoreWorker.perform_async(entry.id)
+  end
+end
+
+# 4. The batch is now running. Your callback worker will be invoked when
+#    the last job lands in a terminal state — possibly hours later, on a
+#    completely different worker process.
+```
+
+### The `context` column
+
+`SidekiqBatch#context` is a `jsonb` column for arbitrary per-batch metadata. The gem doesn't read it — it's a slot for *you* to pass information from the code that created the batch through to the callback worker, since the callback only receives `batch_id` and has to rehydrate everything else from the database.
+
+Useful things to stash there:
+
+- **Foreign keys** the callback needs (`leaderboard_id`, `import_id`, `tenant_id`).
+- **Provenance** for debugging or audit (`triggered_by`, `via`, `request_id`).
+- **Configuration** the callback should branch on (`notify_slack: true`, `recompute_strategy: "fast"`).
+
+Keep it small and stable — it's metadata, not a payload. If you find yourself stuffing large arrays in there, that's a sign the data should live in its own table with a `sidekiq_batch_id`.
+
+### What `batch.jobs do … end` actually does
+
+The block establishes a thread-local "enrollment context." While the block runs:
+
+- Every `perform_async` / `perform_bulk` / `set(...).perform_async` call made on this thread is intercepted by the client middleware.
+- For each pushed job, a `SidekiqBatchJob` row is written to Postgres with the job's `jid`, worker class, and args — **before** Sidekiq pushes the payload to Redis. That ordering is the whole point: when the worker later runs and the server middleware looks for a matching row, it's guaranteed to find one.
+- When the block returns, the batch transitions from `pending` to `running` and the context is cleared.
+- Jobs enqueued **outside** the block are not enrolled and don't count toward this batch's completion. Same for jobs that *child* workers enqueue while running — only the original thread's enqueues are captured (by design — keeps the batch's scope predictable).
+
+You don't have to think about jid tracking, race conditions, or middleware ordering — that's the gem's job. You just write the block.
+
+### What the callback worker receives
+
+The callback gets one argument: the batch id. From there it can inspect the batch's full state:
+
+```ruby
+class RebuildLeaderboardCacheWorker
+  include Sidekiq::Worker
+
+  def perform(batch_id)
+    batch          = SidekiqBatch.find(batch_id)
+    leaderboard_id = batch.context.fetch("leaderboard_id")
+
+    Rails.logger.info "Batch #{batch.description} finished: #{batch.progress}"
+    # => Batch Rescore leaderboard 42 finished: {total: 1247, complete: 1247, failed: 0, pending: 0}
+
+    LeaderboardCacheRebuilder.run(leaderboard_id)
+  end
+end
+
+class AlertOpsOfFailedRescoreWorker
+  include Sidekiq::Worker
+
+  def perform(batch_id)
+    batch = SidekiqBatch.find(batch_id)
+
+    batch.failed_jobs.find_each do |bj|
+      Ops.notify(
+        "Rescore worker died: #{bj.worker_class} args=#{bj.args} " \
+        "error=#{bj.error_class}: #{bj.error_message}",
+      )
+    end
+  end
+end
+```
+
+Either `:complete` *or* `:failure` will fire — never both, never twice. `:complete` fires when every enrolled job succeeded; `:failure` fires the moment any job ends in a failed terminal state (retries exhausted, or `retry: false` workers that raised).
+
+### Inspecting a batch from anywhere
+
+```ruby
+batch.progress
+# => { total: 1247, complete: 1101, failed: 3, pending: 143 }
+
+batch.pending_jobs    # ActiveRecord relation
+batch.failed_jobs
+batch.completed_jobs
+
+batch.status          # "pending" | "running" | "complete" | "failed"
+batch.completed_at    # nil while running, set on terminal transition
+batch.context         # the jsonb hash you stashed when creating the batch
+```
+
+## Development
+
+After checking out the repo, run `bin/setup` to install gem dependencies. Then run the suite with:
+
+```bash
+bin/test                  # starts Postgres via docker compose, then runs rspec
+bin/test spec/models      # forwards args through to rspec
+```
+
+The test harness uses [Combustion](https://github.com/pat/combustion) to boot a minimal Rails app under `spec/internal/`, and a Postgres container defined in `docker-compose.yml`. The container uses an ephemeral tmpfs for its data directory, so it's safe to `docker compose down` at any time.
+
+If you'd rather skip the wrapper, the manual flow is:
+
+```bash
+docker compose up -d
+bundle exec rspec
+docker compose down
+```
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/douglasgreyling/sidekiq-batch-jobs.

data/Rakefile

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+require "rubocop/rake_task"
+
+RuboCop::RakeTask.new
+
+task default: %i[spec rubocop]

data/app/models/sidekiq_batch/batch_enrollment_context.rb

@@ -0,0 +1,91 @@
+# frozen_string_literal: true
+
+class SidekiqBatch
+  # Scopes a block of `perform_async` calls to a specific SidekiqBatch.
+  # Active only on the enrolling thread via a thread-local reference;
+  # child jobs spawned at execution time from tracked workers are
+  # pass-through (by design).
+  #
+  # Enrollment itself is performed by `SidekiqBatch::ClientMiddleware`,
+  # which looks up the thread-local context during each client push and
+  # writes a `SidekiqBatchJob` row BEFORE Sidekiq's `raw_push` sends the
+  # job to Redis. The client middleware also sits outermost in the chain
+  # so that dedupe/suppression middleware earlier-added get the final
+  # say on whether a payload should actually be enrolled.
+  class BatchEnrollmentContext
+    THREAD_KEY   = :sidekiq_batch_enrollment_context
+    TXN_BASELINE = :sidekiq_batch_enrollment_txn_baseline
+
+    class Error < StandardError; end
+
+    class NestedError < Error; end
+
+    class TransactionError < Error; end
+
+    class EmptyEnrollmentError < Error; end
+
+    def self.current
+      Thread.current[THREAD_KEY]
+    end
+
+    # Specs running inside a fixture transaction set this to the open-txn
+    # count at the start of each example; anything above the baseline is
+    # a caller-opened transaction and is rejected.
+    def self.transaction_baseline
+      Thread.current[TXN_BASELINE] || 0
+    end
+
+    def initialize(batch)
+      @batch          = batch
+      @inserted_count = 0
+    end
+
+    attr_reader :batch
+
+    def run # rubocop:disable Metrics/AbcSize, Metrics/MethodLength
+      raise NestedError, "jobs {} is already active on this thread" if self.class.current
+      # NOTE: We can't allow the context to work whilst wrapped by a transaction.
+      # For the sake of tracking progress correctly we need the state of Redis and the DB to be one to one
+      raise TransactionError, "jobs {} cannot be called inside an open ActiveRecord transaction" if in_open_transaction?
+
+      Thread.current[THREAD_KEY] = self
+
+      yield
+
+      if @inserted_count.zero?
+        # Empty block means this batch will never have work to complete.
+        # Destroy the batch row so callers who rescue the error don't leave
+        # orphan `pending` rows lying around forever.
+        @batch.destroy
+        raise EmptyEnrollmentError, "jobs {} block enrolled zero jobs"
+      end
+
+      actual_total = @batch.sidekiq_batch_jobs.count
+
+      @batch.update!(total_jobs: actual_total, status: "running")
+      @batch.attempt_completion!
+    ensure
+      Thread.current[THREAD_KEY] = nil
+    end
+
+    # Called by ClientMiddleware after downstream middleware has confirmed
+    # the job will be pushed. Insert is committed immediately (no surrounding
+    # transaction) so Sidekiq's subsequent raw_push hands off a visible row.
+    def enroll(payload)
+      ::SidekiqBatchJob.create!(
+        sidekiq_batch_id: @batch.id,
+        jid:              payload.fetch("jid"),
+        worker_class:     payload.fetch("class"),
+        args:             payload.fetch("args", []),
+        status:           "pending"
+      )
+      @inserted_count += 1
+    end
+
+    private
+
+    def in_open_transaction?
+      ::ActiveRecord::Base.connection.open_transactions > self.class.transaction_baseline
+    end
+  end
+end

data/app/models/sidekiq_batch/client_middleware.rb

@@ -0,0 +1,19 @@
+# frozen_string_literal: true
+
+class SidekiqBatch
+  # Sidekiq client middleware. Registered outermost (via `chain.prepend`)
+  # so that any dedupe/suppression middleware added later with `chain.add`
+  # has already decided whether the job will be pushed by the time we
+  # inspect the yield result. We enroll only when the chain returns a
+  # truthy payload (i.e. the push is going ahead).
+  class ClientMiddleware
+    def call(_worker_class, job, _queue, _redis_pool)
+      result  = yield
+      context = ::SidekiqBatch::BatchEnrollmentContext.current
+
+      context.enroll(job) if result && context
+
+      result
+    end
+  end
+end

data/app/models/sidekiq_batch/middleware.rb

@@ -0,0 +1,76 @@
+# frozen_string_literal: true
+
+class SidekiqBatch
+  # Sidekiq server middleware. Registered last in the chain so it runs
+  # outside Sidekiq's retry middleware — we see the terminal disposition
+  # of every attempt.
+  #
+  # - On success: mark complete, run completion check.
+  # - On failure: only mark failed on the FINAL attempt (retries exhausted
+  #   or retry: false). Non-terminal failures leave the row `pending` and
+  #   skip the completion check; the next attempt will decide.
+  # - Skips gracefully for jobs with no SidekiqBatchJob row (untracked).
+  # - `mark_complete!` / `mark_failed!` are idempotent (atomic
+  #   `WHERE status = 'pending'` update), so replays are no-ops.
+  class Middleware
+    DEFAULT_MAX_RETRIES = 25
+
+    def call(worker, job, _queue) # rubocop:disable Metrics/MethodLength
+      batch_job = ::SidekiqBatchJob.find_by(jid: job["jid"])
+
+      unless batch_job
+        yield
+
+        return
+      end
+
+      begin
+        yield
+      rescue StandardError => e
+        handle_failure(worker, job, batch_job, e)
+        raise
+      end
+
+      handle_success(batch_job)
+    end
+
+    # Also used by the Sidekiq death handler to reconcile jobs that died
+    # without re-entering middleware (SIGKILL, OOM, pod eviction, etc.).
+    def self.handle_death(job, error)
+      batch_job = ::SidekiqBatchJob.find_by(jid: job["jid"])
+
+      return unless batch_job
+      return unless batch_job.mark_failed!(error)
+
+      batch_job.sidekiq_batch.attempt_completion!
+    end
+
+    private
+
+    def handle_success(batch_job)
+      return unless batch_job.mark_complete!
+
+      batch_job.sidekiq_batch.attempt_completion!
+    end
+
+    def handle_failure(worker, job, batch_job, error)
+      return unless final_attempt?(worker, job)
+      return unless batch_job.mark_failed!(error)
+
+      batch_job.sidekiq_batch.attempt_completion!
+    end
+
+    def final_attempt?(worker, job)
+      retry_opt   = worker.class.get_sidekiq_options["retry"]
+      max_retries = case retry_opt
+                    when false, 0 then 0
+                    when Integer then retry_opt
+                    else DEFAULT_MAX_RETRIES
+                    end
+
+      retry_count = job["retry_count"] || 0
+
+      retry_count >= max_retries
+    end
+  end
+end

data/app/models/sidekiq_batch.rb

@@ -0,0 +1,133 @@
+# frozen_string_literal: true
+
+# == Schema Information
+#
+# Table name: sidekiq_batches
+#
+#  id                :bigint           not null, primary key
+#  callback_fired_at :datetime
+#  callbacks         :jsonb            not null
+#  completed_at      :datetime
+#  context           :jsonb            not null
+#  description       :string
+#  status            :integer          default("pending"), not null
+#  total_jobs        :integer          default(0), not null
+#  created_at        :datetime         not null
+#  updated_at        :datetime         not null
+#
+class SidekiqBatch < ActiveRecord::Base
+  EVENTS = %w[complete failure].freeze
+
+  enum :status, { pending: 0, running: 1, complete: 2, failed: 3 }, suffix: :status
+
+  has_many :sidekiq_batch_jobs, dependent: :destroy
+
+  validates :total_jobs, numericality: { greater_than_or_equal_to: 0 }
+
+  def on(event, job_class)
+    event_str = event.to_s
+    raise ArgumentError, "unknown event #{event.inspect}" unless EVENTS.include?(event_str)
+
+    self.callbacks = callbacks.merge(event_str => job_class.to_s)
+
+    save!
+
+    self
+  end
+
+  def jobs(&block)
+    raise ArgumentError, "block required" unless block_given?
+
+    BatchEnrollmentContext.new(self).run(&block)
+
+    self
+  end
+
+  def progress
+    counts = sidekiq_batch_jobs.group(:status).count
+
+    {
+      total:    total_jobs,
+      complete: counts.fetch("complete", 0),
+      failed:   counts.fetch("failed", 0),
+      pending:  counts.fetch("pending", 0)
+    }
+  end
+
+  def pending_jobs
+    sidekiq_batch_jobs.where(status: "pending")
+  end
+
+  def failed_jobs
+    sidekiq_batch_jobs.where(status: "failed")
+  end
+
+  def completed_jobs
+    sidekiq_batch_jobs.where(status: "complete")
+  end
+
+  # Atomic completion check. If all enrolled jobs are terminal, transition the
+  # batch to `complete` or `failed`, fire the registered callback, and stamp
+  # `callback_fired_at`. Returns the resulting status (String) or nil if the
+  # batch is not yet ready to transition.
+  def attempt_completion!
+    sql = self.class.sanitize_sql_array([completion_sql, id])
+
+    result = self.class.connection.exec_query(sql)
+
+    return nil if result.rows.empty?
+
+    reload
+
+    event = failed_status? ? "failure" : "complete"
+
+    fire_callback(event)
+
+    status
+  end
+
+  def fire_callback(event)
+    return nil if callback_fired_at.present?
+
+    job_class_name = callbacks[event.to_s]
+
+    return nil if job_class_name.blank?
+
+    self.class.transaction do
+      update!(callback_fired_at: Time.current)
+      job_class_name.constantize.perform_async(id)
+    end
+
+    job_class_name
+  end
+
+  private
+
+  def completion_sql # rubocop:disable Metrics/MethodLength
+    batch_statuses = self.class.statuses
+    job_statuses   = SidekiqBatchJob.statuses
+
+    <<~SQL.squish
+      UPDATE sidekiq_batches
+      SET
+        status = CASE
+          WHEN EXISTS (
+            SELECT 1 FROM sidekiq_batch_jobs
+            WHERE sidekiq_batch_id = sidekiq_batches.id
+              AND status = #{job_statuses.fetch("failed")}
+          ) THEN #{batch_statuses.fetch("failed")}
+          ELSE #{batch_statuses.fetch("complete")}
+        END,
+        completed_at = NOW(),
+        updated_at   = NOW()
+      WHERE id = ?
+        AND status = #{batch_statuses.fetch("running")}
+        AND NOT EXISTS (
+          SELECT 1 FROM sidekiq_batch_jobs
+          WHERE sidekiq_batch_id = sidekiq_batches.id
+            AND status = #{job_statuses.fetch("pending")}
+        )
+      RETURNING status
+    SQL
+  end
+end

data/app/models/sidekiq_batch_job.rb

@@ -0,0 +1,47 @@
+# frozen_string_literal: true
+
+# == Schema Information
+#
+# Table name: sidekiq_batch_jobs
+#
+#  id               :bigint           not null, primary key
+#  args             :jsonb            not null
+#  error_class      :string
+#  error_message    :text
+#  jid              :string           not null
+#  status           :integer          default("pending"), not null
+#  worker_class     :string           not null
+#  created_at       :datetime         not null
+#  updated_at       :datetime         not null
+#  sidekiq_batch_id :bigint           not null
+#
+class SidekiqBatchJob < ActiveRecord::Base
+  ERROR_MESSAGE_MAX = 4_000
+
+  enum :status, { pending: 0, complete: 1, failed: 2 }, suffix: :status
+
+  belongs_to :sidekiq_batch
+
+  validates :jid, presence: true, uniqueness: true
+  validates :worker_class, presence: true
+
+  # Idempotent: returns true if the row was transitioned from `pending`
+  # to `complete`; false if it was already terminal.
+  def mark_complete!
+    self.class.where(id: id, status: "pending").update_all(
+      status:     self.class.statuses.fetch("complete"),
+      updated_at: Time.current
+    ).positive?
+  end
+
+  # Idempotent: returns true if the row was transitioned from `pending`
+  # to `failed`; false if it was already terminal.
+  def mark_failed!(error)
+    self.class.where(id: id, status: "pending").update_all(
+      status:        self.class.statuses.fetch("failed"),
+      error_class:   error.class.name,
+      error_message: error.message.to_s.truncate(ERROR_MESSAGE_MAX),
+      updated_at:    Time.current
+    ).positive?
+  end
+end

data/docker-compose.yml

@@ -0,0 +1,26 @@
+# Spins up the services the test suite needs.
+#
+# Usage:
+#   docker compose up -d
+#   bundle exec rspec
+#   docker compose down
+#
+# Sidekiq tests run in fake mode (Sidekiq::Testing.fake!) so we don't need
+# Redis here — only Postgres for ActiveRecord.
+
+services:
+  postgres:
+    image: postgres:16-alpine
+    environment:
+      POSTGRES_USER: postgres
+      POSTGRES_PASSWORD: postgres
+      POSTGRES_DB: sidekiq_batch_jobs_test
+    ports:
+      - "5433:5432"
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready -U postgres"]
+      interval: 2s
+      timeout: 5s
+      retries: 10
+    tmpfs:
+      - /var/lib/postgresql/data

data/lib/generators/sidekiq/batch/jobs/install/install_generator.rb

@@ -0,0 +1,29 @@
+# frozen_string_literal: true
+
+require "rails/generators"
+require "rails/generators/active_record"
+
+module Sidekiq
+  module Batch
+    module Jobs
+      module Generators
+        class InstallGenerator < ::Rails::Generators::Base # rubocop:disable Style/Documentation
+          include ::Rails::Generators::Migration
+
+          source_root File.expand_path("templates", __dir__)
+
+          def self.next_migration_number(dirname)
+            ::ActiveRecord::Generators::Base.next_migration_number(dirname)
+          end
+
+          def copy_migration
+            migration_template(
+              "create_sidekiq_batch_tables.rb.tt",
+              "db/migrate/create_sidekiq_batch_tables.rb"
+            )
+          end
+        end
+      end
+    end
+  end
+end

data/lib/generators/sidekiq/batch/jobs/install/templates/create_sidekiq_batch_tables.rb.tt

@@ -0,0 +1,33 @@
+class CreateSidekiqBatchTables < ActiveRecord::Migration[7.1]
+  def change
+    create_table :sidekiq_batches do |t|
+      t.string   :description
+      t.integer  :status,            null: false, default: 0
+      t.integer  :total_jobs,        null: false, default: 0
+      t.jsonb    :callbacks,         null: false, default: {}
+      t.jsonb    :context,           null: false, default: {}
+      t.datetime :completed_at
+      t.datetime :callback_fired_at
+
+      t.timestamps
+    end
+
+    add_index :sidekiq_batches, :status
+    add_index :sidekiq_batches, [:status, :callback_fired_at]
+
+    create_table :sidekiq_batch_jobs do |t|
+      t.references :sidekiq_batch, null: false, foreign_key: { on_delete: :cascade }
+      t.string  :jid,           null: false
+      t.string  :worker_class,  null: false
+      t.jsonb   :args,          null: false, default: []
+      t.integer :status,        null: false, default: 0
+      t.string  :error_class
+      t.text    :error_message
+
+      t.timestamps
+    end
+
+    add_index :sidekiq_batch_jobs, :jid, unique: true
+    add_index :sidekiq_batch_jobs, [:sidekiq_batch_id, :status]
+  end
+end

data/lib/sidekiq/batch/jobs/engine.rb

@@ -0,0 +1,17 @@
+# frozen_string_literal: true
+
+require "rails/engine"
+
+module Sidekiq
+  module Batch
+    module Jobs
+      class Engine < ::Rails::Engine # rubocop:disable Style/Documentation
+        engine_name "sidekiq_batch_jobs"
+
+        config.after_initialize do
+          Sidekiq::Batch::Jobs.install! if Sidekiq::Batch::Jobs.auto_install
+        end
+      end
+    end
+  end
+end

data/lib/sidekiq/batch/jobs/version.rb

@@ -0,0 +1,9 @@
+# frozen_string_literal: true
+
+module Sidekiq
+  module Batch
+    module Jobs
+      VERSION = "0.1.0"
+    end
+  end
+end

data/lib/sidekiq/batch/jobs.rb

@@ -0,0 +1,46 @@
+# frozen_string_literal: true
+
+require "sidekiq"
+require_relative "jobs/version"
+require_relative "jobs/engine" if defined?(Rails::Engine)
+
+module Sidekiq
+  module Batch
+    module Jobs # rubocop:disable Style/Documentation
+      class Error < StandardError; end
+
+      class << self
+        attr_accessor :auto_install
+      end
+      self.auto_install = true
+
+      def self.disable_auto_install!
+        self.auto_install = false
+      end
+
+      # Idempotent — guarded by an installed flag so calling twice (auto-install
+      # plus an explicit call in specs, for example) doesn't double-register
+      # middleware or death handlers.
+      def self.install!
+        return if @installed
+
+        ::Sidekiq.configure_client do |config|
+          config.client_middleware { |chain| chain.prepend ::SidekiqBatch::ClientMiddleware }
+        end
+
+        ::Sidekiq.configure_server do |config|
+          config.client_middleware { |chain| chain.prepend ::SidekiqBatch::ClientMiddleware }
+          config.server_middleware { |chain| chain.add     ::SidekiqBatch::Middleware }
+          config.death_handlers << ->(job, ex) { ::SidekiqBatch::Middleware.handle_death(job, ex) }
+        end
+
+        @installed = true
+      end
+
+      # For tests: forget that install! has run.
+      def self.reset_installed!
+        @installed = false
+      end
+    end
+  end
+end

data/sig/sidekiq/batch/jobs.rbs

@@ -0,0 +1,8 @@
+module Sidekiq
+  module Batch
+    module Jobs
+      VERSION: String
+      # See the writing guide of rbs: https://github.com/ruby/rbs#guides
+    end
+  end
+end

metadata

@@ -0,0 +1,220 @@
+--- !ruby/object:Gem::Specification
+name: sidekiq-batch-jobs
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Douglas Greyling
+bindir: exe
+cert_chain: []
+date: 1980-01-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: activerecord
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 7.1.0
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 7.1.0
+- !ruby/object:Gem::Dependency
+  name: railties
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 7.1.0
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 7.1.0
+- !ruby/object:Gem::Dependency
+  name: sidekiq
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '9'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.0'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '9'
+- !ruby/object:Gem::Dependency
+  name: combustion
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.4'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.4'
+- !ruby/object:Gem::Dependency
+  name: concurrent-ruby
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.2'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.2'
+- !ruby/object:Gem::Dependency
+  name: database_cleaner-active_record
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.2'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '2.2'
+- !ruby/object:Gem::Dependency
+  name: factory_bot
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '6.4'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '6.4'
+- !ruby/object:Gem::Dependency
+  name: pg
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.5'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.5'
+- !ruby/object:Gem::Dependency
+  name: rspec-rails
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '6.1'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '6.1'
+- !ruby/object:Gem::Dependency
+  name: rspec-sidekiq
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.0'
+- !ruby/object:Gem::Dependency
+  name: shoulda-matchers
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '6.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '6.0'
+description: |
+  A hand-rolled alternative to Sidekiq Pro batches. Track a group of Sidekiq
+  jobs as a batch in PostgreSQL, get atomic completion detection, and fire
+  a callback worker when the batch finishes (success or failure).
+email:
+- greyling.douglas@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- README.md
+- Rakefile
+- app/models/sidekiq_batch.rb
+- app/models/sidekiq_batch/batch_enrollment_context.rb
+- app/models/sidekiq_batch/client_middleware.rb
+- app/models/sidekiq_batch/middleware.rb
+- app/models/sidekiq_batch_job.rb
+- docker-compose.yml
+- lib/generators/sidekiq/batch/jobs/install/install_generator.rb
+- lib/generators/sidekiq/batch/jobs/install/templates/create_sidekiq_batch_tables.rb.tt
+- lib/sidekiq/batch/jobs.rb
+- lib/sidekiq/batch/jobs/engine.rb
+- lib/sidekiq/batch/jobs/version.rb
+- sig/sidekiq/batch/jobs.rbs
+homepage: https://github.com/douglasgreyling/sidekiq-batch-jobs
+licenses:
+- MIT
+metadata:
+  homepage_uri: https://github.com/douglasgreyling/sidekiq-batch-jobs
+  source_code_uri: https://github.com/douglasgreyling/sidekiq-batch-jobs
+  rubygems_mfa_required: 'true'
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: 3.2.0
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 4.0.8
+specification_version: 4
+summary: Batch tracking and completion callbacks for Sidekiq, backed by ActiveRecord.
+test_files: []