riverqueue-rails 0.0.0.reserved → 0.13.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: aafcf39071cd336e283033164f15128e0f3fc601c1711e57813894828d143d75
-  data.tar.gz: 8cc01b3759c58443f903e49719f79dd690a2cd632fd10759a0fa1b62fc66631a
+  metadata.gz: 16d36ce8062709aa5462aa2287bf95df03329fcbdd7a1ab145b3d0237b9f3cc3
+  data.tar.gz: fb340cbebbc096af53e530ad10cc927fb89c24b3a4b2e0256f2201403cb0b968
 SHA512:
-  metadata.gz: 01bc308616ddd8c7cff4753fed534e4b18da7f7792af4c8c3e32a4d8cb556546f94f93d9fbf0192b3dafc86635e1deefd2b93e1d1947bf19d710fb71f54ee09c
-  data.tar.gz: 3c3609e9e0fd583a78d16ce8329fca98d2a7aa4f37152322cea8db571c8b16be5b634b27ac497985f1ef83b5396acc8a32cb3862b06f664697dbaa3f785204bb
+  metadata.gz: 1b2948a1bf84ced265b5b7e808060a8c172b7b8c1beac854b02ee5de6b5a432f5c328436cc3b8e5cf96bdb74c77ff766e2f17c4eb2f2077361e5f2f0e92bc3d9
+  data.tar.gz: 7c4e3718bbc6760a68d9f1cd58afaf5a4f584bc5b142a20748b4dbfb5e611644a6b0385834a35c5e429e226cb4350506e8c0b76accb2f75498a169f942fe2ae5

data/README.md

@@ -1,7 +1,191 @@
-# riverqueue-rails
+# River for Rails
 
-This is an official name-reservation placeholder for Rails and Active Job integration for River.
-It is not a functional release and contains no implementation code.
-Do not use this placeholder in an application.
+`riverqueue-rails` integrates Active Job with River's existing PostgreSQL/SQLite
+runtime. It supports Rails 7.2, 8.0, and 8.1 and lives separately from the
+Rails-independent core gem. Database gems remain application-selected.
 
-Visit https://riverqueue.com for project information.
+## Install
+
+```ruby
+gem "riverqueue-rails"
+gem "pg" # Or sqlite3.
+```
+
+```sh
+bin/rails generate river:install
+bin/rails river:migrate
+bin/jobs start
+```
+
+The generator installs `config/initializers/river.rb` and `bin/jobs`. Migration
+tasks use the canonical bundled Go SQL through `River::Migrator`; no separate
+Rails schema is created. `bin/rails river:status` shows migration status.
+Migrate explicitly during deployment, not on every application boot.
+
+## Configuration
+
+```ruby
+Rails.application.configure do
+  config.active_job.queue_adapter = :river unless Rails.env.test?
+  config.river.stop_timeout = 30
+
+  config.river.configure do
+    River::Config.new(
+      job_timeout: 300,
+      logger: Rails.logger,
+      queues: {default: 10, mailers: 5}
+    )
+  end
+end
+```
+
+The block runs lazily after application boot and again when constructing a
+consumer. It must return a fresh `River::Config`. The integration registers the
+reserved `active_job` worker and installs Rails execution wrapping automatically.
+Web processes only insert; `river worker --rails` or `bin/jobs start` starts consumers. Configure all
+actual queue names, including Active Job queue prefixes and Action Mailer queues.
+Counts are per queue, per process. Size the database pool for application work,
+producers, and maintenance as well as consumer threads.
+
+Clients are rebuilt after a fork; Rails/the process server remains responsible
+for its connection-pool fork lifecycle. Do not share clients across Ractors.
+
+## Dedicated worker process
+
+From the Rails application root:
+
+```sh
+RAILS_ENV=production bundle exec river worker --rails
+# Override config.river.stop_timeout for this process:
+RAILS_ENV=production bundle exec river worker --rails --stop-timeout 60
+```
+
+The command boots Rails and builds a consumer using the configuration above.
+Do not start it in a web initializer. Migrate before starting consumers, and use
+an external supervisor for process counts and restarts.
+
+TERM/INT request stop and drain active attempts. At the graceful deadline,
+the runner interrupts attempts and allows five seconds for finalization before
+forcing exit. TSTP requests `stop(wait: false)` but keeps the process alive until
+a later TERM/INT. Successfully finalized interruptions make jobs available again;
+forced termination may leave jobs for stuck-job recovery. Give the supervisor a
+termination window longer than the grace period plus finalization and a margin.
+
+See [Dedicated worker processes](../../docs/workers.md) for signal behavior,
+exit statuses, pool sizing, and rolling replacements. The generated `bin/jobs
+start` delegates to the same runner; CLI options belong to `river worker`.
+
+## Jobs and mail
+
+```ruby
+class FulfillOrderJob < ApplicationJob
+  queue_as :default
+  retry_on PaymentGateway::Unavailable, attempts: 5, wait: 30.seconds
+  discard_on ActiveJob::DeserializationError
+
+  def perform(order)
+    FulfillOrder.call(order)
+  end
+end
+
+FulfillOrderJob.perform_later order
+FulfillOrderJob.set(wait: 10.minutes).perform_later order
+OrderMailer.receipt(order).deliver_later
+```
+
+Active Job handles serialization, GlobalID, callbacks, locale, timezone, and
+custom serializers. Bulk `ActiveJob.perform_all_later` uses River's atomic bulk
+insertion. Priorities must be `nil` (River priority 1) or integers 1 through 4.
+Unsupported priorities raise instead of silently changing meaning.
+
+## Transactions
+
+The default connection class is `ActiveRecord::Base`. Select a different abstract
+Active Record class for producers, consumers, and migration tasks together:
+
+```ruby
+Rails.application.configure do
+  config.river.connection_class = "ApplicationRecord"
+end
+```
+
+Prefer a class name in Rails initializers: it is resolved lazily after boot and
+the insertion client is rebuilt if Rails reloads the class. An actual class
+object is also accepted. Configure its database through ordinary Active Record
+`establish_connection` or `connects_to` configuration. Restart consumers after
+changing connection configuration.
+
+Without the Rails integration, select the class directly on the driver:
+
+```ruby
+River::Driver::ActiveRecord.new(connection_class: ApplicationRecord)
+```
+
+Insert jobs in the selected connection's transaction for atomicity. Sharing a
+database URL alone does not provide atomicity across connections. A dedicated
+queue connection class/database is supported, but does not commit atomically
+with application writes on another connection.
+
+Shard routing and worker management remain explicit. Inserts follow the selected
+class's current Active Record role/shard; consumers need their own appropriate
+connection configuration. A request's `connected_to` block is not propagated to
+worker threads. Use the same database backend/schema layout across roles/shards
+accessed by a driver; no automatic shard discovery or cross-shard transactions
+are provided.
+
+For Rails 8.x, disable after-commit deferral explicitly when relying on atomicity:
+
+```ruby
+class ApplicationJob < ActiveJob::Base
+  self.enqueue_after_transaction_commit = false
+end
+
+ApplicationRecord.transaction do
+  order = Order.create!(status: "pending")
+  FulfillOrderJob.perform_later(order)
+end
+```
+
+The Rails 7.2 adapter default is immediate insertion; its explicit policies use
+`:never` and `:always` instead of Rails 8's `false` and `true`. Explicit application
+after-commit policies are respected on every supported version, but lose the
+atomic-write guarantee. A returned provider ID does not prove an outer transaction
+has committed. Job completion and business writes are not automatically atomic.
+
+## Retries and observability
+
+Active Job owns application retries. `retry_on` creates a new River row with the
+same Active Job UUID, while `provider_job_id` becomes the new River row ID string.
+Unhandled errors, including exhausted `retry_on`, discard the current row without
+an additional River retry cycle. Native River workers retain their normal policy.
+The 25-attempt River budget is retained for crash recovery; stuck claims are
+rescued by normal River maintenance (currently after an hour).
+
+River interruption, cancellation, and snooze exceptions bypass Active Job's
+retry/discard handlers, including `retry_on StandardError`. Avoid swallowing
+these exceptions in broad Ruby `rescue` blocks inside application code.
+
+Handled retries/discards complete the current delivery row. Its metadata records
+`active_job_outcome` (`retried` or `discarded`), and retried deliveries record
+`active_job_retry_id`. A completed delivery need not mean the logical job succeeded.
+Active Job logging and notifications remain available. Rows store a versioned
+Active Job envelope under the reserved `active_job` kind; Go can share the schema,
+but Ruby must execute these jobs. Use native River kinds for cross-language jobs.
+
+Retry enqueueing and predecessor finalization are not atomic. Crashes can cause
+duplicates; jobs must remain idempotent. Rails' executor/reloader wraps each work
+attempt and periodic constructor, cleaning context and connections. Active Job
+classes are resolved per execution. Restart workers when changing native River
+worker/plugin registrations or periodic configuration; these retain Ruby objects.
+
+## Testing
+
+Keep Rails' `:test` adapter and `ActiveJob::TestHelper` for ordinary application
+tests. Use `:river` and a migrated isolated database for persistence/runtime
+integration tests. Disable transactional fixtures for threaded worker tests.
+The repository's `make test` includes this gem's PostgreSQL and SQLite tests.
+`RIVER_REQUIRE_DATABASES=1 make test` makes missing PostgreSQL an error.
+
+Outside a Rails app, instantiate `ActiveJob::QueueAdapters::RiverAdapter.new(client:)`
+and register `River::Rails::Worker` on your consuming River client. Rails-specific
+boot, configuration, and execution wrapping are then your responsibility.

data/lib/active_job/queue_adapters/river_adapter.rb

@@ -0,0 +1,67 @@
+# frozen_string_literal: true
+
+module ActiveJob
+  module QueueAdapters
+    # Persists Active Job envelopes using the application's River client.
+    class RiverAdapter < AbstractAdapter
+      # An explicit client also allows using the adapter without a Rails app.
+      def initialize(client: nil)
+        @client = client
+      end
+
+      # Inserts an immediate Active Job delivery.
+      def enqueue(job)
+        enqueue_at(job, nil)
+      end
+
+      # Inserts the whole batch atomically and assigns provider IDs on success.
+      def enqueue_all(jobs)
+        results = client.insert_many(jobs.map { |job| params(job, job.scheduled_at) })
+        jobs.zip(results).each do |job, result|
+          job.provider_job_id = result.job.id.to_s
+          job.successfully_enqueued = true
+          record_retry(job)
+        end
+
+        results.length
+      end
+
+      # Inserts a delivery scheduled at an epoch timestamp (nil means now).
+      def enqueue_at(job, timestamp)
+        insertion = params(job, timestamp && Time.at(timestamp).utc)
+        result = client.insert(insertion.args, insert_opts: insertion.insert_opts)
+        job.provider_job_id = result.job.id.to_s
+        record_retry(job)
+        result
+      end
+
+      # Rails 7.2 adapter default: preserve same-connection atomic insertion.
+      def enqueue_after_transaction_commit? = false
+
+      private def client
+        @client || ::Rails.application.config.river.client
+      end
+
+      private def params(job, scheduled_at)
+        priority = job.priority
+        valid_priority = priority.nil? || (priority.is_a?(Integer) && (1..4).cover?(priority))
+        unless valid_priority
+          raise ArgumentError, "River Active Job priority must be nil or an integer from 1 to 4"
+        end
+
+        River::InsertManyParams.new(
+          River::JobArgsHash.new("active_job", {"job" => job.serialize, "version" => 1}),
+          max_attempts: 25, priority: priority || 1,
+          queue: job.queue_name, scheduled_at: scheduled_at
+        )
+      end
+
+      private def record_retry(job)
+        current = ActiveSupport::IsolatedExecutionState[:river_active_job]
+        if current && current.args.fetch("job").fetch("job_id") == job.job_id
+          current.update_metadata("active_job_outcome" => "retried", "active_job_retry_id" => job.provider_job_id)
+        end
+      end
+    end
+  end
+end

data/lib/generators/river/install_generator.rb

@@ -0,0 +1,18 @@
+# frozen_string_literal: true
+
+require "rails/generators"
+
+module River
+  module Generators
+    # Installs configuration and the explicitly started worker entry point.
+    class InstallGenerator < ::Rails::Generators::Base
+      source_root File.expand_path("templates", __dir__)
+
+      def install
+        copy_file "river.rb", "config/initializers/river.rb"
+        copy_file "jobs", "bin/jobs"
+        chmod "bin/jobs", 0o755
+      end
+    end
+  end
+end

data/lib/generators/river/templates/jobs

@@ -0,0 +1,8 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require_relative "../config/environment"
+require "riverqueue-rails"
+
+abort "Usage: bin/jobs start" unless ARGV == ["start"]
+exit River::Rails::Runner.start

data/lib/generators/river/templates/river.rb

@@ -0,0 +1,19 @@
+# frozen_string_literal: true
+
+Rails.application.configure do
+  # Optional: select an abstract connection class by name, resolved after boot.
+  # config.river.connection_class = "ApplicationRecord"
+  config.active_job.queue_adapter = :river unless Rails.env.test?
+
+  config.river.configure do
+    River::Config.new(
+      job_timeout: 300,
+      logger: Rails.logger,
+      queues: {default: 10, mailers: 5}
+    )
+  end
+end
+
+# For atomic enqueueing on the same Active Record connection, set
+# self.enqueue_after_transaction_commit = false in ApplicationJob (Rails 8.x).
+# Explicit after-commit deferral is respected, but is not an atomic SQL write.

data/lib/river/rails/configuration.rb

@@ -0,0 +1,86 @@
+# frozen_string_literal: true
+
+module River
+  module Rails
+    # Application-owned configuration and per-process insertion client.
+    class Configuration
+      # Seconds allowed for graceful termination before attempts are interrupted.
+      attr_accessor :stop_timeout
+
+      def initialize
+        @connection_class = "ActiveRecord::Base"
+        @factory = -> { River::Config.new(logger: ::Rails.logger, queues: {"default" => 10}) }
+        @mutex = Mutex.new
+        @pid = Process.pid
+        @stop_timeout = 30
+      end
+
+      # Builds a consumer. Called after Rails boot, never in a web initializer.
+      def build_client
+        core = @factory.call
+        workers = River::Workers.new
+        core.workers.kinds.each { |kind| workers.add(kind, core.workers.fetch(kind)) }
+        workers.add(Worker)
+        periodic = core.periodic_jobs.map do |job|
+          River::PeriodicJob.new(id: job.id,
+            constructor: -> { ::Rails.application.reloader.wrap { job.constructor.call } },
+            run_on_start: job.run_on_start, schedule: job.schedule)
+        end
+
+        River::Client.new(build_driver,
+          config: core.with(periodic_jobs: periodic, plugins: [ExecutionPlugin.new] + core.plugins, workers: workers))
+      end
+
+      # Builds a driver for the configured connection class, including migrations.
+      def build_driver
+        River::Driver::ActiveRecord.new(connection_class: connection_class)
+      end
+
+      # Returns a lazy insertion-only client for this process.
+      def client
+        if @pid != Process.pid
+          @client = nil
+          @mutex = Mutex.new
+          @pid = Process.pid
+        end
+
+        @mutex.synchronize do
+          klass = connection_class
+          @client = nil if @client && !@client.driver.connection_class.equal?(klass)
+
+          @client ||= River::Client.new(River::Driver::ActiveRecord.new(connection_class: klass),
+            config: @factory.call.with(periodic_jobs: [], queues: {}))
+        end
+      end
+
+      # Resolves names lazily so Rails can reload application connection classes.
+      def connection_class
+        @connection_class.is_a?(String) ? @connection_class.constantize : @connection_class
+      end
+
+      # Selects an abstract Active Record class or its name. Prefer a name in
+      # initializers to avoid retaining a reloadable class object.
+      def connection_class=(value)
+        @mutex.synchronize do
+          @connection_class = value
+          @client = nil
+        end
+      end
+
+      # Supplies a block returning River::Config after application boot.
+      def configure(&block)
+        raise ArgumentError, "configuration block required" unless block
+
+        @factory = block
+        @client = nil
+      end
+    end
+
+    # Wraps each attempt in Rails' reload-safe execution boundary.
+    class ExecutionPlugin
+      def work(_job, operation)
+        ::Rails.application.reloader.wrap { operation.call }
+      end
+    end
+  end
+end

data/lib/river/rails/railtie.rb

@@ -0,0 +1,27 @@
+# frozen_string_literal: true
+
+require "rails/railtie"
+
+module River
+  module Rails
+    class Railtie < ::Rails::Railtie
+      config.river = Configuration.new
+
+      rake_tasks do
+        namespace :river do
+          desc "Apply River's canonical database migrations"
+          task migrate: :environment do
+            River::Migrator.new(::Rails.application.config.river.build_driver).migrate
+          end
+
+          desc "Show River migration status"
+          task status: :environment do
+            River::Migrator.new(::Rails.application.config.river.build_driver).status.each do |migration|
+              puts "#{migration.applied ? "applied" : "pending"} #{migration.version} #{migration.name}"
+            end
+          end
+        end
+      end
+    end
+  end
+end

data/lib/river/rails/runner.rb

@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+module River
+  module Rails
+    # Runs one worker process; an external supervisor owns process restarts.
+    class Runner
+      # Boot Rails before calling. TERM/INT stop fetching and drain active jobs.
+      def self.start(out: $stdout, stop_timeout: nil)
+        config = ::Rails.application.config.river
+        River::WorkerRunner.new(config.build_client, out: out,
+          stop_timeout: stop_timeout || config.stop_timeout).run
+      end
+    end
+  end
+end

data/lib/river/rails/worker.rb

@@ -0,0 +1,46 @@
+# frozen_string_literal: true
+
+module River
+  module Rails
+    # Keep River's control-flow exceptions out of Active Job retry/discard
+    # handlers, including broad retry_on StandardError declarations.
+    module ExecutionControl
+      def rescue_with_handler(exception)
+        if ActiveSupport::IsolatedExecutionState[:river_active_job] &&
+            [River::ClientRuntime::Interrupted, River::JobCancelError, River::JobSnoozeError].any? { |type| exception.is_a?(type) }
+          raise exception
+        end
+
+        super
+      end
+    end
+
+    ActiveJob::Base.prepend(ExecutionControl)
+
+    # Dispatches serialized jobs through Active Job's own execution machinery.
+    class Worker
+      def self.kind = "active_job"
+
+      # Active Job owns retries for reported errors; River still rescues crashes.
+      def retry?(_job, _error) = false
+
+      def work(job)
+        previous = ActiveSupport::IsolatedExecutionState[:river_active_job]
+        ActiveSupport::IsolatedExecutionState[:river_active_job] = job
+        raise ArgumentError, "Unsupported Active Job envelope version" unless job.args.fetch("version") == 1
+
+        payload = job.args.fetch("job").merge("provider_job_id" => job.id.to_s)
+        ActiveJob::Base.execute(payload)
+      ensure
+        ActiveSupport::IsolatedExecutionState[:river_active_job] = previous
+      end
+    end
+
+    ActiveSupport::Notifications.subscribe("discard.active_job") do |*, payload|
+      job = ActiveSupport::IsolatedExecutionState[:river_active_job]
+      if job && payload.fetch(:job).job_id == job.args.fetch("job").fetch("job_id")
+        job.update_metadata("active_job_outcome" => "discarded")
+      end
+    end
+  end
+end

data/lib/riverqueue-rails.rb

@@ -0,0 +1,9 @@
+# frozen_string_literal: true
+
+require "active_job"
+require "riverqueue-activerecord"
+require_relative "river/rails/configuration"
+require_relative "river/rails/worker"
+require_relative "river/rails/runner"
+require_relative "active_job/queue_adapters/river_adapter"
+require_relative "river/rails/railtie"

metadata

@@ -1,28 +1,87 @@
 --- !ruby/object:Gem::Specification
 name: riverqueue-rails
 version: !ruby/object:Gem::Version
-  version: 0.0.0.reserved
+  version: 0.13.0
 platform: ruby
 authors:
-- River Software
+- Blake Gentry
+- Brandur Leach
 bindir: bin
 cert_chain: []
 date: 1980-01-02 00:00:00.000000000 Z
-dependencies: []
-description: Reserved for Rails and Active Job integration for River. This prerelease
-  contains documentation only, no implementation, and is not usable in applications.
-email: support@riverqueue.com
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: activejob
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.2'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '8.2'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.2'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '8.2'
+- !ruby/object:Gem::Dependency
+  name: railties
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.2'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '8.2'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.2'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '8.2'
+- !ruby/object:Gem::Dependency
+  name: riverqueue-activerecord
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '='
+      - !ruby/object:Gem::Version
+        version: 0.13.0
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '='
+      - !ruby/object:Gem::Version
+        version: 0.13.0
 executables: []
 extensions: []
 extra_rdoc_files: []
 files:
 - README.md
+- lib/active_job/queue_adapters/river_adapter.rb
+- lib/generators/river/install_generator.rb
+- lib/generators/river/templates/jobs
+- lib/generators/river/templates/river.rb
+- lib/river/rails/configuration.rb
+- lib/river/rails/railtie.rb
+- lib/river/rails/runner.rb
+- lib/river/rails/worker.rb
+- lib/riverqueue-rails.rb
 homepage: https://riverqueue.com
 licenses:
 - MPL-2.0
-metadata:
-  allowed_push_host: https://rubygems.org
-  rubygems_mfa_required: 'true'
+metadata: {}
 rdoc_options: []
 require_paths:
 - lib
@@ -30,7 +89,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: '0'
+      version: '3.2'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
@@ -39,5 +98,5 @@ required_rubygems_version: !ruby/object:Gem::Requirement
 requirements: []
 rubygems_version: 4.0.9
 specification_version: 4
-summary: Official name-reservation placeholder for riverqueue-rails.
+summary: Active Job and Rails integration for River.
 test_files: []