marj 5.0.0 → 6.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 450ae15f6cf892215323b4eeeccdbf8a75c2db6162b0a75378c411f6e0bdac9c
-  data.tar.gz: 5b1fabd1000b8439e158aabc943deef507045252efe6805cd26a89629e39efe0
+  metadata.gz: 1fd27db3bb7a04aee0b4e6cf043e35bde5329143194584bcb03b75ef00896f5d
+  data.tar.gz: d89fd27053c6beeb6aeddcaac59fd1a0ddd4b2a481a4dd0f9b96b3cbbbc29bde
 SHA512:
-  metadata.gz: 1e49dcbc43034f454fb57bf1dca72972eba67719cd8c9aa1aa43b10f91287d2bab41bb97025dbca688de9e6630f0c0d494e6607100382165f56a0f93a38fc5e4
-  data.tar.gz: d075bd8f29864ea83be5ce0099f925f9bb527f9373940427c98cfce3545ee25a565a1fb6293023938bbd39b320ab55015b924281a31bdabb43085398551342cc
+  metadata.gz: 5b1304cba5e2780fa9fd24432624e37a0a1746efefa9ffaf11573875a2333b1ae9fda8f419b94f30f953c59be0896dae6b90e808b089bbac2324cbe787d2ed8d
+  data.tar.gz: c9b6926e5ef0cad9fc33d510eb9102d570e4ace04322cfc62c44367651a42034c7be5f2dbbf88fc851c13fc6acfc005056d342676bcd505f356696b8fa0305c1

data/README.md

@@ -2,69 +2,88 @@
 
 A minimal database-backed ActiveJob queueing backend.
 
-## Purpose
-
-To provide a database-backed ActiveJob queueing backend with as few features
-as possible and the minimum backend-specific API required.
-
 ## Quick Links
 
-API docs: https://gemdocs.org/gems/marj/5.0.0/ <br>
+API docs: https://gemdocs.org/gems/marj/6.1.0/ <br>
 RubyGems: https://rubygems.org/gems/marj <br>
 Changelog: https://github.com/nicholasdower/marj/releases <br>
 Issues: https://github.com/nicholasdower/marj/issues <br>
 Development: https://github.com/nicholasdower/marj/blob/master/CONTRIBUTING.md
 
-## Features
+## Motivation
 
-### Provided
+There are already several great database-backed ActiveJob queueing backends:
+- [Delayed::Job](https://github.com/collectiveidea/delayed_job)
+- [Que](https://github.com/que-rb/que)
+- [GoodJob](https://github.com/bensheldon/good_job)
+- [Solid Queue](https://github.com/basecamp/solid_queue)
 
-- Enqueued jobs are written to the database.
-- Successfully executed jobs are deleted from the database.
-- Failed jobs which should be retried are updated in the database.
-- Failed jobs which should not be retried are deleted from the database.
-- An method is provided to query enqueued jobs.
-- An method is provided to discard enqueued jobs.
-- An `ActiveRecord` class is provided to query the database directly.
+Any of these which support your RDBMS is likely to work well for you. But you
+may find them to be more featureful and complex than what you require.
+
+Marj aims to be a minimal alternative.
 
-### Not Provided
+## Goal
 
-- Automatic job execution
-- Timeouts
-- Concurrency controls
-- Observability
-- A user interace
+To be the database-backend ActiveJob queueing backend with:
+- The simplest setup
+- The fewest configuration options
+- The fewest features
+- The fewest backend-specific APIs
+- The fewest lines of code
 
-Note that because Marj does not automatically execute jobs, clients are
-responsible for retrieving and either executing or discarding jobs.
+## Features
+
+Marj supports and has been tested with MySQL, PostgreSQL and SQLite.
+
+It provides the following features:
+- Enqueued jobs are written to the database.
+- Enqueued jobs can be queried, executed and discarded.
+- Executed jobs are re-enqueued or discarded, depending on the result.
+- Compatibility with [Mission Control Jobs](https://github.com/basecamp/mission_control-jobs).
+
+## Extensions
+
+Marj does not provide the following features by default, but each can easily
+be added to your application with a few lines of code. See [Example Usage](#example-usage)
+and [Extension Examples](#extension-examples) for suggestions.
+- [Automatic job execution](#example-usage)
+- [Concurrency protection](#concurrency-protection)
+- [Job timeouts](#job-timeouts)
+- [Error storage](#error-storage)
+- [Discarded job retention](#discarded-job-retention)
+- [Observability](#observability)
+- [Multi-DB Support](#multi-db-support)
 
 ## API
 
-The ActiveJob API already provides methods for enqueueing and performing jobs:
+Marj relies on existing ActiveJob APIs, for example:
 
 ```ruby
-queue_adapter.enqueue(job)  # Enqueue
-job.enqueue                 # Enqueue
-job.perform_now             # Perform
+job.enqueue         # Enqueues a job
+job.perform_now     # Executes a job
 ```
 
-Marj works with these existing methods and additionally extends the ActiveJob API
-with methods for querying and discarding jobs:
+Additionally, it extends the ActiveJob API with methods required for a
+minimal queueing backend implementaion:
 
 ```ruby
-queue_adapter.query(args)   # Query
-SomeJob.query(args)         # Query
-queue_adapter.discard(job)  # Discard
-job.discard                 # Discard
+SomeJob.query(args) # Queries for enqeueued jobs
+job.discard         # Runs discard callbacks and, by default, deletes the job
+job.delete          # Deletes the job
 ```
 
+## Requirements
+
+- `activejob >= 7.1`
+- `activerecord >= 7.1`
+
 ## Setup
 
 ### 1. Install
 
 ```shell
-bundle add activejob activerecord marj  # via Bundler
-gem install activejob activerecord marj # or globally
+bundle add marj
 ```
 
 ### 2. Create the database table
@@ -85,6 +104,8 @@ class CreateJobs < ActiveRecord::Migration[7.1]
       table.string   :timezone,             null: false
     end
 
+    add_index :jobs, %i[job_class]
+    add_index :jobs, %i[queue_name]
     add_index :jobs, %i[enqueued_at]
     add_index :jobs, %i[scheduled_at]
     add_index :jobs, %i[priority scheduled_at enqueued_at]
@@ -101,29 +122,26 @@ end
 ```ruby
 require 'marj'
 
+# Choose one of the following:
 Rails.configuration.active_job.queue_adapter = :marj # Globally, with Rails
 ActiveJob::Base.queue_adapter = :marj                # Globally, without Rails
 SomeJob.queue_adapter = :marj                        # Single job
 ```
 
-### 4. Include the Marj module (optional)
+### 4. Optionally, include the Marj module
 
-By default, jobs can be queried and discarded via the `MarjAdapter` or the
-`Marj` module:
+Without any additional setup, jobs can be queried and discarded via the `Marj`
+module:
 
 ```ruby
 Marj.query(:all)
-ActiveJob::Base.queue_adapter.query(:all)
 Marj.discard(job)
-ActiveJob::Base.queue_adapter.discard(job)
 ```
 
 But it is also convenient to query or discard via job classes:
 
 ```ruby
-ApplicationJob.query(:all)
 SomeJob.query(:all)
-ApplicationJob.discard(job)
 SomeJob.discard(job)
 job.discard
 ```
@@ -131,11 +149,9 @@ job.discard
 In order to enable this functionality, you must include the `Marj` module:
 
 ```ruby
-class ApplicationJob < ActiveJob::Base
+class SomeJob < ActiveJob::Base
   include Marj
-end
 
-class SomeJob < ApplicationJob
   def perform; end
 end
 ```
@@ -143,25 +159,27 @@ end
 ## Example Usage
 
 ```ruby
-# Enqueue and manually run a job
+# Enqueue a job
 job = SomeJob.perform_later('foo')
-job.perform_now
 
-# Retrieve and execute a job
-Marj.query(:due, :first).perform_now
+# Query jobs
+Marj.query(:all)
+Marj.query(:due)
+Marj.query(:due, queue: :foo)
+Marj.query('8720417d-8fff-4fcf-bc16-22aaef8543d2')
+
+# Execute a job
+job.perform_now
 
-# Run all due jobs (single DB query)
+# Execute jobs which are due (single query)
 Marj.query(:due).map(&:perform_now)
 
-# Run all due jobs (multiple DB queries)
+# Execute jobs which are due (multiple queries)
 loop do
-  break unless Marj.query(:due, :first)&.tap(&:perform_now)
+  Marj.query(:due, :first)&.tap(&:perform_now) || break
 end
 
-# Run all jobs in a specific queue which are due to be executed
-Marj.query(:due, queue: :foo).map(&:perform_now)
-
-# Run jobs as they become due
+# Execute jobs as they become due
 loop do
   Marj.query(:due).each(&:perform_now) rescue logger.error($!)
 ensure
@@ -175,17 +193,54 @@ By default, jobs enqeued during tests will be written to the database. Enqueued
 jobs can be executed via:
 
 ```ruby
-Marj.due.perform_all
+SomeJob.query(:due).map(&:perform_now)
 ```
 
-Alternatively, to use [ActiveJob::QueueAdapters::TestAdapter](https://api.rubyonrails.org/classes/ActiveJob/QueueAdapters/TestAdapter.html):
+Alternatively, to use ActiveJob's [TestAdapter](https://api.rubyonrails.org/classes/ActiveJob/QueueAdapters/TestAdapter.html):
 ```ruby
 ActiveJob::Base.queue_adapter = :test
 ```
 
 ## Extension Examples
 
-### Timeouts
+Most missing features can easily be added to your application with a few lines
+of code.
+
+### Concurrency Protection
+
+To prevent two threads from executing the same job simultaneously, consider
+moving jobs to a different queue before executing them.
+
+```ruby
+class ApplicationJob < ActiveJob::Base
+  around_perform do |job, block|
+    if job.queue_name.start_with?('claimed')
+      raise "Job #{job.job_id} already claimed"
+    end
+
+    updated = Marj::Record
+      .where(job_id: job.job_id, queue_name: job.queue_name)
+      .update_all(
+        queue_name: "claimed-#{job.queue_name}",
+        scheduled_at: Time.now.utc
+      )
+    unless updated == 1
+      raise "Failed to claim job #{job.job_id}. #{updated} records updated"
+    end
+    
+    begin
+      block.call
+    rescue StandardError
+      Marj::Record
+        .where(job_id: job.job_id, queue_name: "claimed-#{job.queue_name}")
+        .update_all(queue_name: job.queue_name, scheduled_at: job.scheduled_at)
+      raise
+    end
+  end
+end
+```
+
+### Job Timeouts
 
 ```ruby
 class ApplicationJob < ActiveJob::Base
@@ -205,7 +260,7 @@ class ApplicationJob < ActiveJob::Base
 end
 ```
 
-### Last Error
+### Error Storage
 
 ```ruby
 class AddLastErrorToJobs < ActiveRecord::Migration[7.1]
@@ -246,51 +301,77 @@ class ApplicationJob < ActiveJob::Base
 end
 ```
 
-### Multiple Tables/Databases
+### Discarded Job Retention
 
-It is possible to create a custom record class in order to, for instance,
-write jobs to multiple databases/tables within a single application.
+By default, discarded jobs are deleted from the database. If desired, this
+behavior can be overridden, for instance to move jobs to a differnt queue:
 
 ```ruby
-class CreateMyJobs < ActiveRecord::Migration[7.1]
-  def self.up
-    create_table :my_jobs, id: :string, primary_key: :job_id do |table|
-      table.string   :job_class,            null: false
-      table.text     :arguments,            null: false
-      table.string   :queue_name,           null: false
-      table.integer  :priority
-      table.integer  :executions,           null: false
-      table.text     :exception_executions, null: false
-      table.datetime :enqueued_at,          null: false
-      table.datetime :scheduled_at
-      table.string   :locale,               null: false
-      table.string   :timezone,             null: false
-    end
+ActiveJob::Base.queue_adapter = MarjAdapter.new(
+  discard: proc { _1.enqueue(queue: 'discarded') }
+)
+
+# To retrieve a discarded job, query the discarded queue:
+job = Marj.query(:first, queue_name: 'discarded')
+
+# To delete, use one of the following:
+Marj.delete(job)
+job.delete
+```
 
-    add_index :my_jobs, %i[enqueued_at]
-    add_index :my_jobs, %i[scheduled_at]
-    add_index :my_jobs, %i[priority scheduled_at enqueued_at]
+### Observability
+
+For instance with Prometheus metrics:
+
+```ruby
+class ApplicationJob < ActiveJob::Base
+  counter = Prometheus::Client::Counter.new(
+    :job_events_total, docstring: '...', labels: [:job, :event]
+  )
+
+  around_enqueue do |job, block|
+    counter.increment(labels: { job: job.class.name, event: 'before_enqueue' })
+    block.call
+    counter.increment(labels: { job: job.class.name, event: 'after_enqueue' })
+  rescue Exception
+    counter.increment(labels: { job: job.class.name, event: 'enqueue_error' })
+    raise
   end
 
-  def self.down
-    drop_table :my_jobs
+  around_perform do |job, block|
+    counter.increment(labels: { job: job.class.name, event: 'before_perform' })
+    block.call
+    counter.increment(labels: { job: job.class.name, event: 'after_perform' })
+  rescue Exception => e # rubocop:disable Lint/RescueException
+    counter.increment(labels: { job: job.class.name, event: 'perform_error' })
+    raise
+  end
+
+  after_discard do |job|
+    counter.increment(labels: { job: job.class.name, event: 'after_discard' })
   end
 end
+```
+
+### Multi-DB Support
+
+It is possible to create a custom record class in order to, for instance,
+use a different class or table name, or write jobs to multiple
+databases/tables within a single application.
+
+Assuming you have a jobs tabled named `my_jobs`:
 
+```ruby
 class MyRecord < Marj::Record
   self.table_name = 'my_jobs'
 end
 
-CreateMyJobs.migrate(:up)
-
 class MyJob < ActiveJob::Base
-  self.queue_adapter = MarjAdapter.new('MyRecord')
-
   include Marj
 
-  def perform(msg)
-    puts msg
-  end
+  self.queue_adapter = MarjAdapter.new('MyRecord')
+
+  def perform; end
 end
 
 MyJob.perform_later('oh, hi')
@@ -378,7 +459,7 @@ job = SomeJob.new.deserialize(other_job.serialize)
 job = SomeJob.perform_later
 job = SomeJob.perform_later(args)
 
-# Create without enqueueing and run (only enqueued on failure if retryable)
+# Create without enqueueing and run (only enqueued on retryable failure)
 SomeJob.perform_now
 SomeJob.perform_now(args)
 ```
@@ -399,7 +480,7 @@ SomeJob.perform_later(SomeJob.new(args))
 SomeJob.perform_later(args)
 SomeJob.set(options).perform_later(args)
 
-# After a failure during execution
+# After a retryable execution failure
 SomeJob.perform_now(args)
 ActiveJob::Base.execute(SomeJob.new(args).serialize)
 
@@ -413,7 +494,7 @@ SomeJob.set(options).perform_all_later(SomeJob.new, SomeJob.new, options:)
 ### Executing Jobs
 
 ```ruby
-# Executed without enqueueing, enqueued on failure if retryable
+# Executed without enqueueing, enqueued on retryable failure
 SomeJob.new(args).perform_now
 SomeJob.perform_now(args)
 ActiveJob::Base.execute(SomeJob.new(args).serialize)

data/lib/marj/mission_control.rb

@@ -0,0 +1,119 @@
+# frozen_string_literal: true
+
+require_relative '../marj_adapter'
+require 'mission_control/jobs'
+require 'mission_control/jobs/adapter'
+
+# :nocov:
+module Marj
+  module MissionControl
+    include ::MissionControl::Jobs::Adapter
+
+    def queues
+      record_class.group(:queue_name).count(:queue_name).map { |k, v| { name: k, size: v, active: true } }
+    end
+
+    def queue_size(queue_name)
+      record_class.where(queue_name: queue_name).count
+    end
+
+    def clear_queue(queue_name)
+      Marj::Record.where(queue_name: queue_name).delete_all
+    end
+
+    def pause_queue(_queue_name)
+      raise 'not supported: pause queue'
+    end
+
+    def resume_queue(_queue_name)
+      raise 'not supported: resume queue'
+    end
+
+    def queue_paused?(_queue_name)
+      false
+    end
+
+    def supported_statuses
+      %i[pending failed scheduled]
+    end
+
+    def supported_filters(_jobs_relation)
+      %i[queue_name job_class_name]
+    end
+
+    def exposes_workers?
+      false
+    end
+
+    def workers
+      raise 'not supported: workers'
+    end
+
+    def find_worker(_worker_id)
+      raise 'not supported: find workers'
+    end
+
+    def jobs_count(jobs_relation)
+      ar_relation(jobs_relation).count
+    end
+
+    def fetch_jobs(jobs_relation)
+      ar_relation(jobs_relation).each_with_index.map { |record, index| to_job(record, jobs_relation, index) }
+    end
+
+    def retry_all_jobs(jobs_relation)
+      ar_relation(jobs_relation).map { |record| record.to_job.perform_now }
+    end
+
+    def retry_job(job, _jobs_relation)
+      Marj::Record.find(job.job_id).to_job.perform_now
+    end
+
+    def discard_all_jobs(jobs_relation)
+      ar_relation(jobs_relation).map { |record| discard(record.to_job) }
+    end
+
+    def discard_job(job, _jobs_relation)
+      discard(Marj::Record.find(job.job_id).to_job)
+    end
+
+    def find_job(job_id, jobs_relation)
+      to_job(record_class.find_by(job_id: job_id), jobs_relation)
+    end
+
+    private
+
+    def ar_relation(jobs_relation)
+      relation = Marj::Record.all.offset(jobs_relation.offset_value).limit(jobs_relation.limit_value)
+      relation = relation.where.not(executions: 0) if jobs_relation.status == :failed
+      relation = relation.where.not(scheduled_at: nil) if jobs_relation.status == :scheduled
+      relation = relation.where(job_class: jobs_relation.job_class_name) if jobs_relation.job_class_name
+      relation = relation.where(queue_name: jobs_relation.queue_name) if jobs_relation.queue_name
+      relation
+    end
+
+    def to_job(record, jobs_relation, index = 0)
+      return nil unless record
+
+      job = record.to_job
+      job_data = job.serialize
+      ActiveJob::JobProxy.new(job_data).tap do |proxy|
+        if job.executions.positive?
+          proxy.last_execution_error = ActiveJob::ExecutionError.new(
+            error_class: Exception, message: 'unknown', backtrace: []
+          )
+          proxy.failed_at = job.enqueued_at
+          proxy.status = :failed
+        elsif job.scheduled_at
+          proxy.status = :scheduled
+        else
+          proxy.status = :pending
+        end
+        proxy.raw_data = job_data
+        proxy.position = jobs_relation.offset_value + index
+        proxy.arguments = job.arguments # For some reason MissionControl sets the arguments to the entire job data
+      end
+    end
+  end
+end
+# :nocov:

data/lib/marj/record.rb

@@ -84,8 +84,8 @@ module Marj
       #  - If a job succeeds, after_perform will be called.
       #  - If a job fails and should be retried, enqueue will be called. This is handled by the queue adapter.
       #  - If a job exceeds its max attempts, after_discard will be called.
-      job.singleton_class.after_perform { |_j| record.destroy! }
-      job.singleton_class.after_discard { |_j, _exception| record.destroy! }
+      job.singleton_class.after_perform { |_j| job.queue_adapter.delete(job) }
+      job.singleton_class.after_discard { |_j, _exception| job.queue_adapter.discard(job, run_callbacks: false) }
       job.singleton_class.instance_variable_set(:@record, record)
 
       job

data/lib/marj.rb

@@ -6,13 +6,15 @@ require_relative 'marj_adapter'
 #
 # The {Marj} module provides the following methods:
 # - +query+ - Queries enqueued jobs
-# - +discard+ - Discards a job
+# - +discard+ - Discards a job, by default by executing after_discard callbacks and delegating to delete
+# - +delete+ - Deletes a job
 #
 # It is possible to call the above methods on the {Marj} module itself or on any class which includes it.
 #
 # Example usage:
 #   Marj.query(:first) # Returns the first job
 #   Marj.discard(job)  # Discards the specified job
+#   Marj.delete(job)  # Deletes the specified job
 #
 #   class ApplicationJob < ActiveJob::Base
 #     include Marj
@@ -25,33 +27,37 @@ require_relative 'marj_adapter'
 #   job = ApplicationJob.query(:first) # Returns the first enqueued job
 #   job = SomeJob.query(:first)        # Returns the first enqueued job with job_class SomeJob
 #   ApplicationJob.discard(job)        # Discards the specified job
+#   ApplicationJob.delete(job)         # Deletes the specified job
 #   job.discard                        # Discards the job
+#   job.delete                         # Deletes the job
 #
 # See https://github.com/nicholasdower/marj
 module Marj
   # The Marj version.
-  VERSION = '5.0.0'
+  VERSION = '6.1.0'
 
   Kernel.autoload(:Record, File.expand_path(File.join('marj', 'record.rb'), __dir__))
 
   # Provides the {query} and {discard} class methods.
   module ClassMethods
-    # Queries enqueued jobs.
+    # Queries enqueued jobs. Similar to +ActiveRecord.where+ with a few additional features:
+    # - Symbol arguments are treated as +ActiveRecord+ scopes.
+    # - If only a job ID is specified, the corresponding job is returned.
+    # - If +:limit+ is specified, the maximum number of jobs is limited.
+    # - If +:order+ is specified, the jobs are ordered by the given attribute.
     #
-    # Similar to +ActiveRecord.where+ with a few additional features.
+    # By default jobs are ordered by when they should be executed.
     #
     # Example usage:
-    #   query(:all)             # Delegates to Marj::Record.all
-    #   query(:due)             # Delegates to Marj::Record.due
-    #   query(:all, limit: 10)  # Returns a maximum of 10 jobs
-    #   query(job_class: Foo)   # Returns all jobs with job_class Foo
-    #
-    #   query('123')            # Returns the job with id '123' or nil if no such job exists
-    #   query(id: '123')        # Same as above
-    #   query(job_id: '123')    # Same as above
-    #
-    #   query(queue: 'foo')     # Returns all jobs in the 'foo' queue
-    #   query(job_queue: 'foo') # Same as above
+    #   query                       # Returns all jobs
+    #   query(:all)                 # Returns all jobs
+    #   query(:due)                 # Returns jobs which are due to be executed
+    #   query(:due, limit: 10)      # Returns at most 10 jobs which are due to be executed
+    #   query(job_class: Foo)       # Returns all jobs with job_class Foo
+    #   query(:due, job_class: Foo) # Returns jobs which are due to be executed with job_class Foo
+    #   query(queue_name: 'foo')    # Returns all jobs in the 'foo' queue
+    #   query(job_id: '123')        # Returns the job with job_id '123' or nil if no such job exists
+    #   query('123')                # Returns the job with job_id '123' or nil if no such job exists
     def query(*args, **kwargs)
       kwargs[:job_class] ||= self if self < ActiveJob::Base && name != 'ApplicationJob'
       queue_adapter.query(*args, **kwargs)
@@ -63,6 +69,13 @@ module Marj
     def discard(job)
       queue_adapter.discard(job)
     end
+
+    # Deletes the record associated with the specified job.
+    #
+    # @return [ActiveJob::Base] the deleted job
+    def delete(job)
+      queue_adapter.delete(job)
+    end
   end
 
   # (see ClassMethods#query)
@@ -75,13 +88,25 @@ module Marj
     queue_adapter.discard(job)
   end
 
-  # Discards this job.
+  # (see ClassMethods#delete)
+  def self.delete(job)
+    queue_adapter.delete(job)
+  end
+
+  # Deletes this job.
   #
   # @return [ActiveJob::Base] this job
   def discard
     self.class.queue_adapter.discard(self)
   end
 
+  # Deletes the record associated with this job.
+  #
+  # @return [ActiveJob::Base] this job
+  def delete
+    self.class.queue_adapter.delete(self)
+  end
+
   def self.included(clazz)
     clazz.extend(ClassMethods)
   end
@@ -92,3 +117,15 @@ module Marj
   end
   private_class_method :queue_adapter
 end
+
+# :nocov:
+if defined?(Rails)
+  begin
+    require 'mission_control/jobs'
+    require_relative 'marj/mission_control'
+    MarjAdapter.include(Marj::MissionControl)
+  rescue LoadError
+    # ignore
+  end
+end
+# :nocov:

data/lib/marj_adapter.rb

@@ -2,11 +2,13 @@
 
 # ActiveJob queue adapter for Marj.
 #
-# In addition to the standard +ActiveJob+ queue adapter API, this adapter provides a +query+ method which can be used to
-# query enqueued jobs and a +discard+ method which can be used to discard enqueued jobs.
+# In addition to the standard +ActiveJob+ queue adapter API, this adapter provides:
+# - A +query+ method which can be used to query enqueued jobs
+# - A +discard+ method which can be used to discard enqueued jobs.
+# - A +delete+ method which can be used to delete enqueued jobs.
 #
-# Although it is possible to access the adapter directly in order to query or discard, it is recommended to use the
-# {Marj} module.
+# Although it is possible to access the adapter directly in order to query, discard or delete, it is recommended to use
+# the {Marj} module.
 #
 # See https://github.com/nicholasdower/marj
 class MarjAdapter
@@ -15,9 +17,17 @@ class MarjAdapter
 
   # Creates a new adapter which will enqueue jobs using the given +ActiveRecord+ class.
   #
-  # @param record_class [Class, String] the +ActiveRecord+ class (or its name) to use to store jobs
-  def initialize(record_class = 'Marj::Record')
+  # @param record_class [Class, String] the +ActiveRecord+ class (or its name) to use, defaults to +Marj::Record+
+  # @param discard [Proc] the proc to use to discard jobs, defaults to delegating to {delete}
+  def initialize(record_class: 'Marj::Record', discard: proc { |job| delete(job) })
     @record_class = record_class
+    @discard_proc = discard
+  end
+
+  # Returns whether jobs enqueued during a transaction should actually be enqueued only after the
+  # transaction is committed.
+  def enqueue_after_transaction_commit?
+    false
   end
 
   # Enqueue a job for immediate execution.
@@ -77,25 +87,25 @@ class MarjAdapter
   end
 
   # Queries enqueued jobs. Similar to +ActiveRecord.where+ with a few additional features:
-  # - Leading symbol arguments are treated as +ActiveRecord+ scopes.
+  # - Symbol arguments are treated as +ActiveRecord+ scopes.
   # - If only a job ID is specified, the corresponding job is returned.
   # - If +:limit+ is specified, the maximum number of jobs is limited.
+  # - If +:order+ is specified, the jobs are ordered by the given attribute.
   #
-  # Example usage:
-  #   query(:all)             # Delegates to Marj::Record.all
-  #   query(:due)             # Delegates to Marj::Record.due
-  #   query(:all, limit: 10)  # Returns a maximum of 10 jobs
-  #   query(job_class: Foo)   # Returns all jobs with job_class Foo
-  #
-  #   query('123')            # Returns the job with id '123' or nil if no such job exists
-  #   query(id: '123')        # Same as above
-  #   query(job_id: '123')    # Same as above
+  # By default jobs are ordered by when they should be executed.
   #
-  #   query(queue: 'foo')     # Returns all jobs in the 'foo' queue
-  #   query(job_queue: 'foo') # Same as above
+  # Example usage:
+  #   query                       # Returns all jobs
+  #   query(:all)                 # Returns all jobs
+  #   query(:due)                 # Returns jobs which are due to be executed
+  #   query(:due, limit: 10)      # Returns at most 10 jobs which are due to be executed
+  #   query(job_class: Foo)       # Returns all jobs with job_class Foo
+  #   query(:due, job_class: Foo) # Returns jobs which are due to be executed with job_class Foo
+  #   query(queue_name: 'foo')    # Returns all jobs in the 'foo' queue
+  #   query(job_id: '123')        # Returns the job with job_id '123' or nil if no such job exists
+  #   query('123')                # Returns the job with job_id '123' or nil if no such job exists
   def query(*args, **kwargs)
     args, kwargs = args.dup, kwargs.dup.symbolize_keys
-    kwargs = kwargs.merge(queue_name: kwargs.delete(:queue)) if kwargs.key?(:queue)
     kwargs = kwargs.merge(job_id: kwargs.delete(:id)) if kwargs.key?(:id)
     kwargs[:job_id] = args.shift if args.size == 1 && args.first.is_a?(String) && args.first.match(JOB_ID_REGEX)
 
@@ -103,18 +113,16 @@ class MarjAdapter
       return record_class.find_by(job_id: kwargs[:job_id])&.to_job
     end
 
-    symbol_args = []
-    symbol_args << args.shift while args.first.is_a?(Symbol)
-    order_by = kwargs.delete(:order)
-    order_by = :queue_name if [:queue, 'queue'].include?(order_by)
+    symbol_args, args = args.partition { _1.is_a?(Symbol) }
+    symbol_args.delete(:all)
     limit = kwargs.delete(:limit)
-    symbol_args.shift if symbol_args.first == :all
     relation = record_class.all
-    relation = relation.order(order_by) if order_by
-    relation = relation.by_due_date unless relation.order_values.any?
+    relation = relation.order(kwargs.delete(:order)) if kwargs.key?(:order)
     relation = relation.where(*args, **kwargs) if args.any? || kwargs.any?
     relation = relation.limit(limit) if limit
     relation = relation.send(symbol_args.shift) while symbol_args.any?
+    relation = relation.by_due_date if relation.is_a?(ActiveRecord::Relation) && relation.order_values.empty?
+
     if relation.is_a?(Enumerable)
       relation.map(&:to_job)
     elsif relation.is_a?(record_class)
@@ -126,12 +134,46 @@ class MarjAdapter
 
   # Discards the specified job.
   #
+  # @param job [ActiveJob::Base] the job being discarded
+  # @param run_callbacks [Boolean] whether to run the +after_discard+ callbacks
   # @return [ActiveJob::Base] the discarded job
-  def discard(job)
+  def discard(job, run_callbacks: true)
+    job.tap do
+      @discard_proc.call(job)
+      run_after_discard_callbacks(job) if run_callbacks
+    end
+  end
+
+  # Deletes the record associated with the specified job.
+  #
+  # @return [ActiveJob::Base] the deleted job
+  def delete(job)
+    job.tap { destroy_record(job) }
+  end
+
+  private
+
+  # Returns the +ActiveRecord+ class to use to store jobs.
+  #
+  # @return [Class] the +ActiveRecord+ class
+  def record_class
+    @record_class = @record_class.is_a?(String) ? @record_class.constantize : @record_class
+  end
+
+  # Destroys the record associated with the given job if it exists.
+  #
+  # @return [ActiveRecord::Base, NilClass] the destroyed record or +nil+ if no such record exists
+  def destroy_record(job)
     record = job.singleton_class.instance_variable_get(:@record)
     record ||= Marj::Record.find_by(job_id: job.job_id)&.tap { _1.send(:register_callbacks, job) }
     record&.destroy
+  end
 
+  # Invokes the specified job's +after_discard+ callbacks.
+  #
+  # @param job [ActiveJob::Base] the job being discarded
+  # @return [NilClass] the given job
+  def run_after_discard_callbacks(job)
     # Copied from ActiveJob::Exceptions#run_after_discard_procs
     exceptions = []
     job.after_discard_procs.each do |blk|
@@ -141,12 +183,6 @@ class MarjAdapter
     end
     raise exceptions.last if exceptions.any?
 
-    job
-  end
-
-  private
-
-  def record_class
-    @record_class = @record_class.is_a?(String) ? @record_class.constantize : @record_class
+    nil
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: marj
 version: !ruby/object:Gem::Version
-  version: 5.0.0
+  version: 6.1.0
 platform: ruby
 authors:
 - Nick Dower
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2024-01-31 00:00:00.000000000 Z
+date: 2024-08-20 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activejob
@@ -48,6 +48,7 @@ files:
 - LICENSE.txt
 - README.md
 - lib/marj.rb
+- lib/marj/mission_control.rb
 - lib/marj/record.rb
 - lib/marj_adapter.rb
 homepage: https://github.com/nicholasdower/marj
@@ -55,8 +56,8 @@ licenses:
 - MIT
 metadata:
   bug_tracker_uri: https://github.com/nicholasdower/marj/issues
-  changelog_uri: https://github.com/nicholasdower/marj/releases/tag/v5.0.0
-  documentation_uri: https://www.rubydoc.info/github/nicholasdower/marj/v5.0.0
+  changelog_uri: https://github.com/nicholasdower/marj/releases/tag/v6.1.0
+  documentation_uri: https://www.rubydoc.info/github/nicholasdower/marj/v6.1.0
   homepage_uri: https://github.com/nicholasdower/marj
   rubygems_mfa_required: 'true'
   source_code_uri: https://github.com/nicholasdower/marj