marj 4.1.0 → 6.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 8ea99cb466811e6f77fb6476e7e8ca02b7345c8d297ebdb1954b9109f772bfd9
4
- data.tar.gz: 35a1ed99e2afe6934a5a7ea9b40375a93d5c79b9c46d304aac49457913794fba
3
+ metadata.gz: 6d0039931eb3f0bd43ce090b0b581d6544b26fda022729f6371f4c8ac393076e
4
+ data.tar.gz: 6abb66afe0bc56064be4ecee3f3e59e78adcf10afb83654529387b785f817335
5
5
  SHA512:
6
- metadata.gz: 0a1568a19c7faeedc80d0a17cf71c536a5dcb9b497645c87772213b5a20ca22b8ae250099826a75654d36260b5d34d77c896b81b28cc87e3ec1e4e181b5440bb
7
- data.tar.gz: 701727d777c8f9c990bdca6d66f9cf3d76d41eddddd77f1dbcbb1c75cff5b75b7acce065cb5c83b01c23dff3a6878218b4b6f3c4528d0e307ae8bc8e192d91e9
6
+ metadata.gz: ec25e50e772072eb6d13ebcb04a67e8ee4602782b62edab07e9521817993f6d92bac376bff90a6aeca1c724fa61dc4ba04ac1fe8ce9659139d2dcd98c397bce4
7
+ data.tar.gz: f78b5ba741202818a5374c95fddcb6258b091d4feba4b2de1d1f9de8d1422b99348bf5e354619d215b32b9b7f8d4055ef95d271055cf489e4049b06b40d3463c
data/README.md CHANGED
@@ -4,42 +4,89 @@ A minimal database-backed ActiveJob queueing backend.
4
4
 
5
5
  ## Quick Links
6
6
 
7
- API docs: https://gemdocs.org/gems/marj/latest <br>
7
+ API docs: https://gemdocs.org/gems/marj/6.0.0/ <br>
8
8
  RubyGems: https://rubygems.org/gems/marj <br>
9
9
  Changelog: https://github.com/nicholasdower/marj/releases <br>
10
10
  Issues: https://github.com/nicholasdower/marj/issues <br>
11
11
  Development: https://github.com/nicholasdower/marj/blob/master/CONTRIBUTING.md
12
12
 
13
+ ## Motivation
14
+
15
+ There are already several great database-backed ActiveJob queueing backends:
16
+ - [Delayed::Job](https://github.com/collectiveidea/delayed_job)
17
+ - [Que](https://github.com/que-rb/que)
18
+ - [GoodJob](https://github.com/bensheldon/good_job)
19
+ - [Solid Queue](https://github.com/basecamp/solid_queue)
20
+
21
+ Any of these which support your RDBMS is likely to work well for you. But you
22
+ may find them to be more featureful and complex than what you require.
23
+
24
+ Marj aims to be a minimal alternative.
25
+
26
+ ## Goal
27
+
28
+ To be the database-backend ActiveJob queueing backend with:
29
+ - The simplest setup
30
+ - The fewest configuration options
31
+ - The fewest features
32
+ - The fewest backend-specific APIs
33
+ - The fewest lines of code
34
+
13
35
  ## Features
14
36
 
37
+ Marj supports and has been tested with MySQL, PostgreSQL and SQLite.
38
+
39
+ It provides the following features:
15
40
  - Enqueued jobs are written to the database.
16
- - Successfully executed jobs are deleted from the database.
17
- - Failed jobs which should be retried are updated in the database.
18
- - Failed jobs which should not be retried are deleted from the database.
19
- - An interface is provided to retrieve, execute, discard and re-enqueue jobs.
20
- - An `ActiveRecord` class is provided to query the database directly.
41
+ - Enqueued jobs can be queried, executed and discarded.
42
+ - Executed jobs are re-enqueued or discarded, depending on the result.
43
+ - Compatibility with [Mission Control Jobs](https://github.com/basecamp/mission_control-jobs).
44
+
45
+ ## Extensions
46
+
47
+ Marj does not provide the following features by default, but each can easily
48
+ be added to your application with a few lines of code. See [Example Usage](#example-usage)
49
+ and [Extension Examples](#extension-examples) for suggestions.
50
+ - [Automatic job execution](#example-usage)
51
+ - [Concurrency protection](#concurrency-protection)
52
+ - [Job timeouts](#job-timeouts)
53
+ - [Error storage](#error-storage)
54
+ - [Discarded job retention](#discarded-job-retention)
55
+ - [Observability](#observability)
56
+ - [Multi-DB Support](#multi-db-support)
57
+
58
+ ## API
59
+
60
+ Marj relies on existing ActiveJob APIs, for example:
61
+
62
+ ```ruby
63
+ job.enqueue # Enqueues a job
64
+ job.perform_now # Performs a job
65
+ ```
66
+
67
+ Additionally, it extends the ActiveJob API with methods required for a
68
+ minimal queueing backend implementaion:
69
+
70
+ ```ruby
71
+ SomeJob.query(args) # Queries for enqeueued jobs
72
+ job.discard # Runs discard callbacks and, by default, deletes the job
73
+ job.delete # Deletes the job
74
+ ```
21
75
 
22
- ## Features Not Provided
76
+ ## Requirements
23
77
 
24
- - Workers
25
- - Timeouts
26
- - Concurrency Controls
27
- - Observability
28
- - A User Interace
78
+ - `activejob >= 7.1`
79
+ - `activerecord >= 7.1`
29
80
 
30
81
  ## Setup
31
82
 
32
83
  ### 1. Install
33
84
 
34
85
  ```shell
35
- bundle add activejob activerecord marj
36
-
37
- # or
38
-
39
- gem install activejob activerecord marj
86
+ bundle add marj
40
87
  ```
41
88
 
42
- ### 3. Create the database table
89
+ ### 2. Create the database table
43
90
 
44
91
  ```ruby
45
92
  class CreateJobs < ActiveRecord::Migration[7.1]
@@ -68,149 +115,74 @@ class CreateJobs < ActiveRecord::Migration[7.1]
68
115
  end
69
116
  ```
70
117
 
71
- Note that by default, Marj uses a table named `jobs`. To override the default
72
- table name, set `Marj.table_name` before loading `ActiveRecord`.
73
-
74
- ### 4. Configure the queue adapter
118
+ ### 3. Configure the queue adapter
75
119
 
76
120
  ```ruby
77
121
  require 'marj'
78
122
 
123
+ # Choose one of the following:
79
124
  Rails.configuration.active_job.queue_adapter = :marj # Globally, with Rails
80
125
  ActiveJob::Base.queue_adapter = :marj # Globally, without Rails
81
126
  SomeJob.queue_adapter = :marj # Single job
82
127
  ```
83
128
 
84
- ## Example Usage
85
-
86
- ```ruby
87
- # Enqueue and manually run a job:
88
- job = SomeJob.perform_later('foo')
89
- job.perform_now
129
+ ### 4. Optionally, include the Marj module
90
130
 
91
- # Retrieve and execute a job
92
- Marj.due.next.perform_now
93
-
94
- # Run all due jobs (single DB query)
95
- Marj.due.perform_all
96
-
97
- # Run all due jobs (multiple DB queries)
98
- Marj.due.perform_all(batch_size: 1)
99
-
100
- # Run all due jobs in a specific queue:
101
- Marj.queue('foo').due.perform_all
102
-
103
- # Run jobs as they become due:
104
- loop do
105
- Marj.due.perform_all rescue logger.error($!)
106
- ensure
107
- sleep 5.seconds
108
- end
109
- ```
110
-
111
- ## Jobs Interface
112
-
113
- The `Marj` module provides methods for interacting with enqueued jobs. These
114
- methods accept, return and yield +ActiveJob+ objects rather than +ActiveRecord+
115
- objects. Returned jobs are orderd by due date. To query the database directly,
116
- use `Marj::Record`.
117
-
118
- Example usage:
131
+ Without any additional setup, jobs can be queried and discarded via the `Marj`
132
+ module:
119
133
 
120
134
  ```ruby
121
- Marj.all # Returns all enqueued jobs.
122
- Marj.queue # Returns jobs in the specified queue(s).
123
- Marj.due # Returns jobs which are due to be executed.
124
- Marj.next # Returns the next job(s) to be executed.
125
- Marj.count # Returns the number of enqueued jobs.
126
- Marj.where # Returns jobs matching the specified criteria.
127
- Marj.perform_all # Executes all jobs.
128
- Marj.discard_all # Discards all jobs.
129
- Marj.discard # Discards the specified job.
135
+ Marj.query(:all)
136
+ Marj.discard(job)
130
137
  ```
131
138
 
132
- Query methods can also be chained:
139
+ But it is also convenient to query or discard via job classes:
133
140
 
134
141
  ```ruby
135
- Marj.due.where(job_class: SomeJob).next # Returns the next SomeJob that is due
142
+ SomeJob.query(:all)
143
+ SomeJob.discard(job)
144
+ job.discard
136
145
  ```
137
146
 
138
- ## Custom Jobs Interface
139
-
140
- The `Marj::JobsInterface` can be added to any class or module. For example, to
141
- add it to all jobs classes:
147
+ In order to enable this functionality, you must include the `Marj` module:
142
148
 
143
149
  ```ruby
144
- class ApplicationJob < ActiveJob::Base
145
- extend Marj::JobsInterface
150
+ class SomeJob < ActiveJob::Base
151
+ include Marj
146
152
 
147
- def self.all
148
- Marj::Relation.new(
149
- self == ApplicationJob ?
150
- Marj::Record.ordered : Marj::Record.where(job_class: self)
151
- )
152
- end
153
+ def perform; end
153
154
  end
154
-
155
- class SomeJob < ApplicationJob; end
156
-
157
- ApplicationJob.due # Returns all jobs which are due to be executed.
158
- SomeJob.due # Returns SomeJobs which are due to be executed.
159
155
  ```
160
156
 
161
- ## Multiple Tables
162
-
163
- It is possible to create a custom record class in order to, for instance,
164
- write jobs to multiple databases/tables within a single application.
157
+ ## Example Usage
165
158
 
166
159
  ```ruby
167
- class CreateMyJobs < ActiveRecord::Migration[7.1]
168
- def self.up
169
- create_table :my_jobs, id: :string, primary_key: :job_id do |table|
170
- table.string :job_class, null: false
171
- table.text :arguments, null: false
172
- table.string :queue_name, null: false
173
- table.integer :priority
174
- table.integer :executions, null: false
175
- table.text :exception_executions, null: false
176
- table.datetime :enqueued_at, null: false
177
- table.datetime :scheduled_at
178
- table.string :locale, null: false
179
- table.string :timezone, null: false
180
- end
181
-
182
- add_index :my_jobs, %i[enqueued_at]
183
- add_index :my_jobs, %i[scheduled_at]
184
- add_index :my_jobs, %i[priority scheduled_at enqueued_at]
185
- end
186
-
187
- def self.down
188
- drop_table :my_jobs
189
- end
190
- end
191
-
192
- class MyRecord < Marj::Record
193
- self.table_name = 'my_jobs'
194
- end
195
-
196
- CreateMyJobs.migrate(:up)
160
+ # Enqueue a job
161
+ job = SomeJob.perform_later('foo')
197
162
 
198
- class MyJob < ActiveJob::Base
199
- self.queue_adapter = MarjAdapter.new('MyRecord')
163
+ # Query jobs
164
+ SomeJob.query(:all)
165
+ SomeJob.query(:due)
166
+ SomeJob.query(:due, queue: :foo)
167
+ SomeJob.query('8720417d-8fff-4fcf-bc16-22aaef8543d2')
200
168
 
201
- extend Marj::JobsInterface
169
+ # Execute a job
170
+ job.perform_now
202
171
 
203
- def self.all
204
- Marj::Relation.new(MyRecord.all)
205
- end
172
+ # Execute jobs which are due (single query)
173
+ SomeJob.query(:due).map(&:perform_now)
206
174
 
207
- def perform(msg)
208
- puts msg
209
- end
175
+ # Execute jobs which are due (multiple queries)
176
+ loop {
177
+ SomeJob.query(:due, :first)&.tap(&:perform_now) || break
210
178
  end
211
179
 
212
- MyJob.perform_later('oh, hi')
213
- MyJob.due.next.perform_now
180
+ # Execute jobs as they become due
181
+ loop do
182
+ SomeJob.query(:due).each(&:perform_now) rescue logger.error($!)
183
+ ensure
184
+ sleep 5.seconds
185
+ end
214
186
  ```
215
187
 
216
188
  ## Testing
@@ -219,17 +191,54 @@ By default, jobs enqeued during tests will be written to the database. Enqueued
219
191
  jobs can be executed via:
220
192
 
221
193
  ```ruby
222
- Marj.due.perform_all
194
+ SomeJob.query(:due).map(&:perform_now)
223
195
  ```
224
196
 
225
- Alternatively, to use [ActiveJob::QueueAdapters::TestAdapter](https://api.rubyonrails.org/classes/ActiveJob/QueueAdapters/TestAdapter.html):
197
+ Alternatively, to use ActiveJob's [TestAdapter](https://api.rubyonrails.org/classes/ActiveJob/QueueAdapters/TestAdapter.html):
226
198
  ```ruby
227
199
  ActiveJob::Base.queue_adapter = :test
228
200
  ```
229
201
 
230
202
  ## Extension Examples
231
203
 
232
- ### Timeouts
204
+ Most missing features can easily be added to your application with a few lines
205
+ of code.
206
+
207
+ ### Concurrency Protection
208
+
209
+ To prevent two threads from executing the same job simultaneously, consider
210
+ moving jobs to a different queue before executing them.
211
+
212
+ ```ruby
213
+ class ApplicationJob < ActiveJob::Base
214
+ around_perform do |job, block|
215
+ if job.queue_name.start_with?('claimed')
216
+ raise "Job #{job.job_id} already claimed"
217
+ end
218
+
219
+ updated = Marj::Record
220
+ .where(job_id: job.job_id, queue_name: job.queue_name)
221
+ .update_all(
222
+ queue_name: "claimed-#{job.queue_name}",
223
+ scheduled_at: Time.now.utc
224
+ )
225
+ unless updated == 1
226
+ raise "Failed to claim job #{job.job_id}. #{updated} records updated"
227
+ end
228
+
229
+ begin
230
+ block.call
231
+ rescue StandardError
232
+ Marj::Record
233
+ .where(job_id: job.job_id, queue_name: "claimed-#{job.queue_name}")
234
+ .update_all(queue_name: job.queue_name, scheduled_at: job.scheduled_at)
235
+ raise
236
+ end
237
+ end
238
+ end
239
+ ```
240
+
241
+ ### Job Timeouts
233
242
 
234
243
  ```ruby
235
244
  class ApplicationJob < ActiveJob::Base
@@ -249,7 +258,7 @@ class ApplicationJob < ActiveJob::Base
249
258
  end
250
259
  ```
251
260
 
252
- ### Last Error
261
+ ### Error Storage
253
262
 
254
263
  ```ruby
255
264
  class AddLastErrorToJobs < ActiveRecord::Migration[7.1]
@@ -290,6 +299,83 @@ class ApplicationJob < ActiveJob::Base
290
299
  end
291
300
  ```
292
301
 
302
+ ### Discarded Job Retention
303
+
304
+ By default, discarded jobs are deleted from the database. If desired, this
305
+ behavior can be overridden, for instance to move jobs to a differnt queue:
306
+
307
+ ```ruby
308
+ ActiveJob::Base.queue_adapter = MarjAdapter.new(
309
+ discard: proc { _1.enqueue(queue: 'discarded') }
310
+ )
311
+
312
+ # To retrieve a discarded job, query the discarded queue:
313
+ job = Marj.query(:first, queue_name: 'discarded')
314
+
315
+ # To delete, use one of the following:
316
+ Marj.delete(job)
317
+ job.delete
318
+ ```
319
+
320
+ ### Observability
321
+
322
+ For instance with Prometheus metrics:
323
+
324
+ ```ruby
325
+ class ApplicationJob < ActiveJob::Base
326
+ counter = Prometheus::Client::Counter.new(
327
+ :job_events_total, docstring: '...', labels: [:job, :event]
328
+ )
329
+
330
+ around_enqueue do |job, block|
331
+ counter.increment(labels: { job: job.class.name, event: 'before_enqueue' })
332
+ block.call
333
+ counter.increment(labels: { job: job.class.name, event: 'after_enqueue' })
334
+ rescue Exception
335
+ counter.increment(labels: { job: job.class.name, event: 'enqueue_error' })
336
+ raise
337
+ end
338
+
339
+ around_perform do |job, block|
340
+ counter.increment(labels: { job: job.class.name, event: 'before_perform' })
341
+ block.call
342
+ counter.increment(labels: { job: job.class.name, event: 'after_perform' })
343
+ rescue Exception => e # rubocop:disable Lint/RescueException
344
+ counter.increment(labels: { job: job.class.name, event: 'perform_error' })
345
+ raise
346
+ end
347
+
348
+ after_discard do |job|
349
+ counter.increment(labels: { job: job.class.name, event: 'after_discard' })
350
+ end
351
+ end
352
+ ```
353
+
354
+ ### Multi-DB Support
355
+
356
+ It is possible to create a custom record class in order to, for instance,
357
+ use a different class or table name, or write jobs to multiple
358
+ databases/tables within a single application.
359
+
360
+ Assuming you have a jobs tabled named `my_jobs`:
361
+
362
+ ```ruby
363
+ class MyRecord < Marj::Record
364
+ self.table_name = 'my_jobs'
365
+ end
366
+
367
+ class MyJob < ActiveJob::Base
368
+ include Marj
369
+
370
+ self.queue_adapter = MarjAdapter.new('MyRecord')
371
+
372
+ def perform; end
373
+ end
374
+
375
+ MyJob.perform_later('oh, hi')
376
+ MyJob.query(:due, :first).perform_now
377
+ ```
378
+
293
379
  ## ActiveJob Cheatsheet
294
380
 
295
381
  For more information on ActiveJob, see:
@@ -0,0 +1,119 @@
1
+ # frozen_string_literal: true
2
+
3
+ require_relative '../marj_adapter'
4
+ require 'mission_control/jobs'
5
+ require 'mission_control/jobs/adapter'
6
+
7
+ # :nocov:
8
+ module Marj
9
+ module MissionControl
10
+ include ::MissionControl::Jobs::Adapter
11
+
12
+ def queues
13
+ record_class.group(:queue_name).count(:queue_name).map { |k, v| { name: k, size: v, active: true } }
14
+ end
15
+
16
+ def queue_size(queue_name)
17
+ record_class.where(queue_name: queue_name).count
18
+ end
19
+
20
+ def clear_queue(queue_name)
21
+ Marj::Record.where(queue_name: queue_name).delete_all
22
+ end
23
+
24
+ def pause_queue(_queue_name)
25
+ raise 'not supported: pause queue'
26
+ end
27
+
28
+ def resume_queue(_queue_name)
29
+ raise 'not supported: resume queue'
30
+ end
31
+
32
+ def queue_paused?(_queue_name)
33
+ false
34
+ end
35
+
36
+ def supported_statuses
37
+ %i[pending failed scheduled]
38
+ end
39
+
40
+ def supported_filters(_jobs_relation)
41
+ %i[queue_name job_class_name]
42
+ end
43
+
44
+ def exposes_workers?
45
+ false
46
+ end
47
+
48
+ def workers
49
+ raise 'not supported: workers'
50
+ end
51
+
52
+ def find_worker(_worker_id)
53
+ raise 'not supported: find workers'
54
+ end
55
+
56
+ def jobs_count(jobs_relation)
57
+ ar_relation(jobs_relation).count
58
+ end
59
+
60
+ def fetch_jobs(jobs_relation)
61
+ ar_relation(jobs_relation).each_with_index.map { |record, index| to_job(record, jobs_relation, index) }
62
+ end
63
+
64
+ def retry_all_jobs(jobs_relation)
65
+ ar_relation(jobs_relation).map { |record| record.to_job.perform_now }
66
+ end
67
+
68
+ def retry_job(job, _jobs_relation)
69
+ Marj::Record.find(job.job_id).to_job.perform_now
70
+ end
71
+
72
+ def discard_all_jobs(jobs_relation)
73
+ ar_relation(jobs_relation).map { |record| discard(record.to_job) }
74
+ end
75
+
76
+ def discard_job(job, _jobs_relation)
77
+ discard(Marj::Record.find(job.job_id).to_job)
78
+ end
79
+
80
+ def find_job(job_id, jobs_relation)
81
+ to_job(record_class.find_by(job_id: job_id), jobs_relation)
82
+ end
83
+
84
+ private
85
+
86
+ def ar_relation(jobs_relation)
87
+ relation = Marj::Record.all.offset(jobs_relation.offset_value).limit(jobs_relation.limit_value)
88
+ relation = relation.where.not(executions: 0) if jobs_relation.status == :failed
89
+ relation = relation.where.not(scheduled_at: nil) if jobs_relation.status == :scheduled
90
+ relation = relation.where(job_class: jobs_relation.job_class_name) if jobs_relation.job_class_name
91
+ relation = relation.where(queue_name: jobs_relation.queue_name) if jobs_relation.queue_name
92
+ relation
93
+ end
94
+
95
+ def to_job(record, jobs_relation, index = 0)
96
+ return nil unless record
97
+
98
+ job = record.to_job
99
+ job_data = job.serialize
100
+ ActiveJob::JobProxy.new(job_data).tap do |proxy|
101
+ if job.executions.positive?
102
+ proxy.last_execution_error = ActiveJob::ExecutionError.new(
103
+ error_class: Exception, message: 'unknown', backtrace: []
104
+ )
105
+ proxy.failed_at = job.enqueued_at
106
+ proxy.status = :failed
107
+ elsif job.scheduled_at
108
+ proxy.status = :scheduled
109
+ else
110
+ proxy.status = :pending
111
+ end
112
+ proxy.raw_data = job_data
113
+ proxy.position = jobs_relation.offset_value + index
114
+ proxy.arguments = job.arguments # For some reason MissionControl sets the arguments to the entire job data
115
+ end
116
+ end
117
+ end
118
+ end
119
+ # :nocov:
data/lib/marj/record.rb CHANGED
@@ -6,7 +6,7 @@ require 'active_record'
6
6
  module Marj
7
7
  # The default +ActiveRecord+ class.
8
8
  class Record < ActiveRecord::Base
9
- self.table_name = Marj.table_name
9
+ self.table_name = :jobs
10
10
 
11
11
  # Order by +enqueued_at+ rather than +job_id+ (the default).
12
12
  self.implicit_order_column = 'enqueued_at'
@@ -50,9 +50,47 @@ module Marj
50
50
  # discarded.
51
51
  #
52
52
  # @return [ActiveJob::Base]
53
- def as_job
54
- Marj.send(:to_job, self)
53
+ def to_job
54
+ # See register_callbacks for details on how callbacks are used.
55
+ job = job_class.new.tap { register_callbacks(_1) }
56
+
57
+ # ActiveJob::Base#deserialize expects serialized arguments. But the record arguments have already been
58
+ # deserialized by a custom ActiveRecord serializer (see below). So instead we use the raw arguments string.
59
+ job_data = attributes.merge('arguments' => JSON.parse(read_attribute_before_type_cast(:arguments)))
60
+
61
+ # ActiveJob::Base#deserialize expects dates to be strings rather than Time objects.
62
+ job_data = job_data.to_h { |k, v| [k, %w[enqueued_at scheduled_at].include?(k) ? v&.iso8601 : v] }
63
+
64
+ job.deserialize(job_data)
65
+
66
+ # ActiveJob deserializes arguments on demand when a job is performed. Until then they are empty. That's strange.
67
+ # Instead, deserialize them now. Also, clear `serialized_arguments` to prevent ActiveJob from overwriting changes
68
+ # to arguments when serializing later.
69
+ job.arguments = arguments
70
+ job.serialized_arguments = nil
71
+
72
+ job
73
+ end
74
+
75
+ # Registers callbacks for the given job which destroy this record when the job succeeds or is discarded.
76
+ #
77
+ # @param job [ActiveJob::Base]
78
+ # @return [ActiveJob::Base]
79
+ def register_callbacks(job)
80
+ raise 'callbacks already registered' if job.singleton_class.instance_variable_get(:@record)
81
+
82
+ record = self
83
+ # We need to detect three cases:
84
+ # - If a job succeeds, after_perform will be called.
85
+ # - If a job fails and should be retried, enqueue will be called. This is handled by the queue adapter.
86
+ # - If a job exceeds its max attempts, after_discard will be called.
87
+ job.singleton_class.after_perform { |_j| job.queue_adapter.delete(job) }
88
+ job.singleton_class.after_discard { |_j, _exception| job.queue_adapter.discard(job, run_callbacks: false) }
89
+ job.singleton_class.instance_variable_set(:@record, record)
90
+
91
+ job
55
92
  end
93
+ private :register_callbacks
56
94
 
57
95
  class << self
58
96
  # Returns an +ActiveRecord::Relation+ scope for enqueued jobs with a +scheduled_at+ that is either +null+ or in
@@ -63,11 +101,16 @@ module Marj
63
101
  where('scheduled_at IS NULL OR scheduled_at <= ?', Time.now.utc)
64
102
  end
65
103
 
66
- # Returns an +ActiveRecord::Relation+ scope for jobs ordered by +priority+ (+null+ last), then +scheduled_at+
67
- # (+null+ last), then +enqueued_at+.
104
+ # Returns an +ActiveRecord::Relation+ scope for jobs ordered by due date.
105
+ #
106
+ # Jobs are ordered by the following criteria, in order:
107
+ # 1. past or null scheduled_at before future scheduled_at
108
+ # 2. ascending priority, nulls last
109
+ # 3. ascending scheduled_at, nulls last
110
+ # 4. ascending enqueued_at
68
111
  #
69
112
  # @return [ActiveRecord::Relation]
70
- def ordered
113
+ def by_due_date
71
114
  order(
72
115
  Arel.sql(<<~SQL.squish, Time.now.utc)
73
116
  CASE WHEN scheduled_at IS NULL OR scheduled_at <= ? THEN 0 ELSE 1 END,
data/lib/marj.rb CHANGED
@@ -1,164 +1,131 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  require_relative 'marj_adapter'
4
- require_relative 'marj/jobs_interface'
5
- require_relative 'marj/relation'
6
4
 
7
5
  # A minimal database-backed ActiveJob queueing backend.
8
6
  #
9
- # The {Marj} module provides methods for interacting with enqueued jobs. These methods accept, return and yield
10
- # +ActiveJob+ objects rather than +ActiveRecord+ objects. Returned jobs are ordered by due date. To query the database
11
- # directly, use {Record}.
7
+ # The {Marj} module provides the following methods:
8
+ # - +query+ - Queries enqueued jobs
9
+ # - +discard+ - Discards a job, by default by executing after_discard callbacks and delegating to delete
10
+ # - +delete+ - Deletes a job
11
+ #
12
+ # It is possible to call the above methods on the {Marj} module itself or on any class which includes it.
12
13
  #
13
14
  # Example usage:
14
- # Marj.all # Returns all enqueued jobs.
15
- # Marj.queue # Returns jobs in the specified queue(s).
16
- # Marj.due # Returns jobs which are due to be executed.
17
- # Marj.next # Returns the next job(s) to be executed.
18
- # Marj.count # Returns the number of enqueued jobs.
19
- # Marj.where # Returns jobs matching the specified criteria.
20
- # Marj.perform_all # Executes all jobs.
21
- # Marj.discard_all # Discards all jobs.
22
- # Marj.discard # Discards the specified job.
15
+ # Marj.query(:first) # Returns the first job
16
+ # Marj.discard(job) # Discards the specified job
17
+ # Marj.delete(job) # Deletes the specified job
18
+ #
19
+ # class ApplicationJob < ActiveJob::Base
20
+ # include Marj
21
+ # end
23
22
  #
24
- # Query methods can also be chained:
25
- # Marj.due.where(job_class: SomeJob).next # Returns the next SomeJob that is due
23
+ # class SomeJob < ApplicationJob;
24
+ # def perform; end
25
+ # end
26
26
  #
27
- # Note that by default, Marj uses {Marj::Record} to interact with the +jobs+ table. To use a different record class, set
28
- # {record_class}. To simply override the table name, set {table_name} before loading +ActiveRecord+.
27
+ # job = ApplicationJob.query(:first) # Returns the first enqueued job
28
+ # job = SomeJob.query(:first) # Returns the first enqueued job with job_class SomeJob
29
+ # ApplicationJob.discard(job) # Discards the specified job
30
+ # ApplicationJob.delete(job) # Deletes the specified job
31
+ # job.discard # Discards the job
32
+ # job.delete # Deletes the job
29
33
  #
30
34
  # See https://github.com/nicholasdower/marj
31
35
  module Marj
32
36
  # The Marj version.
33
- VERSION = '4.1.0'
37
+ VERSION = '6.0.0'
34
38
 
35
39
  Kernel.autoload(:Record, File.expand_path(File.join('marj', 'record.rb'), __dir__))
36
40
 
37
- @table_name = :jobs
38
- @record_class = 'Marj::Record'
39
-
40
- class << self
41
- include Marj::JobsInterface
42
-
43
- # @!attribute record_class
44
- # The name of the +ActiveRecord+ class. Defaults to +Marj::Record+.
45
- # @return [Class, String]
46
-
47
- attr_writer :record_class
48
-
49
- def record_class
50
- @record_class = @record_class.is_a?(String) ? @record_class.constantize : @record_class
51
- end
52
-
53
- # @!attribute table_name
54
- # The name of the database table. Defaults to +:jobs+.
55
- # @return [Symbol, String]
56
- attr_accessor :table_name
57
-
58
- # Returns a {Marj::Relation} for all jobs in the order they should be executed.
41
+ # Provides the {query} and {discard} class methods.
42
+ module ClassMethods
43
+ # Queries enqueued jobs. Similar to +ActiveRecord.where+ with a few additional features:
44
+ # - Symbol arguments are treated as +ActiveRecord+ scopes.
45
+ # - If only a job ID is specified, the corresponding job is returned.
46
+ # - If +:limit+ is specified, the maximum number of jobs is limited.
47
+ # - If +:order+ is specified, the jobs are ordered by the given attribute.
59
48
  #
60
- # @return [Marj::Relation]
61
- def all
62
- Marj::Relation.new(Marj.record_class.ordered)
49
+ # By default jobs are ordered by when they should be executed.
50
+ #
51
+ # Example usage:
52
+ # query # Returns all jobs
53
+ # query(:all) # Returns all jobs
54
+ # query(:due) # Returns jobs which are due to be executed
55
+ # query(:due, limit: 10) # Returns at most 10 jobs which are due to be executed
56
+ # query(job_class: Foo) # Returns all jobs with job_class Foo
57
+ # query(:due, job_class: Foo) # Returns jobs which are due to be executed with job_class Foo
58
+ # query(queue_name: 'foo') # Returns all jobs in the 'foo' queue
59
+ # query(job_id: '123') # Returns the job with job_id '123' or nil if no such job exists
60
+ # query('123') # Returns the job with job_id '123' or nil if no such job exists
61
+ def query(*args, **kwargs)
62
+ kwargs[:job_class] ||= self if self < ActiveJob::Base && name != 'ApplicationJob'
63
+ queue_adapter.query(*args, **kwargs)
63
64
  end
64
65
 
65
66
  # Discards the specified job.
66
67
  #
67
- # @return [Integer] the number of discarded jobs
68
+ # @return [ActiveJob::Base] the discarded job
68
69
  def discard(job)
69
- all.where(job_id: job.job_id).discard_all
70
+ queue_adapter.discard(job)
70
71
  end
71
72
 
72
- private
73
-
74
- # Creates a job instance for the given record which will update the database when successfully executed, enqueued or
75
- # discarded.
73
+ # Deletes the record associated with the specified job.
76
74
  #
77
- # @param record [ActiveRecord::Base]
78
- # @return [ActiveJob::Base] the new job instance
79
- def to_job(record)
80
- # See register_callbacks for details on how callbacks are used.
81
- job = record.job_class.new.tap { register_callbacks(_1, record) }
82
-
83
- # ActiveJob::Base#deserialize expects serialized arguments. But the record arguments have already been
84
- # deserialized by a custom ActiveRecord serializer (see below). So instead we use the raw arguments string.
85
- job_data = record.attributes.merge('arguments' => JSON.parse(record.read_attribute_before_type_cast(:arguments)))
86
-
87
- # ActiveJob::Base#deserialize expects dates to be strings rather than Time objects.
88
- job_data = job_data.to_h { |k, v| [k, %w[enqueued_at scheduled_at].include?(k) ? v&.iso8601 : v] }
75
+ # @return [ActiveJob::Base] the deleted job
76
+ def delete(job)
77
+ queue_adapter.delete(job)
78
+ end
79
+ end
89
80
 
90
- job.deserialize(job_data)
81
+ # (see ClassMethods#query)
82
+ def self.query(*args, **kwargs)
83
+ queue_adapter.query(*args, **kwargs)
84
+ end
91
85
 
92
- # ActiveJob deserializes arguments on demand when a job is performed. Until then they are empty. That's strange.
93
- # Instead, deserialize them now. Also, clear `serialized_arguments` to prevent ActiveJob from overwriting changes
94
- # to arguments when serializing later.
95
- job.arguments = record.arguments
96
- job.serialized_arguments = nil
86
+ # (see ClassMethods#discard)
87
+ def self.discard(job)
88
+ queue_adapter.discard(job)
89
+ end
97
90
 
98
- job
99
- end
91
+ # (see ClassMethods#delete)
92
+ def self.delete(job)
93
+ queue_adapter.delete(job)
94
+ end
100
95
 
101
- # Registers callbacks for the given job which destroy the given database record when the job succeeds or is
102
- # discarded.
103
- #
104
- # @param job [ActiveJob::Base]
105
- # @param record [ActiveRecord::Base]
106
- # @return [ActiveJob::Base]
107
- def register_callbacks(job, record)
108
- raise 'callbacks already registered' if job.singleton_class.instance_variable_get(:@record)
96
+ # Deletes this job.
97
+ #
98
+ # @return [ActiveJob::Base] this job
99
+ def discard
100
+ self.class.queue_adapter.discard(self)
101
+ end
109
102
 
110
- # We need to detect three cases:
111
- # - If a job succeeds, after_perform will be called.
112
- # - If a job fails and should be retried, enqueue will be called. This is handled by the enqueue method.
113
- # - If a job exceeds its max attempts, after_discard will be called.
114
- job.singleton_class.after_perform { |_j| record.destroy! }
115
- job.singleton_class.after_discard { |_j, _exception| record.destroy! }
116
- job.singleton_class.instance_variable_set(:@record, record)
103
+ # Deletes the record associated with this job.
104
+ #
105
+ # @return [ActiveJob::Base] this job
106
+ def delete
107
+ self.class.queue_adapter.delete(self)
108
+ end
117
109
 
118
- job
119
- end
110
+ def self.included(clazz)
111
+ clazz.extend(ClassMethods)
112
+ end
113
+ private_class_method :included
120
114
 
121
- # Enqueue a job for execution at the specified time.
122
- #
123
- # @param job [ActiveJob::Base] the job to enqueue
124
- # @param record_class [Class] the +ActiveRecord+ class
125
- # @param time [Time, NilClass] optional time at which to execute the job
126
- # @return [ActiveJob::Base] the enqueued job
127
- def enqueue(job, record_class, time = nil)
128
- job.scheduled_at = time
129
- # Argument serialization is done by ActiveJob. ActiveRecord expects deserialized arguments.
130
- serialized = job.serialize.symbolize_keys!.without(:provider_job_id).merge(arguments: job.arguments)
115
+ def self.queue_adapter
116
+ ActiveJob::Base.queue_adapter
117
+ end
118
+ private_class_method :queue_adapter
119
+ end
131
120
 
132
- # When a job is enqueued, we must create/update the corresponding database record. We also must ensure callbacks
133
- # are registered on the job instance so that when the job is executed, the database record is deleted or updated
134
- # (depending on the result).
135
- #
136
- # We keep track of whether callbacks have been registered by setting the @record instance variable on the job's
137
- # singleton class. This holds a reference to the record. This ensures that if execute is called on a record
138
- # instance, any updates to the database are reflected on that record instance.
139
- if (existing_record = job.singleton_class.instance_variable_get(:@record))
140
- # This job instance has already been associated with a database row.
141
- if record_class.exists?(job_id: job.job_id)
142
- # The database row still exists, we simply need to update it.
143
- existing_record.update!(serialized)
144
- else
145
- # Someone else deleted the database row, we need to recreate and reload the existing record instance. We don't
146
- # want to register the new instance because someone might still have a reference to the existing one.
147
- record_class.create!(serialized)
148
- existing_record.reload
149
- end
150
- else
151
- # This job instance has not been associated with a database row.
152
- if (new_record = record_class.find_by(job_id: job.job_id))
153
- # The database row already exists. Update it.
154
- new_record.update!(serialized)
155
- else
156
- # The database row does not exist. Create it.
157
- new_record = record_class.create!(serialized)
158
- end
159
- register_callbacks(job, new_record)
160
- end
161
- job
162
- end
121
+ # :nocov:
122
+ if defined?(Rails)
123
+ begin
124
+ require 'mission_control/jobs'
125
+ require_relative 'marj/mission_control'
126
+ MarjAdapter.include(Marj::MissionControl)
127
+ rescue LoadError
128
+ # ignore
163
129
  end
164
130
  end
131
+ # :nocov:
data/lib/marj_adapter.rb CHANGED
@@ -2,13 +2,26 @@
2
2
 
3
3
  # ActiveJob queue adapter for Marj.
4
4
  #
5
+ # In addition to the standard +ActiveJob+ queue adapter API, this adapter provides:
6
+ # - A +query+ method which can be used to query enqueued jobs
7
+ # - A +discard+ method which can be used to discard enqueued jobs.
8
+ # - A +delete+ method which can be used to delete enqueued jobs.
9
+ #
10
+ # Although it is possible to access the adapter directly in order to query, discard or delete, it is recommended to use
11
+ # the {Marj} module.
12
+ #
5
13
  # See https://github.com/nicholasdower/marj
6
14
  class MarjAdapter
15
+ JOB_ID_REGEX = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/.freeze
16
+ private_constant :JOB_ID_REGEX
17
+
7
18
  # Creates a new adapter which will enqueue jobs using the given +ActiveRecord+ class.
8
19
  #
9
- # @param record_class [Class, String] the +ActiveRecord+ class (or its name) to use to store jobs
10
- def initialize(record_class = 'Marj::Record')
20
+ # @param record_class [Class, String] the +ActiveRecord+ class (or its name) to use, defaults to +Marj::Record+
21
+ # @param discard [Proc] the proc to use to discard jobs, defaults to delegating to {delete}
22
+ def initialize(record_class: 'Marj::Record', discard: proc { |job| delete(job) })
11
23
  @record_class = record_class
24
+ @discard_proc = discard
12
25
  end
13
26
 
14
27
  # Enqueue a job for immediate execution.
@@ -16,7 +29,7 @@ class MarjAdapter
16
29
  # @param job [ActiveJob::Base] the job to enqueue
17
30
  # @return [ActiveJob::Base] the enqueued job
18
31
  def enqueue(job)
19
- Marj.send(:enqueue, job, record_class)
32
+ enqueue_at(job)
20
33
  end
21
34
 
22
35
  # Enqueue a job for execution at the specified time.
@@ -24,13 +37,146 @@ class MarjAdapter
24
37
  # @param job [ActiveJob::Base] the job to enqueue
25
38
  # @param timestamp [Numeric, NilClass] optional number of seconds since Unix epoch at which to execute the job
26
39
  # @return [ActiveJob::Base] the enqueued job
27
- def enqueue_at(job, timestamp)
28
- Marj.send(:enqueue, job, record_class, timestamp ? Time.at(timestamp).utc : nil)
40
+ def enqueue_at(job, timestamp = nil)
41
+ job.scheduled_at = timestamp ? Time.at(timestamp).utc : nil
42
+
43
+ # Argument serialization is done by ActiveJob. ActiveRecord expects deserialized arguments.
44
+ serialized = job.serialize.symbolize_keys!.without(:provider_job_id).merge(arguments: job.arguments)
45
+
46
+ # Serialize sets locale to I18n.locale.to_s and enqueued_at to Time.now.utc.iso8601(9).
47
+ # Update the job to reflect what is being enqueued.
48
+ job.locale = serialized[:locale]
49
+ job.enqueued_at = Time.iso8601(serialized[:enqueued_at]).utc
50
+
51
+ # When a job is enqueued, we must create/update the corresponding database record. We also must ensure callbacks
52
+ # are registered on the job instance so that when the job is executed, the database record is deleted or updated
53
+ # (depending on the result).
54
+ #
55
+ # We keep track of whether callbacks have been registered by setting the @record instance variable on the job's
56
+ # singleton class. This holds a reference to the record. This ensures that if execute is called on a record
57
+ # instance, any updates to the database are reflected on that record instance.
58
+ if (existing_record = job.singleton_class.instance_variable_get(:@record))
59
+ # This job instance has already been associated with a database row.
60
+ if record_class.exists?(job_id: job.job_id)
61
+ # The database row still exists, we simply need to update it.
62
+ existing_record.update!(serialized)
63
+ else
64
+ # Someone else deleted the database row, we need to recreate and reload the existing record instance. We don't
65
+ # want to register the new instance because someone might still have a reference to the existing one.
66
+ record_class.create!(serialized)
67
+ existing_record.reload
68
+ end
69
+ else
70
+ # This job instance has not been associated with a database row.
71
+ if (new_record = record_class.find_by(job_id: job.job_id))
72
+ # The database row already exists. Update it.
73
+ new_record.update!(serialized)
74
+ else
75
+ # The database row does not exist. Create it.
76
+ new_record = record_class.create!(serialized)
77
+ end
78
+ new_record.send(:register_callbacks, job)
79
+ end
80
+ job
81
+ end
82
+
83
+ # Queries enqueued jobs. Similar to +ActiveRecord.where+ with a few additional features:
84
+ # - Symbol arguments are treated as +ActiveRecord+ scopes.
85
+ # - If only a job ID is specified, the corresponding job is returned.
86
+ # - If +:limit+ is specified, the maximum number of jobs is limited.
87
+ # - If +:order+ is specified, the jobs are ordered by the given attribute.
88
+ #
89
+ # By default jobs are ordered by when they should be executed.
90
+ #
91
+ # Example usage:
92
+ # query # Returns all jobs
93
+ # query(:all) # Returns all jobs
94
+ # query(:due) # Returns jobs which are due to be executed
95
+ # query(:due, limit: 10) # Returns at most 10 jobs which are due to be executed
96
+ # query(job_class: Foo) # Returns all jobs with job_class Foo
97
+ # query(:due, job_class: Foo) # Returns jobs which are due to be executed with job_class Foo
98
+ # query(queue_name: 'foo') # Returns all jobs in the 'foo' queue
99
+ # query(job_id: '123') # Returns the job with job_id '123' or nil if no such job exists
100
+ # query('123') # Returns the job with job_id '123' or nil if no such job exists
101
+ def query(*args, **kwargs)
102
+ args, kwargs = args.dup, kwargs.dup.symbolize_keys
103
+ kwargs = kwargs.merge(job_id: kwargs.delete(:id)) if kwargs.key?(:id)
104
+ kwargs[:job_id] = args.shift if args.size == 1 && args.first.is_a?(String) && args.first.match(JOB_ID_REGEX)
105
+
106
+ if args.empty? && kwargs.size == 1 && kwargs.key?(:job_id)
107
+ return record_class.find_by(job_id: kwargs[:job_id])&.to_job
108
+ end
109
+
110
+ symbol_args, args = args.partition { _1.is_a?(Symbol) }
111
+ symbol_args.delete(:all)
112
+ limit = kwargs.delete(:limit)
113
+ relation = record_class.all
114
+ relation = relation.order(kwargs.delete(:order)) if kwargs.key?(:order)
115
+ relation = relation.where(*args, **kwargs) if args.any? || kwargs.any?
116
+ relation = relation.limit(limit) if limit
117
+ relation = relation.send(symbol_args.shift) while symbol_args.any?
118
+ relation = relation.by_due_date if relation.is_a?(ActiveRecord::Relation) && relation.order_values.empty?
119
+
120
+ if relation.is_a?(Enumerable)
121
+ relation.map(&:to_job)
122
+ elsif relation.is_a?(record_class)
123
+ relation.to_job
124
+ else
125
+ relation
126
+ end
127
+ end
128
+
129
+ # Discards the specified job.
130
+ #
131
+ # @param job [ActiveJob::Base] the job being discarded
132
+ # @param run_callbacks [Boolean] whether to run the +after_discard+ callbacks
133
+ # @return [ActiveJob::Base] the discarded job
134
+ def discard(job, run_callbacks: true)
135
+ job.tap do
136
+ @discard_proc.call(job)
137
+ run_after_discard_callbacks(job) if run_callbacks
138
+ end
139
+ end
140
+
141
+ # Deletes the record associated with the specified job.
142
+ #
143
+ # @return [ActiveJob::Base] the deleted job
144
+ def delete(job)
145
+ job.tap { destroy_record(job) }
29
146
  end
30
147
 
31
148
  private
32
149
 
150
+ # Returns the +ActiveRecord+ class to use to store jobs.
151
+ #
152
+ # @return [Class] the +ActiveRecord+ class
33
153
  def record_class
34
154
  @record_class = @record_class.is_a?(String) ? @record_class.constantize : @record_class
35
155
  end
156
+
157
+ # Destroys the record associated with the given job if it exists.
158
+ #
159
+ # @return [ActiveRecord::Base, NilClass] the destroyed record or +nil+ if no such record exists
160
+ def destroy_record(job)
161
+ record = job.singleton_class.instance_variable_get(:@record)
162
+ record ||= Marj::Record.find_by(job_id: job.job_id)&.tap { _1.send(:register_callbacks, job) }
163
+ record&.destroy
164
+ end
165
+
166
+ # Invokes the specified job's +after_discard+ callbacks.
167
+ #
168
+ # @param job [ActiveJob::Base] the job being discarded
169
+ # @return [NilClass] the given job
170
+ def run_after_discard_callbacks(job)
171
+ # Copied from ActiveJob::Exceptions#run_after_discard_procs
172
+ exceptions = []
173
+ job.after_discard_procs.each do |blk|
174
+ instance_exec(job, nil, &blk)
175
+ rescue StandardError => e
176
+ exceptions << e
177
+ end
178
+ raise exceptions.last if exceptions.any?
179
+
180
+ nil
181
+ end
36
182
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: marj
3
3
  version: !ruby/object:Gem::Version
4
- version: 4.1.0
4
+ version: 6.0.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Nick Dower
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2024-01-28 00:00:00.000000000 Z
11
+ date: 2024-02-05 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: activejob
@@ -48,17 +48,16 @@ files:
48
48
  - LICENSE.txt
49
49
  - README.md
50
50
  - lib/marj.rb
51
- - lib/marj/jobs_interface.rb
51
+ - lib/marj/mission_control.rb
52
52
  - lib/marj/record.rb
53
- - lib/marj/relation.rb
54
53
  - lib/marj_adapter.rb
55
54
  homepage: https://github.com/nicholasdower/marj
56
55
  licenses:
57
56
  - MIT
58
57
  metadata:
59
58
  bug_tracker_uri: https://github.com/nicholasdower/marj/issues
60
- changelog_uri: https://github.com/nicholasdower/marj/releases/tag/v4.1.0
61
- documentation_uri: https://www.rubydoc.info/github/nicholasdower/marj/v4.1.0
59
+ changelog_uri: https://github.com/nicholasdower/marj/releases/tag/v6.0.0
60
+ documentation_uri: https://www.rubydoc.info/github/nicholasdower/marj/v6.0.0
62
61
  homepage_uri: https://github.com/nicholasdower/marj
63
62
  rubygems_mfa_required: 'true'
64
63
  source_code_uri: https://github.com/nicholasdower/marj
@@ -1,113 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- module Marj
4
- # The interface provided by {Marj} and {Marj::Relation}.
5
- #
6
- # To create a custom jobs interface, for example for all job classes in your application:
7
- # class ApplicationJob < ActiveJob::Base
8
- # extend Marj::JobsInterface
9
- #
10
- # def self.all
11
- # Marj::Relation.new(self == ApplicationJob ? Marj::Record.ordered : Marj::Record.where(job_class: self))
12
- # end
13
- # end
14
- #
15
- # class SomeJob < ApplicationJob
16
- # def perform(msg)
17
- # puts msg
18
- # end
19
- # end
20
- #
21
- # This will allow you to query jobs via the +ApplicationJob+ class:
22
- # ApplicationJob.next # Returns the next job of any type
23
- #
24
- # Or to query jobs via a specific job class:
25
- # SomeJob.next # Returns the next SomeJob
26
- #
27
- # Alternatively, to create a jobs interface for a single job class:
28
- # class SomeJob < ActiveJob::Base
29
- # extend Marj::JobsInterface
30
- #
31
- # def self.all
32
- # Marj::Relation.new(Marj::Record.where(job_class: self).ordered)
33
- # end
34
- # end
35
- module JobsInterface
36
- def self.included(clazz)
37
- return if clazz == Marj::Relation
38
-
39
- clazz.delegate :queue, :next, :count, :where, :due, :perform_all, :discard_all, to: :all
40
- end
41
- private_class_method :included
42
-
43
- def self.extended(clazz)
44
- clazz.singleton_class.delegate :queue, :next, :count, :where, :due, :perform_all, :discard_all, to: :all
45
- end
46
- private_class_method :extended
47
-
48
- # Returns a {Marj::Relation} for jobs in the specified queue(s).
49
- #
50
- # @param queue [String, Symbol] the queue to query
51
- # @param queues [Array<String>, Array<Symbol>] more queues to query
52
- # @return [Marj::Relation]
53
- def queue(queue, *queues)
54
- Marj::Relation.new(all.where(queue_name: queues.dup.unshift(queue)))
55
- end
56
-
57
- # Returns the next job or the next N jobs if +limit+ is specified. If no jobs exist, returns +nil+.
58
- #
59
- # @param limit [Integer, NilClass]
60
- # @return [ActiveJob::Base, NilClass]
61
- def next(limit = nil)
62
- all.first(limit)&.then { _1.is_a?(Array) ? _1.map(&:as_job) : _1.as_job }
63
- end
64
-
65
- # Returns a count of jobs, optionally either matching the specified column name criteria or where the specified
66
- # block returns +true+.
67
- #
68
- # @param column_name [String, Symbol, NilClass]
69
- # @param block [Proc, NilClass]
70
- # @return [Integer]
71
- def count(column_name = nil, &block)
72
- block_given? ? all.count(column_name) { |r| block.call(r.as_job) } : all.count(column_name)
73
- end
74
-
75
- # Returns a {Marj::Relation} for jobs matching the specified criteria.
76
- #
77
- # @param args [Array]
78
- # @return [Marj::Relation]
79
- def where(*args)
80
- Marj::Relation.new(all.where(*args))
81
- end
82
-
83
- # Returns a {Marj::Relation} for enqueued jobs with a +scheduled_at+ that is either +null+ or in the past.
84
- #
85
- # @return [Marj::Relation]
86
- def due
87
- Marj::Relation.new(all.due)
88
- end
89
-
90
- # Calls +perform_now+ on each job.
91
- #
92
- # @param batch_size [Integer, NilClass] the number of jobs to fetch at a time, or +nil+ to fetch all jobs at once
93
- # @return [Array] the results returned by each job
94
- def perform_all(batch_size: nil)
95
- if batch_size
96
- [].tap do |results|
97
- while (jobs = all.limit(batch_size).map(&:as_job)).any?
98
- results.concat(jobs.map { |job| ActiveJob::Callbacks.run_callbacks(:execute) { job.perform_now } })
99
- end
100
- end
101
- else
102
- all.map(&:as_job).map { |job| ActiveJob::Callbacks.run_callbacks(:execute) { job.perform_now } }
103
- end
104
- end
105
-
106
- # Discards all jobs.
107
- #
108
- # @return [Numeric] the number of discarded jobs
109
- def discard_all
110
- all.delete_all
111
- end
112
- end
113
- end
data/lib/marj/relation.rb DELETED
@@ -1,72 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- require_relative 'jobs_interface'
4
-
5
- module Marj
6
- # Returned by {Marj::JobsInterface} query methods to enable chaining and +Enumerable+ methods.
7
- class Relation
8
- include Enumerable
9
- include Marj::JobsInterface
10
-
11
- attr_reader :all
12
- private :all
13
-
14
- # Returns a {Marj::Relation} which wraps the specified +ActiveRecord+ relation.
15
- def initialize(ar_relation)
16
- @all = ar_relation
17
- end
18
-
19
- # Yields each job in this relation.
20
- #
21
- # @param block [Proc]
22
- # @return [Array] the jobs in this relation
23
- def each(&block)
24
- all.map(&:as_job).each(&block)
25
- end
26
-
27
- # Provides +pretty_inspect+ output containing arrays of jobs rather than arrays of records, similar to the output
28
- # produced when calling +pretty_inspect+ on +ActiveRecord::Relation+.
29
- #
30
- # Instead of the default +pretty_inspect+ output:
31
- # > Marj.all
32
- # =>
33
- # #<Marj::Relation:0x000000012728bd88
34
- # @ar_relation=
35
- # [#<Marj::Record:0x0000000126c42080
36
- # job_id: "1382cb98-c518-46ca-a0cc-d831e11a0714",
37
- # job_class: TestJob,
38
- # arguments: ["foo"],
39
- # queue_name: "default",
40
- # priority: nil,
41
- # executions: 0,
42
- # exception_executions: {},
43
- # enqueued_at: 2024-01-25 15:31:06.115773 UTC,
44
- # scheduled_at: nil,
45
- # locale: "en",
46
- # timezone: "UTC">]>
47
- #
48
- # Produces:
49
- # > Marj.all
50
- # =>
51
- # [#<TestJob:0x000000010b63cef8
52
- # @_scheduled_at_time=nil,
53
- # @arguments=[],
54
- # @enqueued_at=2024-01-25 15:31:06 UTC,
55
- # @exception_executions={},
56
- # @executions=0,
57
- # @job_id="1382cb98-c518-46ca-a0cc-d831e11a0714",
58
- # @locale="en",
59
- # @priority=nil,
60
- # @provider_job_id=nil,
61
- # @queue_name="default",
62
- # @scheduled_at=nil,
63
- # @serialized_arguments=["foo"],
64
- # @timezone="UTC">]
65
- #
66
- # @param pp [PP]
67
- # @return [NilClass]
68
- def pretty_print(pp)
69
- pp.pp(to_a)
70
- end
71
- end
72
- end