marj 2.0.1 → 3.0.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +151 -59
- data/lib/marj/jobs.rb +29 -0
- data/lib/marj/jobs_interface.rb +86 -0
- data/lib/marj/record.rb +17 -0
- data/lib/marj/record_interface.rb +94 -0
- data/lib/marj/relation.rb +112 -0
- data/lib/marj.rb +101 -3
- data/lib/marj_adapter.rb +15 -2
- metadata +10 -7
- data/lib/marj_config.rb +0 -15
- data/lib/marj_record.rb +0 -144
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 7afb73261695ca8eb0005d59b292ec659a9c10124a2b3984242e3f8e209fc897
|
4
|
+
data.tar.gz: 5f5fea852ab9a8e7eb9ca6fa59ed6cb4e0537651bf3edf6713ec0cad4a1c1ed0
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 348506cb34c7956e4c17554223c81d47353905ebdfb684c965d4deeb2b3a47727b5cb971211e2f2c48e039c8c5c1d9a1b02a6e9647be247ca0d86472a357f1b0
|
7
|
+
data.tar.gz: 94069a23d1882ff62c5b630e5e50357c63f230f8fc4a86828d3e4d785402f6c325fb82fb2d398c5ec937a572018b8c4027f5569fb818dcbc1dd2258e1cc9ac54
|
data/README.md
CHANGED
@@ -1,6 +1,6 @@
|
|
1
1
|
# Marj - Minimal ActiveRecord Jobs
|
2
2
|
|
3
|
-
|
3
|
+
A minimal database-backed ActiveJob queueing backend.
|
4
4
|
|
5
5
|
## Quick Links
|
6
6
|
|
@@ -16,6 +16,8 @@ Development: https://github.com/nicholasdower/marj/blob/master/CONTRIBUTING.md
|
|
16
16
|
- Successfully executed jobs are deleted from the database.
|
17
17
|
- Failed jobs which should be retried are updated in the database.
|
18
18
|
- Failed jobs which should not be retried are deleted from the database.
|
19
|
+
- An interface is provided to retrieve, execute, discard and re-enqueue jobs.
|
20
|
+
- An `ActiveRecord` model class is provided to query the database directly.
|
19
21
|
|
20
22
|
## Features Not Provided
|
21
23
|
|
@@ -24,38 +26,20 @@ Development: https://github.com/nicholasdower/marj/blob/master/CONTRIBUTING.md
|
|
24
26
|
- Concurrency Controls
|
25
27
|
- Observability
|
26
28
|
- A User Interace
|
27
|
-
- Anything else you might dream up.
|
28
|
-
|
29
|
-
## Interface
|
30
|
-
|
31
|
-
- `Marj` - An ActiveRecord model class
|
32
|
-
- `Marj.ready` - Used to retrieve jobs ready to be executed
|
33
|
-
- `Marj#execute` - Used to execute jobs retrieved from the database
|
34
|
-
- `MarjConfig.table_name=` - Used to override the default table name
|
35
29
|
|
36
30
|
## Setup
|
37
31
|
|
38
32
|
### 1. Install
|
39
33
|
|
40
34
|
```shell
|
41
|
-
bundle add marj
|
35
|
+
bundle add activejob activerecord marj
|
42
36
|
|
43
37
|
# or
|
44
38
|
|
45
|
-
gem install marj
|
46
|
-
```
|
47
|
-
|
48
|
-
### 2. Configure
|
49
|
-
|
50
|
-
By default, the database table is named "jobs". To use a different table name:
|
51
|
-
|
52
|
-
```ruby
|
53
|
-
require 'marj'
|
54
|
-
|
55
|
-
MarjConfig.table_name = 'some_name'
|
39
|
+
gem install activejob activerecord marj
|
56
40
|
```
|
57
41
|
|
58
|
-
### 3. Create the
|
42
|
+
### 3. Create the jobs table
|
59
43
|
|
60
44
|
```ruby
|
61
45
|
class CreateJobs < ActiveRecord::Migration[7.1]
|
@@ -89,18 +73,56 @@ end
|
|
89
73
|
```ruby
|
90
74
|
require 'marj'
|
91
75
|
|
92
|
-
#
|
93
|
-
|
94
|
-
|
95
|
-
|
76
|
+
Rails.configuration.active_job.queue_adapter = :marj # Globally, with Rails
|
77
|
+
ActiveJob::Base.queue_adapter = :marj # Globally, without Rails
|
78
|
+
SomeJob.queue_adapter = :marj # Single job
|
79
|
+
```
|
80
|
+
|
81
|
+
## Jobs Interface
|
82
|
+
|
83
|
+
`Marj::Jobs` provides a query interface (`Marj::JobsInterface`) which can be
|
84
|
+
used to retrieve, execute and discard enqueued jobs. It returns, yields and
|
85
|
+
accepts `ActiveJob` objects rather than `ActiveRecord` objects. Jobs are
|
86
|
+
orderd by due date. To query the database directly, use `Marj::Record`.
|
87
|
+
|
88
|
+
```ruby
|
89
|
+
Marj::Jobs.all # Returns all enqueued jobs.
|
90
|
+
Marj::Jobs.queue # Returns jobs in the specified queue(s).
|
91
|
+
Marj::Jobs.due # Returns jobs which are due to be executed.
|
92
|
+
Marj::Jobs.next # Returns the next job(s) to be executed.
|
93
|
+
Marj::Jobs.count # Returns the number of enqueued jobs.
|
94
|
+
Marj::Jobs.where # Returns jobs matching the specified criteria.
|
95
|
+
Marj::Jobs.perform_all # Executes all jobs.
|
96
|
+
Marj::Jobs.discard_all # Discards all jobs.
|
97
|
+
Marj::Jobs.discard # Discards the specified job.
|
98
|
+
```
|
99
|
+
|
100
|
+
`all`, `queue`, `due` and `where` return a `Marj::Relation` which provides
|
101
|
+
the same `Marj::JobsInterface`. This can be used to chain query methods like:
|
102
|
+
|
103
|
+
```ruby
|
104
|
+
Marj::Jobs.due.where(job_class: SomeJob).next
|
105
|
+
```
|
106
|
+
|
107
|
+
Note that the `Marj::JobsInterface` can be added to any class or module. For
|
108
|
+
example, to add it to all jobs classes:
|
96
109
|
|
97
|
-
|
98
|
-
ActiveJob::Base
|
110
|
+
```ruby
|
111
|
+
class ApplicationJob < ActiveJob::Base
|
112
|
+
extend Marj::JobsInterface
|
99
113
|
|
100
|
-
|
101
|
-
|
102
|
-
|
114
|
+
def self.all
|
115
|
+
Marj::Relation.new(
|
116
|
+
self == ApplicationJob ?
|
117
|
+
Marj::Record.ordered : Marj::Record.where(job_class: self)
|
118
|
+
)
|
119
|
+
end
|
103
120
|
end
|
121
|
+
|
122
|
+
class SomeJob < ApplicationJob; end
|
123
|
+
|
124
|
+
ApplicationJob.due # Returns all jobs which are due to be executed.
|
125
|
+
SomeJob.due # Returns SomeJobs which are due to be executed.
|
104
126
|
```
|
105
127
|
|
106
128
|
## Example Usage
|
@@ -110,35 +132,96 @@ end
|
|
110
132
|
job = SomeJob.perform_later('foo')
|
111
133
|
job.perform_now
|
112
134
|
|
113
|
-
#
|
114
|
-
|
115
|
-
Marj.first.execute
|
135
|
+
# Retrieve and execute a job
|
136
|
+
Marj::Jobs.due.next.perform_now
|
116
137
|
|
117
|
-
# Run all
|
118
|
-
Marj.
|
138
|
+
# Run all due jobs (single DB query)
|
139
|
+
Marj::Jobs.due.perform_all
|
119
140
|
|
120
|
-
# Run all
|
121
|
-
|
141
|
+
# Run all due jobs (multiple DB queries)
|
142
|
+
Marj::Jobs.due.perform_all(batch_size: 1)
|
122
143
|
|
123
|
-
# Run all
|
124
|
-
|
144
|
+
# Run all due jobs in a specific queue:
|
145
|
+
Marj::Jobs.queue('foo').due.perform_all
|
125
146
|
|
126
|
-
# Run jobs as they become
|
147
|
+
# Run all jobs indefinitely, as they become due:
|
127
148
|
loop do
|
128
|
-
|
129
|
-
rescue Exception => e
|
130
|
-
logger.error(e)
|
149
|
+
Marj::Jobs.due.perform_all rescue logger.error($!)
|
131
150
|
ensure
|
132
151
|
sleep 5.seconds
|
133
152
|
end
|
134
153
|
```
|
135
154
|
|
155
|
+
## Customization
|
156
|
+
|
157
|
+
It is possible to create a custom record class and jobs interface. This enables,
|
158
|
+
for instance, writing jobs to multiple databases/tables within a single
|
159
|
+
application.
|
160
|
+
|
161
|
+
```
|
162
|
+
class CreateMyJobs < ActiveRecord::Migration[7.1]
|
163
|
+
def self.up
|
164
|
+
create_table :my_jobs, id: :string, primary_key: :job_id do |table|
|
165
|
+
table.string :job_class, null: false
|
166
|
+
table.text :arguments, null: false
|
167
|
+
table.string :queue_name, null: false
|
168
|
+
table.integer :priority
|
169
|
+
table.integer :executions, null: false
|
170
|
+
table.text :exception_executions, null: false
|
171
|
+
table.datetime :enqueued_at, null: false
|
172
|
+
table.datetime :scheduled_at
|
173
|
+
table.string :locale, null: false
|
174
|
+
table.string :timezone, null: false
|
175
|
+
end
|
176
|
+
|
177
|
+
add_index :my_jobs, %i[enqueued_at]
|
178
|
+
add_index :my_jobs, %i[scheduled_at]
|
179
|
+
add_index :my_jobs, %i[priority scheduled_at enqueued_at]
|
180
|
+
end
|
181
|
+
|
182
|
+
def self.down
|
183
|
+
drop_table :my_jobs
|
184
|
+
end
|
185
|
+
end
|
186
|
+
|
187
|
+
class MyRecord < ActiveRecord::Base
|
188
|
+
include Marj::RecordInterface
|
189
|
+
|
190
|
+
self.table_name = 'my_jobs'
|
191
|
+
end
|
192
|
+
|
193
|
+
CreateMyJobs.migrate(:up)
|
194
|
+
|
195
|
+
class ApplicationJob < ActiveJob::Base
|
196
|
+
self.queue_adapter = MarjAdapter.new('MyRecord')
|
197
|
+
|
198
|
+
extend Marj::JobsInterface
|
199
|
+
|
200
|
+
def self.all
|
201
|
+
Marj::Relation.new(
|
202
|
+
self == ApplicationJob ?
|
203
|
+
MyRecord.ordered : MyRecord.where(job_class: self)
|
204
|
+
)
|
205
|
+
end
|
206
|
+
end
|
207
|
+
|
208
|
+
class MyJob < ApplicationJob
|
209
|
+
def perform(msg)
|
210
|
+
puts msg
|
211
|
+
end
|
212
|
+
end
|
213
|
+
|
214
|
+
MyJob.perform_later('oh, hi')
|
215
|
+
MyJob.due.next.perform_now
|
216
|
+
```
|
217
|
+
|
136
218
|
## Testing
|
137
219
|
|
138
|
-
By default, jobs enqeued during tests will be written to the database. Enqueued
|
220
|
+
By default, jobs enqeued during tests will be written to the database. Enqueued
|
221
|
+
jobs can be executed via:
|
139
222
|
|
140
223
|
```ruby
|
141
|
-
Marj.
|
224
|
+
Marj::Jobs.due.perform_all
|
142
225
|
```
|
143
226
|
|
144
227
|
Alternatively, to use [ActiveJob::QueueAdapters::TestAdapter](https://api.rubyonrails.org/classes/ActiveJob/QueueAdapters/TestAdapter.html):
|
@@ -158,7 +241,9 @@ class ApplicationJob < ActiveJob::Base
|
|
158
241
|
|
159
242
|
around_perform do |job, block|
|
160
243
|
if (timeout = job.class.instance_variable_get(:@timeout))
|
161
|
-
::Timeout.timeout(timeout, StandardError, 'execution expired')
|
244
|
+
::Timeout.timeout(timeout, StandardError, 'execution expired') do
|
245
|
+
block.call
|
246
|
+
end
|
162
247
|
else
|
163
248
|
block.call
|
164
249
|
end
|
@@ -185,7 +270,9 @@ class ApplicationJob < ActiveJob::Base
|
|
185
270
|
def last_error=(error)
|
186
271
|
if error.is_a?(Exception)
|
187
272
|
backtrace = error.backtrace&.map { |line| "\t#{line}" }&.join("\n")
|
188
|
-
error = backtrace ?
|
273
|
+
error = backtrace ?
|
274
|
+
"#{error.class}: #{error.message}\n#{backtrace}" :
|
275
|
+
"#{error.class}: #{error.message}"
|
189
276
|
end
|
190
277
|
|
191
278
|
@last_error = error&.truncate(10_000, omission: '… (truncated)')
|
@@ -217,10 +304,8 @@ For more information on ActiveJob, see:
|
|
217
304
|
|
218
305
|
```ruby
|
219
306
|
# With Rails
|
220
|
-
|
221
|
-
|
222
|
-
config.active_job.queue_adapter = FooAdapter.new # Uses FooAdapter directly
|
223
|
-
end
|
307
|
+
Rails.configuration.active_job.queue_adapter = :foo # Instantiates FooAdapter
|
308
|
+
Rails.configuration.active_job.queue_adapter = FooAdapter.new
|
224
309
|
|
225
310
|
# Without Rails
|
226
311
|
ActiveJob::Base.queue_adapter = :foo # Instantiates FooAdapter
|
@@ -237,11 +322,11 @@ SomeJob.queue_adapter = FooAdapter.new # Uses FooAdapter directly
|
|
237
322
|
- `config.active_job.queue_name_prefix`
|
238
323
|
- `config.active_job.queue_name_delimiter`
|
239
324
|
- `config.active_job.retry_jitter`
|
325
|
+
- `SomeJob.queue_name`
|
326
|
+
- `SomeJob.queue_as`
|
240
327
|
- `SomeJob.queue_name_prefix`
|
241
328
|
- `SomeJob.queue_name_delimiter`
|
242
329
|
- `SomeJob.retry_jitter`
|
243
|
-
- `SomeJob.queue_name`
|
244
|
-
- `SomeJob.queue_as`
|
245
330
|
|
246
331
|
### Options
|
247
332
|
|
@@ -274,33 +359,40 @@ SomeJob.queue_adapter = FooAdapter.new # Uses FooAdapter directly
|
|
274
359
|
# Create without enqueueing
|
275
360
|
job = SomeJob.new
|
276
361
|
job = SomeJob.new(args)
|
362
|
+
job = SomeJob.new.deserialize(other_job.serialize)
|
277
363
|
|
278
364
|
# Create and enqueue
|
279
365
|
job = SomeJob.perform_later
|
280
366
|
job = SomeJob.perform_later(args)
|
281
367
|
|
282
|
-
# Create and run (enqueued on failure)
|
368
|
+
# Create without enqueueing and run (only enqueued on failure if retryable)
|
283
369
|
SomeJob.perform_now
|
284
370
|
SomeJob.perform_now(args)
|
285
371
|
```
|
286
372
|
|
287
373
|
### Enqueueing Jobs
|
288
374
|
|
375
|
+
Jobs are enqueued via the `ActiveJob::Base#enqueue` method. This method returns
|
376
|
+
the job on success. If an error is raised during enqueueing, that error will
|
377
|
+
propagate to the caller, unless the error is an `ActiveJob::EnqueueError`. In
|
378
|
+
this case, `enqueue` will return `false` and `job.enqueue_error` will be set.
|
379
|
+
|
289
380
|
```ruby
|
290
381
|
SomeJob.new(args).enqueue
|
291
382
|
SomeJob.new(args).enqueue(options)
|
292
383
|
|
384
|
+
# Via perform_later
|
385
|
+
SomeJob.perform_later(SomeJob.new(args))
|
293
386
|
SomeJob.perform_later(args)
|
294
387
|
SomeJob.set(options).perform_later(args)
|
295
388
|
|
296
|
-
#
|
389
|
+
# After a failure during execution
|
297
390
|
SomeJob.perform_now(args)
|
391
|
+
ActiveJob::Base.execute(SomeJob.new(args).serialize)
|
298
392
|
|
299
393
|
# Enqueue multiple
|
300
394
|
ActiveJob.perform_all_later(SomeJob.new, SomeJob.new)
|
301
395
|
ActiveJob.perform_all_later(SomeJob.new, SomeJob.new, options:)
|
302
|
-
|
303
|
-
# Enqueue multiple
|
304
396
|
SomeJob.set(options).perform_all_later(SomeJob.new, SomeJob.new)
|
305
397
|
SomeJob.set(options).perform_all_later(SomeJob.new, SomeJob.new, options:)
|
306
398
|
```
|
@@ -308,7 +400,7 @@ SomeJob.set(options).perform_all_later(SomeJob.new, SomeJob.new, options:)
|
|
308
400
|
### Executing Jobs
|
309
401
|
|
310
402
|
```ruby
|
311
|
-
# Executed without enqueueing, enqueued on failure if
|
403
|
+
# Executed without enqueueing, enqueued on failure if retryable
|
312
404
|
SomeJob.new(args).perform_now
|
313
405
|
SomeJob.perform_now(args)
|
314
406
|
ActiveJob::Base.execute(SomeJob.new(args).serialize)
|
data/lib/marj/jobs.rb
ADDED
@@ -0,0 +1,29 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require_relative 'jobs_interface'
|
4
|
+
require_relative 'relation'
|
5
|
+
|
6
|
+
module Marj
|
7
|
+
# Provides methods for querying, performing and discarding jobs. Returns, yields and accepts
|
8
|
+
# +ActiveJob+ objects rather than +ActiveRecord+ objects. To query the database directly, use
|
9
|
+
# {Marj::Record}.
|
10
|
+
#
|
11
|
+
# To create a custom jobs interface, see {Marj::JobsInterface}.
|
12
|
+
module Jobs
|
13
|
+
singleton_class.include Marj::JobsInterface
|
14
|
+
|
15
|
+
# Returns a {Marj::Relation} for all jobs in the order they should be executed.
|
16
|
+
#
|
17
|
+
# @return [Marj::Relation]
|
18
|
+
def self.all
|
19
|
+
Marj::Relation.new(Marj::Record.ordered)
|
20
|
+
end
|
21
|
+
|
22
|
+
# Discards the specified job.
|
23
|
+
#
|
24
|
+
# @return [Integer] the number of discarded jobs
|
25
|
+
def self.discard(job)
|
26
|
+
all.where(job_id: job.job_id).discard_all
|
27
|
+
end
|
28
|
+
end
|
29
|
+
end
|
@@ -0,0 +1,86 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Marj
|
4
|
+
# The interface provided by {Marj::Jobs} and {Marj::Relation}. Include to create a custom jobs interface.
|
5
|
+
#
|
6
|
+
# To create a jobs interface for all job classes:
|
7
|
+
# class ApplicationJob < ActiveJob::Base
|
8
|
+
# extend Marj::JobsInterface
|
9
|
+
#
|
10
|
+
# def self.all
|
11
|
+
# Marj::Relation.new(self == ApplicationJob ? Marj::Record.ordered : Marj::Record.where(job_class: self))
|
12
|
+
# end
|
13
|
+
# end
|
14
|
+
#
|
15
|
+
# ApplicationJob.next
|
16
|
+
# SomeJob.next
|
17
|
+
#
|
18
|
+
# To create a jobs interface for a single job class:
|
19
|
+
# class SomeJob < ActiveJob::Base
|
20
|
+
# extend Marj::JobsInterface
|
21
|
+
#
|
22
|
+
# def self.all
|
23
|
+
# Marj::Relation.new(Marj::Record.where(job_class: self).ordered)
|
24
|
+
# end
|
25
|
+
# end
|
26
|
+
#
|
27
|
+
# SomeJob.next
|
28
|
+
module JobsInterface
|
29
|
+
# Returns a {Marj::Relation} for jobs in the specified queue(s).
|
30
|
+
#
|
31
|
+
# @param queue [String, Symbol] the queue to query
|
32
|
+
# @param queues [Array<String, Array<Symbol>] more queues to query
|
33
|
+
# @return [Marj::Relation]
|
34
|
+
def queue(queue, *queues)
|
35
|
+
all.queue(queue, *queues)
|
36
|
+
end
|
37
|
+
|
38
|
+
# Returns the next job or the next N jobs if +limit+ is specified. If no jobs exist, returns +nil+.
|
39
|
+
#
|
40
|
+
# @param limit [Integer, NilClass]
|
41
|
+
# @return [ActiveJob::Base, NilClass]
|
42
|
+
def next(limit = nil)
|
43
|
+
all.next(limit)
|
44
|
+
end
|
45
|
+
|
46
|
+
# Returns a count of jobs, optionally either matching the specified column name criteria or where the specified
|
47
|
+
# block returns +true+.
|
48
|
+
#
|
49
|
+
# @param column_name [String, Symbol, NilClass]
|
50
|
+
# @param block [Proc, NilClass]
|
51
|
+
# @return [Integer]
|
52
|
+
def count(column_name = nil, &block)
|
53
|
+
all.count(column_name, &block)
|
54
|
+
end
|
55
|
+
|
56
|
+
# Returns a {Marj::Relation} for jobs matching the specified criteria.
|
57
|
+
#
|
58
|
+
# @param args [Array]
|
59
|
+
# @return [Marj::Relation]
|
60
|
+
def where(*args)
|
61
|
+
all.where(*args)
|
62
|
+
end
|
63
|
+
|
64
|
+
# Returns a {Marj::Relation} for enqueued jobs with a +scheduled_at+ that is either +null+ or in the past.
|
65
|
+
#
|
66
|
+
# @return [Marj::Relation]
|
67
|
+
def due
|
68
|
+
all.due
|
69
|
+
end
|
70
|
+
|
71
|
+
# Calls +perform_now+ on each job.
|
72
|
+
#
|
73
|
+
# @param batch_size [Integer, NilClass] the number of jobs to fetch at a time, or +nil+ to fetch all jobs at once
|
74
|
+
# @return [Array] the results returned by each job
|
75
|
+
def perform_all(batch_size: nil)
|
76
|
+
all.perform_all(batch_size: batch_size)
|
77
|
+
end
|
78
|
+
|
79
|
+
# Discards all jobs.
|
80
|
+
#
|
81
|
+
# @return [Numeric] the number of discarded jobs
|
82
|
+
def discard_all
|
83
|
+
all.discard_all
|
84
|
+
end
|
85
|
+
end
|
86
|
+
end
|
data/lib/marj/record.rb
ADDED
@@ -0,0 +1,17 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'active_job'
|
4
|
+
require 'active_record'
|
5
|
+
require_relative 'record_interface'
|
6
|
+
|
7
|
+
module Marj
|
8
|
+
# The Marj ActiveRecord model class.
|
9
|
+
#
|
10
|
+
# See https://github.com/nicholasdower/marj
|
11
|
+
class Record < ActiveRecord::Base
|
12
|
+
include Marj::RecordInterface
|
13
|
+
extend Marj::RecordInterface::ClassMethods # Added explicitly to generate docs
|
14
|
+
|
15
|
+
self.table_name = 'jobs'
|
16
|
+
end
|
17
|
+
end
|
@@ -0,0 +1,94 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'active_job/arguments'
|
4
|
+
|
5
|
+
module Marj
|
6
|
+
# Provides base functionality for {Marj::Record}. Can be used to create a custom +ActiveRecord+ model class.
|
7
|
+
#
|
8
|
+
# Example Usage:
|
9
|
+
# class MyRecord < ActiveRecord::Base
|
10
|
+
# include Marj::RecordInterface
|
11
|
+
#
|
12
|
+
# self.table_name = 'my_jobs'
|
13
|
+
# end
|
14
|
+
module RecordInterface
|
15
|
+
# Adds {ClassMethods}, custom serializers and an implicit order column to the including class.
|
16
|
+
#
|
17
|
+
# @param clazz [Class] the including class
|
18
|
+
def self.included(clazz)
|
19
|
+
clazz.extend(ClassMethods)
|
20
|
+
|
21
|
+
# Order by +enqueued_at+ rather than +job_id+ (the default).
|
22
|
+
clazz.implicit_order_column = 'enqueued_at'
|
23
|
+
|
24
|
+
# Using a custom serializer for exception_executions so that we can interact with it as a hash rather than a
|
25
|
+
# string.
|
26
|
+
clazz.serialize(:exception_executions, coder: JSON)
|
27
|
+
|
28
|
+
# Using a custom serializer for arguments so that we can interact with as an array rather than a string.
|
29
|
+
# This enables code like:
|
30
|
+
# Marj::Record.next.arguments.first
|
31
|
+
# Marj::Record.next.update!(arguments: ['foo', 1, Time.now])
|
32
|
+
clazz.serialize(:arguments, coder: Class.new do
|
33
|
+
def self.dump(arguments)
|
34
|
+
return ActiveJob::Arguments.serialize(arguments).to_json if arguments.is_a?(Array)
|
35
|
+
return arguments if arguments.is_a?(String) || arguments.nil?
|
36
|
+
|
37
|
+
raise "invalid arguments: #{arguments}"
|
38
|
+
end
|
39
|
+
|
40
|
+
def self.load(arguments)
|
41
|
+
arguments ? ActiveJob::Arguments.deserialize(JSON.parse(arguments)) : nil
|
42
|
+
end
|
43
|
+
end)
|
44
|
+
|
45
|
+
# Using a custom serializer for job_class so that we can interact with it as a class rather than a string.
|
46
|
+
clazz.serialize(:job_class, coder: Class.new do
|
47
|
+
def self.dump(clazz)
|
48
|
+
return clazz.name if clazz.is_a?(Class)
|
49
|
+
return clazz if clazz.is_a?(String) || clazz.nil?
|
50
|
+
|
51
|
+
raise "invalid class: #{clazz}"
|
52
|
+
end
|
53
|
+
|
54
|
+
def self.load(str)
|
55
|
+
str&.constantize
|
56
|
+
end
|
57
|
+
end)
|
58
|
+
end
|
59
|
+
|
60
|
+
# Class methods for {Marj::RecordInterface}.
|
61
|
+
module ClassMethods
|
62
|
+
# Returns an +ActiveRecord::Relation+ scope for enqueued jobs with a +scheduled_at+ that is either +null+ or in
|
63
|
+
# the past.
|
64
|
+
#
|
65
|
+
# @return [ActiveRecord::Relation]
|
66
|
+
def due
|
67
|
+
where('scheduled_at IS NULL OR scheduled_at <= ?', Time.now.utc)
|
68
|
+
end
|
69
|
+
|
70
|
+
# Returns an +ActiveRecord::Relation+ scope for jobs ordered by +priority+ (+null+ last), then +scheduled_at+
|
71
|
+
# (+null+ last), then +enqueued_at+.
|
72
|
+
#
|
73
|
+
# @return [ActiveRecord::Relation]
|
74
|
+
def ordered
|
75
|
+
order(
|
76
|
+
Arel.sql(<<~SQL.squish, Time.now.utc)
|
77
|
+
CASE WHEN scheduled_at IS NULL OR scheduled_at <= ? THEN 0 ELSE 1 END,
|
78
|
+
CASE WHEN priority IS NULL THEN 1 ELSE 0 END, priority,
|
79
|
+
CASE WHEN scheduled_at IS NULL THEN 1 ELSE 0 END, scheduled_at,
|
80
|
+
enqueued_at
|
81
|
+
SQL
|
82
|
+
)
|
83
|
+
end
|
84
|
+
end
|
85
|
+
|
86
|
+
# Returns a job object for this record which will update the database when successfully executed, enqueued or
|
87
|
+
# discarded.
|
88
|
+
#
|
89
|
+
# @return [ActiveJob::Base]
|
90
|
+
def as_job
|
91
|
+
Marj.send(:to_job, self)
|
92
|
+
end
|
93
|
+
end
|
94
|
+
end
|
@@ -0,0 +1,112 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require_relative 'jobs_interface'
|
4
|
+
|
5
|
+
module Marj
|
6
|
+
# Returned by {Marj::JobsInterface} query methods to enable chaining and +Enumerable+ methods.
|
7
|
+
class Relation
|
8
|
+
include Enumerable
|
9
|
+
include Marj::JobsInterface
|
10
|
+
|
11
|
+
# Returns a Marj::Relation which wraps the specified +ActiveRecord+ relation.
|
12
|
+
def initialize(ar_relation)
|
13
|
+
@ar_relation = ar_relation
|
14
|
+
end
|
15
|
+
|
16
|
+
# (see Marj::JobsInterface#queue)
|
17
|
+
def queue(queue, *queues)
|
18
|
+
Marj::Relation.new(@ar_relation.where(queue_name: queues.dup.unshift(queue)))
|
19
|
+
end
|
20
|
+
|
21
|
+
# (see Marj::JobsInterface#next)
|
22
|
+
def next(limit = nil)
|
23
|
+
@ar_relation.first(limit)&.then { _1.is_a?(Array) ? _1.map(&:as_job) : _1.as_job }
|
24
|
+
end
|
25
|
+
|
26
|
+
# (see Marj::JobsInterface#count)
|
27
|
+
def count(column_name = nil, &block)
|
28
|
+
block_given? ? @ar_relation.count(column_name) { |r| block.call(r.as_job) } : @ar_relation.count(column_name)
|
29
|
+
end
|
30
|
+
|
31
|
+
# (see Marj::JobsInterface#where)
|
32
|
+
def where(*args)
|
33
|
+
Marj::Relation.new(@ar_relation.where(*args))
|
34
|
+
end
|
35
|
+
|
36
|
+
# (see Marj::JobsInterface#due)
|
37
|
+
def due
|
38
|
+
Marj::Relation.new(@ar_relation.due)
|
39
|
+
end
|
40
|
+
|
41
|
+
# (see Marj::JobsInterface#perform_all)
|
42
|
+
def perform_all(batch_size: nil)
|
43
|
+
if batch_size
|
44
|
+
[].tap do |results|
|
45
|
+
while (jobs = @ar_relation.limit(batch_size).map(&:as_job)).any?
|
46
|
+
results.concat(jobs.map { |job| ActiveJob::Callbacks.run_callbacks(:execute) { job.perform_now } })
|
47
|
+
end
|
48
|
+
end
|
49
|
+
else
|
50
|
+
@ar_relation.map(&:as_job).map { |job| ActiveJob::Callbacks.run_callbacks(:execute) { job.perform_now } }
|
51
|
+
end
|
52
|
+
end
|
53
|
+
|
54
|
+
# (see Marj::JobsInterface#discard_all)
|
55
|
+
def discard_all
|
56
|
+
@ar_relation.delete_all
|
57
|
+
end
|
58
|
+
|
59
|
+
# Yields each job in this relation.
|
60
|
+
#
|
61
|
+
# @param block [Proc]
|
62
|
+
# @return [Array] the jobs in this relation
|
63
|
+
def each(&block)
|
64
|
+
@ar_relation.map(&:as_job).each(&block)
|
65
|
+
end
|
66
|
+
|
67
|
+
# Provides +pretty_inspect+ output containing arrays of jobs rather than arrays of records, similar to the output
|
68
|
+
# produced when calling +pretty_inspect+ on +ActiveRecord::Relation+.
|
69
|
+
#
|
70
|
+
# Instead of the default +pretty_inspect+ output:
|
71
|
+
# > Marj::Jobs.all
|
72
|
+
# =>
|
73
|
+
# #<Marj::Relation:0x000000012728bd88
|
74
|
+
# @ar_relation=
|
75
|
+
# [#<Marj::Record:0x0000000126c42080
|
76
|
+
# job_id: "1382cb98-c518-46ca-a0cc-d831e11a0714",
|
77
|
+
# job_class: TestJob,
|
78
|
+
# arguments: ["foo"],
|
79
|
+
# queue_name: "default",
|
80
|
+
# priority: nil,
|
81
|
+
# executions: 0,
|
82
|
+
# exception_executions: {},
|
83
|
+
# enqueued_at: 2024-01-25 15:31:06.115773 UTC,
|
84
|
+
# scheduled_at: nil,
|
85
|
+
# locale: "en",
|
86
|
+
# timezone: "UTC">]>
|
87
|
+
#
|
88
|
+
# Produces:
|
89
|
+
# > Marj::Jobs.all
|
90
|
+
# =>
|
91
|
+
# [#<TestJob:0x000000010b63cef8
|
92
|
+
# @_scheduled_at_time=nil,
|
93
|
+
# @arguments=[],
|
94
|
+
# @enqueued_at=2024-01-25 15:31:06 UTC,
|
95
|
+
# @exception_executions={},
|
96
|
+
# @executions=0,
|
97
|
+
# @job_id="1382cb98-c518-46ca-a0cc-d831e11a0714",
|
98
|
+
# @locale="en",
|
99
|
+
# @priority=nil,
|
100
|
+
# @provider_job_id=nil,
|
101
|
+
# @queue_name="default",
|
102
|
+
# @scheduled_at=nil,
|
103
|
+
# @serialized_arguments=["foo"],
|
104
|
+
# @timezone="UTC">]
|
105
|
+
#
|
106
|
+
# @param pp [PP]
|
107
|
+
# @return [NilClass]
|
108
|
+
def pretty_print(pp)
|
109
|
+
pp.pp(to_a)
|
110
|
+
end
|
111
|
+
end
|
112
|
+
end
|
data/lib/marj.rb
CHANGED
@@ -1,8 +1,106 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
|
+
require_relative 'marj_adapter'
|
4
|
+
require_relative 'marj/jobs'
|
5
|
+
require_relative 'marj/jobs_interface'
|
6
|
+
require_relative 'marj/record_interface'
|
7
|
+
require_relative 'marj/relation'
|
8
|
+
|
9
|
+
# A minimal database-backed ActiveJob queueing backend.
|
10
|
+
#
|
3
11
|
# See https://github.com/nicholasdower/marj
|
12
|
+
module Marj
|
13
|
+
# The Marj version.
|
14
|
+
VERSION = '3.0.0'
|
4
15
|
|
5
|
-
|
6
|
-
|
16
|
+
Kernel.autoload(:Record, File.expand_path(File.join('marj', 'record.rb'), __dir__))
|
17
|
+
|
18
|
+
class << self
|
19
|
+
private
|
20
|
+
|
21
|
+
# Creates a job instance for the given record which will update the database when successfully executed, enqueued or
|
22
|
+
# discarded.
|
23
|
+
#
|
24
|
+
# @param record [ActiveRecord::Base]
|
25
|
+
# @return [ActiveJob::Base] the new job instance
|
26
|
+
def to_job(record)
|
27
|
+
# See register_callbacks for details on how callbacks are used.
|
28
|
+
job = record.job_class.new.tap { register_callbacks(_1, record) }
|
29
|
+
|
30
|
+
# ActiveJob::Base#deserialize expects serialized arguments. But the record arguments have already been
|
31
|
+
# deserialized by a custom ActiveRecord serializer (see below). So instead we use the raw arguments string.
|
32
|
+
job_data = record.attributes.merge('arguments' => JSON.parse(record.read_attribute_before_type_cast(:arguments)))
|
33
|
+
|
34
|
+
# ActiveJob::Base#deserialize expects dates to be strings rather than Time objects.
|
35
|
+
job_data = job_data.to_h { |k, v| [k, %w[enqueued_at scheduled_at].include?(k) ? v&.iso8601 : v] }
|
36
|
+
|
37
|
+
job.tap { job.deserialize(job_data) }
|
38
|
+
end
|
39
|
+
private :to_job
|
40
|
+
|
41
|
+
# Registers callbacks for the given job which destroy the given database record when the job succeeds or is
|
42
|
+
# discarded.
|
43
|
+
#
|
44
|
+
# @param job [ActiveJob::Base]
|
45
|
+
# @param record [ActiveRecord::Base]
|
46
|
+
# @return [ActiveJob::Base]
|
47
|
+
def register_callbacks(job, record)
|
48
|
+
raise 'callbacks already registered' if job.singleton_class.instance_variable_get(:@record)
|
49
|
+
|
50
|
+
# We need to detect three cases:
|
51
|
+
# - If a job succeeds, after_perform will be called.
|
52
|
+
# - If a job fails and should be retried, enqueue will be called. This is handled by the enqueue method.
|
53
|
+
# - If a job exceeds its max attempts, after_discard will be called.
|
54
|
+
job.singleton_class.after_perform { |_j| record.destroy! }
|
55
|
+
job.singleton_class.after_discard { |_j, _exception| record.destroy! }
|
56
|
+
job.singleton_class.instance_variable_set(:@record, record)
|
57
|
+
|
58
|
+
job
|
59
|
+
end
|
60
|
+
private :register_callbacks
|
61
|
+
|
62
|
+
# Enqueue a job for execution at the specified time.
|
63
|
+
#
|
64
|
+
# @param job [ActiveJob::Base] the job to enqueue
|
65
|
+
# @param record_class [Class] the +ActiveRecord+ model class
|
66
|
+
# @param time [Time, NilClass] optional time at which to execute the job
|
67
|
+
# @return [ActiveJob::Base] the enqueued job
|
68
|
+
def enqueue(job, record_class, time = nil)
|
69
|
+
job.scheduled_at = time
|
70
|
+
# Argument serialization is done by ActiveJob. ActiveRecord expects deserialized arguments.
|
71
|
+
serialized = job.serialize.symbolize_keys!.without(:provider_job_id).merge(arguments: job.arguments)
|
7
72
|
|
8
|
-
|
73
|
+
# When a job is enqueued, we must create/update the corresponding database record. We also must ensure callbacks
|
74
|
+
# are registered on the job instance so that when the job is executed, the database record is deleted or updated
|
75
|
+
# (depending on the result).
|
76
|
+
#
|
77
|
+
# We keep track of whether callbacks have been registered by setting the @record instance variable on the job's
|
78
|
+
# singleton class. This holds a reference to the record. This ensures that if execute is called on a record
|
79
|
+
# instance, any updates to the database are reflected on that record instance.
|
80
|
+
if (existing_record = job.singleton_class.instance_variable_get(:@record))
|
81
|
+
# This job instance has already been associated with a database row.
|
82
|
+
if record_class.exists?(job_id: job.job_id)
|
83
|
+
# The database row still exists, we simply need to update it.
|
84
|
+
existing_record.update!(serialized)
|
85
|
+
else
|
86
|
+
# Someone else deleted the database row, we need to recreate and reload the existing record instance. We don't
|
87
|
+
# want to register the new instance because someone might still have a reference to the existing one.
|
88
|
+
record_class.create!(serialized)
|
89
|
+
existing_record.reload
|
90
|
+
end
|
91
|
+
else
|
92
|
+
# This job instance has not been associated with a database row.
|
93
|
+
if (new_record = record_class.find_by(job_id: job.job_id))
|
94
|
+
# The database row already exists. Update it.
|
95
|
+
new_record.update!(serialized)
|
96
|
+
else
|
97
|
+
# The database row does not exist. Create it.
|
98
|
+
new_record = record_class.create!(serialized)
|
99
|
+
end
|
100
|
+
register_callbacks(job, new_record)
|
101
|
+
end
|
102
|
+
job
|
103
|
+
end
|
104
|
+
private :enqueue
|
105
|
+
end
|
106
|
+
end
|
data/lib/marj_adapter.rb
CHANGED
@@ -4,12 +4,19 @@
|
|
4
4
|
#
|
5
5
|
# See https://github.com/nicholasdower/marj
|
6
6
|
class MarjAdapter
|
7
|
+
# Creates a new adapter which will enqueue jobs using the given +ActiveRecord+ model class.
|
8
|
+
#
|
9
|
+
# @param record_class [Class, String] the +ActiveRecord+ model class (or its name) to use to store jobs
|
10
|
+
def initialize(record_class = 'Marj::Record')
|
11
|
+
@record_class = record_class
|
12
|
+
end
|
13
|
+
|
7
14
|
# Enqueue a job for immediate execution.
|
8
15
|
#
|
9
16
|
# @param job [ActiveJob::Base] the job to enqueue
|
10
17
|
# @return [ActiveJob::Base] the enqueued job
|
11
18
|
def enqueue(job)
|
12
|
-
Marj.send(:enqueue, job)
|
19
|
+
Marj.send(:enqueue, job, record_class)
|
13
20
|
end
|
14
21
|
|
15
22
|
# Enqueue a job for execution at the specified time.
|
@@ -18,6 +25,12 @@ class MarjAdapter
|
|
18
25
|
# @param timestamp [Numeric, NilClass] optional number of seconds since Unix epoch at which to execute the job
|
19
26
|
# @return [ActiveJob::Base] the enqueued job
|
20
27
|
def enqueue_at(job, timestamp)
|
21
|
-
Marj.send(:enqueue, job, timestamp ? Time.at(timestamp).utc : nil)
|
28
|
+
Marj.send(:enqueue, job, record_class, timestamp ? Time.at(timestamp).utc : nil)
|
29
|
+
end
|
30
|
+
|
31
|
+
private
|
32
|
+
|
33
|
+
def record_class
|
34
|
+
@record_class = @record_class.is_a?(String) ? @record_class.constantize : @record_class
|
22
35
|
end
|
23
36
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: marj
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version:
|
4
|
+
version: 3.0.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Nick Dower
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2024-01-
|
11
|
+
date: 2024-01-27 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activejob
|
@@ -38,7 +38,7 @@ dependencies:
|
|
38
38
|
- - ">="
|
39
39
|
- !ruby/object:Gem::Version
|
40
40
|
version: '7.1'
|
41
|
-
description: Marj (Minimal ActiveRecord Jobs) is
|
41
|
+
description: Marj (Minimal ActiveRecord Jobs) is a minimal database-backed ActiveJob
|
42
42
|
queueing backend.
|
43
43
|
email: nicholasdower@gmail.com
|
44
44
|
executables: []
|
@@ -48,16 +48,19 @@ files:
|
|
48
48
|
- LICENSE.txt
|
49
49
|
- README.md
|
50
50
|
- lib/marj.rb
|
51
|
+
- lib/marj/jobs.rb
|
52
|
+
- lib/marj/jobs_interface.rb
|
53
|
+
- lib/marj/record.rb
|
54
|
+
- lib/marj/record_interface.rb
|
55
|
+
- lib/marj/relation.rb
|
51
56
|
- lib/marj_adapter.rb
|
52
|
-
- lib/marj_config.rb
|
53
|
-
- lib/marj_record.rb
|
54
57
|
homepage: https://github.com/nicholasdower/marj
|
55
58
|
licenses:
|
56
59
|
- MIT
|
57
60
|
metadata:
|
58
61
|
bug_tracker_uri: https://github.com/nicholasdower/marj/issues
|
59
|
-
changelog_uri: https://github.com/nicholasdower/marj/releases/tag/
|
60
|
-
documentation_uri: https://www.rubydoc.info/github/nicholasdower/marj/
|
62
|
+
changelog_uri: https://github.com/nicholasdower/marj/releases/tag/v3.0.0
|
63
|
+
documentation_uri: https://www.rubydoc.info/github/nicholasdower/marj/v3.0.0
|
61
64
|
homepage_uri: https://github.com/nicholasdower/marj
|
62
65
|
rubygems_mfa_required: 'true'
|
63
66
|
source_code_uri: https://github.com/nicholasdower/marj
|
data/lib/marj_config.rb
DELETED
@@ -1,15 +0,0 @@
|
|
1
|
-
# frozen_string_literal: true
|
2
|
-
|
3
|
-
# Marj configuration.
|
4
|
-
#
|
5
|
-
# See https://github.com/nicholasdower/marj
|
6
|
-
class MarjConfig
|
7
|
-
@table_name = 'jobs'
|
8
|
-
|
9
|
-
class << self
|
10
|
-
# The name of the database table. Defaults to "jobs".
|
11
|
-
#
|
12
|
-
# @return [String]
|
13
|
-
attr_accessor :table_name
|
14
|
-
end
|
15
|
-
end
|
data/lib/marj_record.rb
DELETED
@@ -1,144 +0,0 @@
|
|
1
|
-
# frozen_string_literal: true
|
2
|
-
|
3
|
-
require 'active_job'
|
4
|
-
require 'active_record'
|
5
|
-
require_relative 'marj_config'
|
6
|
-
|
7
|
-
# The Marj ActiveRecord model class.
|
8
|
-
#
|
9
|
-
# See https://github.com/nicholasdower/marj
|
10
|
-
class Marj < ActiveRecord::Base
|
11
|
-
# The Marj version.
|
12
|
-
VERSION = '2.0.1'
|
13
|
-
|
14
|
-
# Executes the job associated with this record and returns the result.
|
15
|
-
def execute
|
16
|
-
# Normally we would call ActiveJob::Base#execute which has the following implementation:
|
17
|
-
# ActiveJob::Callbacks.run_callbacks(:execute) do
|
18
|
-
# job = deserialize(job_data)
|
19
|
-
# job.perform_now
|
20
|
-
# end
|
21
|
-
# However, we need to instantiate the job ourselves in order to register callbacks before execution.
|
22
|
-
ActiveJob::Callbacks.run_callbacks(:execute) do
|
23
|
-
# See register_callbacks for details on how callbacks are used.
|
24
|
-
job = job_class.new.tap { Marj.send(:register_callbacks, _1, self) }
|
25
|
-
|
26
|
-
# ActiveJob::Base#deserialize expects serialized arguments. But the record arguments have already been
|
27
|
-
# deserialized by a custom ActiveRecord serializer (see below). So instead we use the raw arguments string.
|
28
|
-
job_data = attributes.merge('arguments' => JSON.parse(read_attribute_before_type_cast(:arguments)))
|
29
|
-
|
30
|
-
# ActiveJob::Base#deserialize expects dates to be strings rather than Time objects.
|
31
|
-
job_data = job_data.to_h { |k, v| [k, %w[enqueued_at scheduled_at].include?(k) ? v&.iso8601 : v] }
|
32
|
-
|
33
|
-
job.deserialize(job_data)
|
34
|
-
job.perform_now
|
35
|
-
end
|
36
|
-
end
|
37
|
-
|
38
|
-
# Returns an ActiveRecord::Relation scope for enqueued jobs with a +scheduled_at+ that is either +null+ or in the
|
39
|
-
# past. Jobs are ordered by +priority+ (+null+ last), then +scheduled_at+ (+null+ last), then +enqueued_at+.
|
40
|
-
#
|
41
|
-
# @return [ActiveRecord::Relation]
|
42
|
-
def self.ready
|
43
|
-
where('scheduled_at is null or scheduled_at <= ?', Time.now.utc).order(
|
44
|
-
Arel.sql(<<~SQL.squish)
|
45
|
-
CASE WHEN priority IS NULL THEN 1 ELSE 0 END, priority,
|
46
|
-
CASE WHEN scheduled_at IS NULL THEN 1 ELSE 0 END, scheduled_at,
|
47
|
-
enqueued_at
|
48
|
-
SQL
|
49
|
-
)
|
50
|
-
end
|
51
|
-
|
52
|
-
self.table_name = MarjConfig.table_name
|
53
|
-
|
54
|
-
# Order by +enqueued_at+ rather than +job_id+ (the default)
|
55
|
-
self.implicit_order_column = 'enqueued_at'
|
56
|
-
|
57
|
-
# Using a custom serializer for exception_executions so that we can interact with it as a hash rather than a string.
|
58
|
-
serialize(:exception_executions, coder: JSON)
|
59
|
-
|
60
|
-
# Using a custom serializer for arguments so that we can interact with as an array rather than a string.
|
61
|
-
# This enables code like:
|
62
|
-
# Marj.first.arguments.first
|
63
|
-
# Marj.first.update!(arguments: ['foo', 1, Time.now])
|
64
|
-
serialize(:arguments, coder: Class.new do
|
65
|
-
def self.dump(arguments)
|
66
|
-
return ActiveJob::Arguments.serialize(arguments).to_json if arguments.is_a?(Array)
|
67
|
-
return arguments if arguments.is_a?(String) || arguments.nil?
|
68
|
-
|
69
|
-
raise "invalid arguments: #{arguments}"
|
70
|
-
end
|
71
|
-
|
72
|
-
def self.load(arguments)
|
73
|
-
arguments ? ActiveJob::Arguments.deserialize(JSON.parse(arguments)) : nil
|
74
|
-
end
|
75
|
-
end)
|
76
|
-
|
77
|
-
# Using a custom serializer for job_class so that we can interact with it as a class rather than a string.
|
78
|
-
serialize(:job_class, coder: Class.new do
|
79
|
-
def self.dump(clazz)
|
80
|
-
return clazz.name if clazz.is_a?(Class)
|
81
|
-
return clazz if clazz.is_a?(String) || clazz.nil?
|
82
|
-
|
83
|
-
raise "invalid class: #{clazz}"
|
84
|
-
end
|
85
|
-
|
86
|
-
def self.load(str)
|
87
|
-
str&.constantize
|
88
|
-
end
|
89
|
-
end)
|
90
|
-
|
91
|
-
# Registers job callbacks used to keep the database record for the specified job in sync.
|
92
|
-
#
|
93
|
-
# @param job [ActiveJob::Base]
|
94
|
-
# @return [ActiveJob::Base]
|
95
|
-
def self.register_callbacks(job, record)
|
96
|
-
raise 'callbacks already registered' if job.singleton_class.instance_variable_get(:@__marj)
|
97
|
-
|
98
|
-
# We need to detect three cases:
|
99
|
-
# - If a job succeeds, after_perform will be called.
|
100
|
-
# - If a job fails and should be retried, enqueue will be called. This is handled by the enqueue method.
|
101
|
-
# - If a job exceeds its max attempts, after_discard will be called.
|
102
|
-
job.singleton_class.after_perform { |_j| record.destroy! }
|
103
|
-
job.singleton_class.after_discard { |_j, _exception| record.destroy! }
|
104
|
-
job.singleton_class.instance_variable_set(:@__marj, record)
|
105
|
-
|
106
|
-
job
|
107
|
-
end
|
108
|
-
private_class_method :register_callbacks
|
109
|
-
|
110
|
-
# Enqueue a job for execution at the specified time.
|
111
|
-
#
|
112
|
-
# @param job [ActiveJob::Base] the job to enqueue
|
113
|
-
# @param time [Time, NilClass] optional time at which to execute the job
|
114
|
-
# @return [ActiveJob::Base] the enqueued job
|
115
|
-
def self.enqueue(job, time = nil)
|
116
|
-
job.scheduled_at = time
|
117
|
-
# Argument serialization is done by ActiveJob. ActiveRecord expects deserialized arguments.
|
118
|
-
serialized = job.serialize.symbolize_keys!.without(:provider_job_id).merge(arguments: job.arguments)
|
119
|
-
|
120
|
-
# When a job is enqueued, we must create/update the corresponding database record. We also must ensure callbacks are
|
121
|
-
# registered on the job instance so that when the job is executed, the database record is deleted or updated
|
122
|
-
# (depending on the result).
|
123
|
-
#
|
124
|
-
# There are three cases:
|
125
|
-
# - The first time a job is enqueued, we need to create the record and register callbacks.
|
126
|
-
# - If a previously enqueued job instance is re-enqueued, for instance after execution fails, callbacks have
|
127
|
-
# already been registered. In this case we only need to update the record.
|
128
|
-
# - It is also possible for new job instance to be created for a job that is already in the database. In this case
|
129
|
-
# we need to update the record and register callbacks.
|
130
|
-
#
|
131
|
-
# We keep track of whether callbacks have been registered by setting the @__marj instance variable on the job's
|
132
|
-
# singleton class. This holds a reference to the record. This allows us to update the record without re-fetching it
|
133
|
-
# and also ensures that if execute is called on a record any updates to the database are reflected on that record
|
134
|
-
# instance.
|
135
|
-
if (record = job.singleton_class.instance_variable_get(:@__marj))
|
136
|
-
record.update!(serialized)
|
137
|
-
else
|
138
|
-
record = Marj.find_or_create_by!(job_id: job.job_id) { _1.assign_attributes(serialized) }
|
139
|
-
register_callbacks(job, record)
|
140
|
-
end
|
141
|
-
job
|
142
|
-
end
|
143
|
-
private_class_method :enqueue
|
144
|
-
end
|