acidic_job 0.7.0 → 0.7.4
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.github/workflows/main.yml +1 -1
- data/Gemfile.lock +2 -4
- data/README.md +8 -6
- data/lib/acidic_job/deliver_transactionally_extension.rb +7 -5
- data/lib/acidic_job/errors.rb +6 -0
- data/lib/acidic_job/perform_wrapper.rb +0 -2
- data/lib/acidic_job/version.rb +1 -1
- data/lib/acidic_job.rb +9 -11
- metadata +3 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 818fce6f9fefae6d2c56930eedcc683f0332f72acbc0ec9b181b1b0855addddb
|
4
|
+
data.tar.gz: 162262b51ac591fee5910b59cda3ebaeb045580e3390f889a37be35212a2a5aa
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: be6dfa507ce2e6cb2e4bb00ae0b57fe7e227d9347ef4a64204c79c52c5b57ec66ec347e37113066b0c4583ddb9c1045fddf3fbf3eacd07942afc528451fa4cd7
|
7
|
+
data.tar.gz: 2c28797e665ce91181a11d20c5cf1183c2cc585cbe5602c5b8dcc870f0403d49c59232d9662b8fcc96926a0082baeea3251bbcb9ac086d6085fffd8d1ad74685
|
data/.github/workflows/main.yml
CHANGED
data/Gemfile.lock
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
PATH
|
2
2
|
remote: .
|
3
3
|
specs:
|
4
|
-
acidic_job (0.7.
|
4
|
+
acidic_job (0.7.4)
|
5
5
|
activerecord (>= 4.0.0)
|
6
6
|
activesupport
|
7
7
|
|
@@ -62,8 +62,6 @@ GEM
|
|
62
62
|
nokogiri (1.12.3)
|
63
63
|
mini_portile2 (~> 2.6.1)
|
64
64
|
racc (~> 1.4)
|
65
|
-
nokogiri (1.12.3-x86_64-darwin)
|
66
|
-
racc (~> 1.4)
|
67
65
|
parallel (1.20.1)
|
68
66
|
parser (3.0.1.1)
|
69
67
|
ast (~> 2.4.1)
|
@@ -144,4 +142,4 @@ DEPENDENCIES
|
|
144
142
|
sqlite3
|
145
143
|
|
146
144
|
BUNDLED WITH
|
147
|
-
2.2.
|
145
|
+
2.2.31
|
data/README.md
CHANGED
@@ -57,14 +57,14 @@ It provides a suite of functionality that empowers you to create complex, robust
|
|
57
57
|
|
58
58
|
### Transactional Steps
|
59
59
|
|
60
|
-
The first and foundational feature `acidic_job` provides is the `
|
60
|
+
The first and foundational feature `acidic_job` provides is the `with_acidity` method, which takes a block of transactional step methods (defined via the `step`) method:
|
61
61
|
|
62
62
|
```ruby
|
63
63
|
class RideCreateJob < ActiveJob::Base
|
64
64
|
include AcidicJob
|
65
65
|
|
66
66
|
def perform(ride_params)
|
67
|
-
|
67
|
+
with_acidity given: { user: current_user, params: ride_params, ride: nil } do
|
68
68
|
step :create_ride_and_audit_record
|
69
69
|
step :create_stripe_charge
|
70
70
|
step :send_receipt
|
@@ -85,20 +85,20 @@ class RideCreateJob < ActiveJob::Base
|
|
85
85
|
end
|
86
86
|
```
|
87
87
|
|
88
|
-
`
|
88
|
+
`with_acidity` takes only the `given:` named parameter and a block where you define the steps of this operation. `step` simply takes the name of a method available in the job. That's all!
|
89
89
|
|
90
90
|
Now, each execution of this job will find or create an `AcidicJob::Key` record, which we leverage to wrap every step in a database transaction. Moreover, this database record allows `acidic_job` to ensure that if your job fails on step 3, when it retries, it will simply jump right back to trying to execute the method defined for the 3rd step, and won't even execute the first two step methods. This means your step methods only need to be idempotent on failure, not on success, since they will never be run again if they succeed.
|
91
91
|
|
92
92
|
### Persisted Attributes
|
93
93
|
|
94
|
-
Any objects passed to the `
|
94
|
+
Any objects passed to the `given` option on the `with_acidity` method are not just made available to each of your step methods, they are made available across retries. This means that you can set an attribute in step 1, access it in step 2, have step 2 fail, have the job retry, jump directly back to step 2 on retry, and have that object still accessible. This is done by serializing all objects to a field on the `AcidicJob::Key` and manually providing getters and setters that sync with the database record.
|
95
95
|
|
96
96
|
```ruby
|
97
97
|
class RideCreateJob < ActiveJob::Base
|
98
98
|
include AcidicJob
|
99
99
|
|
100
100
|
def perform(ride_params)
|
101
|
-
|
101
|
+
with_acidity given: { ride: nil } do
|
102
102
|
step :create_ride_and_audit_record
|
103
103
|
step :create_stripe_charge
|
104
104
|
step :send_receipt
|
@@ -132,7 +132,7 @@ class RideCreateJob < ActiveJob::Base
|
|
132
132
|
include AcidicJob
|
133
133
|
|
134
134
|
def perform(ride_params)
|
135
|
-
|
135
|
+
with_acidity given: { user: current_user, params: ride_params, ride: nil } do
|
136
136
|
step :create_ride_and_audit_record
|
137
137
|
step :create_stripe_charge
|
138
138
|
step :send_receipt
|
@@ -157,6 +157,8 @@ This allows `acidic_job` to use an `after_perform` callback to delete the `Acidi
|
|
157
157
|
|
158
158
|
One final feature for those of you using Sidekiq Pro: an integrated DSL for Sidekiq Batches. By simply adding the `awaits` option to your step declarations, you can attach any number of additional, asynchronous workers to your step. This is profoundly powerful, as it means that you can define a workflow where step 2 is started _if and only if_ step 1 succeeds, but step 1 can have 3 different workers enqueued on 3 different queues, each running in parallel. Once all 3 workers succeed, `acidic_job` will move on to step 2. That's right, by leveraging the power of Sidekiq Batches, you can have workers that are executed in parallel, on separate queues, and asynchronously, but are still blocking—as a group—the next step in your workflow! This unlocks incredible power and flexibility for defining and structuring complex workflows and operations, and in my mind is the number one selling point for Sidekiq Pro.
|
159
159
|
|
160
|
+
In my opinion, any commercial software using Sidekiq should get Sidekiq Pro; it is _absolutely_ worth the money. If, however, you are using `acidic_job` in a non-commercial application, you could use the open-source dropin replacement for this functionality: https://github.com/breamware/sidekiq-batch
|
161
|
+
|
160
162
|
## Development
|
161
163
|
|
162
164
|
After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
|
@@ -2,7 +2,8 @@
|
|
2
2
|
|
3
3
|
module AcidicJob
|
4
4
|
module DeliverTransactionallyExtension
|
5
|
-
|
5
|
+
# rubocop:disable Metrics/MethodLength
|
6
|
+
def deliver_transactionally(_options = {})
|
6
7
|
job = delivery_job_class
|
7
8
|
|
8
9
|
attributes = {
|
@@ -11,14 +12,15 @@ module AcidicJob
|
|
11
12
|
}
|
12
13
|
|
13
14
|
job_args = if job <= ActionMailer::Parameterized::MailDeliveryJob
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
|
15
|
+
[@mailer_class.name, @action.to_s, "deliver_now", { params: @params, args: @args }]
|
16
|
+
else
|
17
|
+
[@mailer_class.name, @action.to_s, "deliver_now", @params, *@args]
|
18
|
+
end
|
18
19
|
|
19
20
|
attributes[:job_args] = job.new(job_args).serialize
|
20
21
|
|
21
22
|
AcidicJob::Staged.create!(attributes)
|
22
23
|
end
|
24
|
+
# rubocop:enable Metrics/MethodLength
|
23
25
|
end
|
24
26
|
end
|
data/lib/acidic_job/errors.rb
CHANGED
@@ -16,4 +16,10 @@ module AcidicJob
|
|
16
16
|
class UnknownJobAdapter < Error; end
|
17
17
|
|
18
18
|
class NoDefinedSteps < Error; end
|
19
|
+
|
20
|
+
class SidekiqBatchRequired < Error; end
|
21
|
+
|
22
|
+
class TooManyParametersForStepMethod < Error; end
|
23
|
+
|
24
|
+
class TooManyParametersForParallelJob < Error; end
|
19
25
|
end
|
@@ -18,7 +18,6 @@ module AcidicJob
|
|
18
18
|
|
19
19
|
private
|
20
20
|
|
21
|
-
# rubocop:disable Metrics/AbcSize
|
22
21
|
def set_arguments_for_perform(*args, **kwargs)
|
23
22
|
# store arguments passed into `perform` so that we can later persist
|
24
23
|
# them to `AcidicJob::Key#job_args` for both ActiveJob and Sidekiq::Worker
|
@@ -32,6 +31,5 @@ module AcidicJob
|
|
32
31
|
[]
|
33
32
|
end
|
34
33
|
end
|
35
|
-
# rubocop:enable Metrics/AbcSize
|
36
34
|
end
|
37
35
|
end
|
data/lib/acidic_job/version.rb
CHANGED
data/lib/acidic_job.rb
CHANGED
@@ -9,6 +9,7 @@ require_relative "acidic_job/key"
|
|
9
9
|
require_relative "acidic_job/staged"
|
10
10
|
require_relative "acidic_job/perform_wrapper"
|
11
11
|
require_relative "acidic_job/perform_transactionally_extension"
|
12
|
+
require_relative "acidic_job/deliver_transactionally_extension"
|
12
13
|
require_relative "acidic_job/sidekiq_callbacks"
|
13
14
|
require "active_support/concern"
|
14
15
|
|
@@ -24,9 +25,7 @@ module AcidicJob
|
|
24
25
|
# Extend ActiveJob with `perform_transactionally` class method
|
25
26
|
klass.include PerformTransactionallyExtension
|
26
27
|
|
27
|
-
if defined?(ActionMailer)
|
28
|
-
ActionMailer::Parameterized::MessageDelivery.include DeliverTransactionallyExtension
|
29
|
-
end
|
28
|
+
ActionMailer::Parameterized::MessageDelivery.include DeliverTransactionallyExtension if defined?(ActionMailer)
|
30
29
|
|
31
30
|
# Ensure our `perform` method always runs first to gather parameters
|
32
31
|
klass.prepend PerformWrapper
|
@@ -62,7 +61,7 @@ module AcidicJob
|
|
62
61
|
IDEMPOTENCY_KEY_LOCK_TIMEOUT = 90
|
63
62
|
|
64
63
|
# takes a block
|
65
|
-
def
|
64
|
+
def with_acidity(given:)
|
66
65
|
# execute the block to gather the info on what steps are defined for this job workflow
|
67
66
|
steps = yield || []
|
68
67
|
|
@@ -77,7 +76,7 @@ module AcidicJob
|
|
77
76
|
# close proximity, one of the two will be aborted by Postgres because we're
|
78
77
|
# using a transaction with SERIALIZABLE isolation level. It may not look
|
79
78
|
# it, but this code is safe from races.
|
80
|
-
key = ensure_idempotency_key_record(idempotency_key_value, workflow,
|
79
|
+
key = ensure_idempotency_key_record(idempotency_key_value, workflow, given)
|
81
80
|
|
82
81
|
# begin the workflow
|
83
82
|
process_key(key)
|
@@ -94,7 +93,7 @@ module AcidicJob
|
|
94
93
|
recovery_point = @key.recovery_point.to_s
|
95
94
|
current_step = @key.workflow[recovery_point]
|
96
95
|
|
97
|
-
if recovery_point == Key::RECOVERY_POINT_FINISHED.to_s
|
96
|
+
if recovery_point == Key::RECOVERY_POINT_FINISHED.to_s # rubocop:disable Style/GuardClause
|
98
97
|
break
|
99
98
|
elsif current_step.nil?
|
100
99
|
raise UnknownRecoveryPoint, "Defined workflow does not reference this step: #{recovery_point}"
|
@@ -181,7 +180,7 @@ module AcidicJob
|
|
181
180
|
raise LockedIdempotencyKey if key.locked_at && key.locked_at > Time.current - IDEMPOTENCY_KEY_LOCK_TIMEOUT
|
182
181
|
|
183
182
|
# Lock the key and update latest run unless the job is already finished.
|
184
|
-
key.update!(last_run_at: Time.current, locked_at: Time.current) unless key.finished?
|
183
|
+
key.update!(last_run_at: Time.current, locked_at: Time.current, workflow: workflow) unless key.finished?
|
185
184
|
else
|
186
185
|
key = Key.create!(
|
187
186
|
idempotency_key: key_val,
|
@@ -277,8 +276,7 @@ module AcidicJob
|
|
277
276
|
elsif callable.arity == 1
|
278
277
|
callable.call(key)
|
279
278
|
else
|
280
|
-
|
281
|
-
raise
|
279
|
+
raise TooManyParametersForStepMethod
|
282
280
|
end
|
283
281
|
|
284
282
|
if result.is_a?(Response)
|
@@ -295,7 +293,7 @@ module AcidicJob
|
|
295
293
|
def enqueue_step_parallel_jobs(jobs)
|
296
294
|
# TODO: GIVE PROPER ERROR
|
297
295
|
# `batch` is available from Sidekiq::Pro
|
298
|
-
raise unless defined?(Sidekiq::Batch)
|
296
|
+
raise SidekiqBatchRequired unless defined?(Sidekiq::Batch)
|
299
297
|
|
300
298
|
batch.jobs do
|
301
299
|
step_batch = Sidekiq::Batch.new
|
@@ -317,7 +315,7 @@ module AcidicJob
|
|
317
315
|
elsif worker.instance_method(:perform).arity == 1
|
318
316
|
worker.perform_async(key.id)
|
319
317
|
else
|
320
|
-
raise
|
318
|
+
raise TooManyParametersForParallelJob
|
321
319
|
end
|
322
320
|
end
|
323
321
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: acidic_job
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.7.
|
4
|
+
version: 0.7.4
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- fractaledmind
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2021-
|
11
|
+
date: 2021-11-19 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activerecord
|
@@ -109,7 +109,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
109
109
|
- !ruby/object:Gem::Version
|
110
110
|
version: '0'
|
111
111
|
requirements: []
|
112
|
-
rubygems_version: 3.
|
112
|
+
rubygems_version: 3.2.22
|
113
113
|
signing_key:
|
114
114
|
specification_version: 4
|
115
115
|
summary: Idempotent operations for Rails apps, built on top of ActiveJob.
|