acidic_job 1.0.0.pre19 → 1.0.0.pre20
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/Gemfile.lock +1 -1
- data/README.md +66 -7
- data/lib/acidic_job/version.rb +1 -1
- data/lib/acidic_job.rb +3 -2
- metadata +1 -1
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: e608eefef4baffbfb2c95dcbef942a53c9ca242c4c593ddbbe0d677ef7d318da
|
4
|
+
data.tar.gz: 65bed32739c1a1a860102e59dac69598c4357512034d8d3e87ed772469b9bac0
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 5a5f6155e20f929c631392f00480f57928455778e13117609457be9b8624e7f16c41661df96ae34a4045582f957b044cbdf811bfa7e187dd627b8b72b80edb90
|
7
|
+
data.tar.gz: 6a663d288bd1ef3ba5cfac8e137fd496ebbca76e08573e50d706ed2570c98b6fe3148b8c9335b55313a091ccd0cc226e3b9023878dd418d400af33fd0280c1b4
|
data/Gemfile.lock
CHANGED
data/README.md
CHANGED
@@ -68,8 +68,10 @@ It provides a suite of functionality that empowers you to create complex, robust
|
|
68
68
|
* Persisted Attributes — when retrying jobs at later steps, we need to ensure that data created in previous steps is still available to later steps on retry.
|
69
69
|
* Transactionally Staged Jobs — enqueue additional jobs within the acidic transaction safely
|
70
70
|
* Custom Idempotency Keys — use something other than the job ID for the idempotency key of the job run
|
71
|
+
* Iterable Steps — define steps that iterate over some collection fully until moving on to the next step
|
71
72
|
* Sidekiq Callbacks — bring ActiveJob-like callbacks into your pure Sidekiq Workers
|
72
73
|
* Sidekiq Batches — leverage the power of Sidekiq Pro's `batch` functionality without the hassle
|
74
|
+
* Run Finished Callbacks — set callbacks for when a job run finishes fully
|
73
75
|
|
74
76
|
### Transactional Steps
|
75
77
|
|
@@ -80,9 +82,10 @@ class RideCreateJob < ActiveJob::Base
|
|
80
82
|
include AcidicJob
|
81
83
|
|
82
84
|
def perform(user_id, ride_params)
|
83
|
-
user = User.find(user_id)
|
85
|
+
@user = User.find(user_id)
|
86
|
+
@params = ride_params
|
84
87
|
|
85
|
-
with_acidity providing: {
|
88
|
+
with_acidity providing: { ride: nil } do
|
86
89
|
step :create_ride_and_audit_record
|
87
90
|
step :create_stripe_charge
|
88
91
|
step :send_receipt
|
@@ -128,7 +131,7 @@ class RideCreateJob < ActiveJob::Base
|
|
128
131
|
end
|
129
132
|
|
130
133
|
def create_stripe_charge
|
131
|
-
Stripe::Charge.create(amount: 20_00, customer:
|
134
|
+
Stripe::Charge.create(amount: 20_00, customer: @ride.user)
|
132
135
|
end
|
133
136
|
|
134
137
|
# ...
|
@@ -139,6 +142,8 @@ end
|
|
139
142
|
|
140
143
|
**Note:** You will note the use of `self.ride = ...` in the code sample above. In order to call the attribute setter method that will sync with the database record, you _must_ use this style. `@ride = ...` and/or `ride = ...` will both fail to sync the value with the database record.
|
141
144
|
|
145
|
+
The default pattern you should follow when defining your `perform` method is to make any values that your `step` methods need access to, but are present at the start of the `perform` method simply instance variables. You only need to `provide` attributes that will be set _during a step_. This means, the initial value will almost always be `nil`.
|
146
|
+
|
142
147
|
### Transactionally Staged Jobs
|
143
148
|
|
144
149
|
A standard problem when inside of database transactions is enqueuing other jobs. On the one hand, you could enqueue a job inside of a transaction that then rollbacks, which would leave that job to fail and retry and fail. On the other hand, you could enqueue a job that is picked up before the transaction commits, which would mean the records are not yet available to this job.
|
@@ -150,9 +155,10 @@ class RideCreateJob < ActiveJob::Base
|
|
150
155
|
include AcidicJob
|
151
156
|
|
152
157
|
def perform(user_id, ride_params)
|
153
|
-
user = User.find(user_id)
|
158
|
+
@user = User.find(user_id)
|
159
|
+
@params = ride_params
|
154
160
|
|
155
|
-
with_acidity providing: {
|
161
|
+
with_acidity providing: { ride: nil } do
|
156
162
|
step :create_ride_and_audit_record
|
157
163
|
step :create_stripe_charge
|
158
164
|
step :send_receipt
|
@@ -211,6 +217,28 @@ end
|
|
211
217
|
|
212
218
|
> **Note:** The signature of the `acidic_by` proc _needs to match the signature_ of the job's `perform` method.
|
213
219
|
|
220
|
+
### Iterable Steps
|
221
|
+
|
222
|
+
Sometimes our workflows have steps that need to iterate over a collection and perform an action for each item in the collection before moving on to the next step in the workflow. In these cases, we can use the `for_each` option when defining our step to specific the collection, and `acidic_job` will pass each item into your step method for processing, keeping the same transactional guarantees as for any step. This means that if your step encounters an error in processing any item in the collection, when your job is retried, the job will jump right back to that step and right back to that item in the collection to try again.
|
223
|
+
|
224
|
+
```ruby
|
225
|
+
class ExampleJob < ActiveJob::Base
|
226
|
+
include AcidicJob
|
227
|
+
|
228
|
+
def perform(record:)
|
229
|
+
with_acidity providing: { collection: [1, 2, 3, 4, 5] } do
|
230
|
+
step :process_item, for_each: :collection
|
231
|
+
step :next_step
|
232
|
+
end
|
233
|
+
end
|
234
|
+
|
235
|
+
def process_item(item)
|
236
|
+
# do whatever work needs to be done with this individual item
|
237
|
+
end
|
238
|
+
end
|
239
|
+
```
|
240
|
+
|
241
|
+
**Note:** The same restrictions apply here as for any persisted attribute — you can only use objects that can be serialized by ActiveRecord.
|
214
242
|
|
215
243
|
### Sidekiq Callbacks
|
216
244
|
|
@@ -229,9 +257,10 @@ class RideCreateJob < ActiveJob::Base
|
|
229
257
|
include AcidicJob
|
230
258
|
|
231
259
|
def perform(user_id, ride_params)
|
232
|
-
user = User.find(user_id)
|
260
|
+
@user = User.find(user_id)
|
261
|
+
@params = ride_params
|
233
262
|
|
234
|
-
with_acidity providing: {
|
263
|
+
with_acidity providing: { ride: nil } do
|
235
264
|
step :create_ride_and_audit_record, awaits: [SomeJob]
|
236
265
|
step :create_stripe_charge, args: [1, 2, 3], kwargs: { some: 'thing' }
|
237
266
|
step :send_receipt
|
@@ -240,6 +269,36 @@ class RideCreateJob < ActiveJob::Base
|
|
240
269
|
end
|
241
270
|
```
|
242
271
|
|
272
|
+
### Run Finished Callbacks
|
273
|
+
|
274
|
+
When working with workflow jobs that make use of the `awaits` feature for a step, it is important to remember that the `after_perform` callback will be called _as soon as the first `awaits` step has enqueued job_, and **not** when the entire job run has finished. `acidic_job` allows the `perform` method to finish so that the queue for the workflow job is cleared to pick up new work while the `awaits` jobs are running. `acidic_job` will automatically re-enqueue the workflow job and progress to the next step when all of the `awaits` jobs have successfully finished. However, this means that `after_perform` **is not necessarily** the same as `after_finish`. In order to provide the opportunity for you to execute callback logic _if and only if_ a job run has finished, we provide callback hooks for the `finish` event.
|
275
|
+
|
276
|
+
For example, you could use this hook to immediately clean up the `AcidicJob::Run` database record whenever the workflow job finishes successfully like so:
|
277
|
+
|
278
|
+
```ruby
|
279
|
+
class RideCreateJob < ActiveJob::Base
|
280
|
+
include AcidicJob
|
281
|
+
set_callback :finish, :after, :delete_run_record
|
282
|
+
|
283
|
+
def perform(user_id, ride_params)
|
284
|
+
@user = User.find(user_id)
|
285
|
+
@params = ride_params
|
286
|
+
|
287
|
+
with_acidity providing: { ride: nil } do
|
288
|
+
step :create_ride_and_audit_record, awaits: [SomeJob]
|
289
|
+
step :create_stripe_charge, args: [1, 2, 3], kwargs: { some: 'thing' }
|
290
|
+
step :send_receipt
|
291
|
+
end
|
292
|
+
end
|
293
|
+
|
294
|
+
def delete_run_record
|
295
|
+
return unless acidic_job_run.succeeded?
|
296
|
+
|
297
|
+
acidic_job_run.destroy!
|
298
|
+
end
|
299
|
+
end
|
300
|
+
```
|
301
|
+
|
243
302
|
## Testing
|
244
303
|
|
245
304
|
When testing acidic jobs, you are likely to run into `ActiveRecord::TransactionIsolationError`s:
|
data/lib/acidic_job/version.rb
CHANGED
data/lib/acidic_job.rb
CHANGED
@@ -55,6 +55,7 @@ module AcidicJob
|
|
55
55
|
klass.define_singleton_method(:acidic_by_job_id) { @acidic_identifier = :job_id }
|
56
56
|
klass.define_singleton_method(:acidic_by_job_args) { @acidic_identifier = :job_args }
|
57
57
|
klass.define_singleton_method(:acidic_by) { |proc| @acidic_identifier = proc }
|
58
|
+
klass.attr_reader(:acidic_job_run)
|
58
59
|
end
|
59
60
|
|
60
61
|
included do
|
@@ -96,10 +97,10 @@ module AcidicJob
|
|
96
97
|
# convert the array of steps into a hash of recovery_points and next steps
|
97
98
|
workflow = define_workflow(@__acidic_job_steps)
|
98
99
|
|
99
|
-
@
|
100
|
+
@acidic_job_run = ensure_run_record(workflow, providing)
|
100
101
|
|
101
102
|
# begin the workflow
|
102
|
-
process_run(@
|
103
|
+
process_run(@acidic_job_run)
|
103
104
|
end
|
104
105
|
|
105
106
|
# DEPRECATED
|