acidic_job 1.0.0.rc3 → 1.0.0.rc5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +6 -6
- data/app/models/acidic_job/entry.rb +4 -2
- data/app/models/acidic_job/execution.rb +25 -12
- data/app/models/acidic_job/value.rb +2 -0
- data/lib/acidic_job/builder.rb +2 -2
- data/lib/acidic_job/context.rb +36 -20
- data/lib/acidic_job/errors.rb +2 -3
- data/lib/acidic_job/log_subscriber.rb +4 -6
- data/lib/acidic_job/plugin_context.rb +26 -3
- data/lib/acidic_job/plugins/transactional_step.rb +4 -4
- data/lib/acidic_job/serializer.rb +28 -0
- data/lib/acidic_job/serializers/job_serializer.rb +1 -1
- data/lib/acidic_job/serializers/range_serializer.rb +1 -3
- data/lib/acidic_job/testing.rb +10 -12
- data/lib/acidic_job/version.rb +1 -1
- data/lib/acidic_job/workflow.rb +44 -32
- data/lib/acidic_job.rb +3 -0
- data/lib/generators/acidic_job/install_generator.rb +5 -5
- data/lib/generators/acidic_job/templates/create_acidic_job_tables_migration.rb.erb +6 -6
- metadata +7 -55
- data/.codacy.yml +0 -4
- data/.github/FUNDING.yml +0 -13
- data/.github/workflows/main.yml +0 -38
- data/.gitignore +0 -18
- data/.rubocop.yml +0 -83
- data/.ruby-version +0 -1
- data/Gemfile +0 -5
- data/Gemfile.lock +0 -201
- data/Rakefile +0 -16
- data/TODO +0 -77
- data/UPGRADE_GUIDE.md +0 -81
- data/acidic_job.gemspec +0 -52
- data/bin/console +0 -21
- data/bin/setup +0 -8
- data/bin/test_all +0 -26
- data/blog_post.md +0 -28
- data/combustion/log/test.log +0 -0
- data/gemfiles/rails_7.0.gemfile +0 -11
- data/gemfiles/rails_7.1.gemfile +0 -11
- data/gemfiles/rails_7.2.gemfile +0 -11
- data/gemfiles/rails_8.0.gemfile +0 -11
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: af6f1d0a08680695a243f3b045eb9eedd0bd7d6568e14acf8caefeeca4dc45f6
|
4
|
+
data.tar.gz: 533f91d24f7a4cfc5cbd9b74ef55f7207abd1ae00984fec545ad5590ca0b73b6
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 12a444e4a971df69a1f6f15c64ce0d3609104fe0e8a42ffc4ece9a6d948a5f799981e8ee087fcac45befea46b4f2d4f1279ef39b625cdc1a4db6618e04a98098
|
7
|
+
data.tar.gz: 05b26e35b45823c32063d41e6ee82b9ad49239de0a412c4c38328ad95391cad60e1e404c04638d0811d27c8073666a5f42cebc36912ac5c44a64d84ab46ad859
|
data/README.md
CHANGED
@@ -24,7 +24,7 @@ With AcidicJob, you can write reliable and repeatable multi-step distributed ope
|
|
24
24
|
Install the gem and add to the application's Gemfile by executing:
|
25
25
|
|
26
26
|
```sh
|
27
|
-
bundle add acidic_job --version "1.0.0.
|
27
|
+
bundle add acidic_job --version "1.0.0.rc3"
|
28
28
|
```
|
29
29
|
|
30
30
|
If `bundler` is not being used to manage dependencies, install the gem by executing:
|
@@ -109,7 +109,7 @@ The block passed to `execute_workflow` is where you define the steps of the work
|
|
109
109
|
The `step` method is the only method available on the yielded workflow builder object, and it simply takes the name of a method available in the job.
|
110
110
|
|
111
111
|
> [!IMPORTANT]
|
112
|
-
> In order to craft resilient workflows, you need to ensure that each step method wraps a single unit of IO-bound work. You **
|
112
|
+
> In order to craft resilient workflows, you need to ensure that each step method wraps a single unit of IO-bound work. You **should not** have a step method that performs multiple IO-bound operations, like writing to your database and calling an external API. Steps should be as granular and self-contained as possible. This allows your own logic to be more durable in case of failures in third-party APIs, network errors, and so on. So, the rule of thumb is to have only one _state mutation_ per step. And this rule of thumb graduates to a hard and fast rule for _foreign state mutations_. You **must** only have **one** foreign state mutation per step, where a foreign state mutation is any operation that writes to a system beyond your own boundaries. This might be creating a charge on Stripe, adding a DNS record, or sending an email.[^1]
|
113
113
|
|
114
114
|
[^1]: I first learned this rule from [Brandur Leach](https://twitter.com/brandur) reminds in his post on [Implementing Stripe-like Idempotency Keys in Postgres](https://brandur.org/idempotency-keys).
|
115
115
|
|
@@ -188,7 +188,7 @@ class Job < ActiveJob::Base
|
|
188
188
|
|
189
189
|
### Orchestrating steps
|
190
190
|
|
191
|
-
In addition to the workflow definition setup, `AcidicJob` also provides a couple of methods to precisely control the workflow step execution. From within any step method, you can call either `repeat_step!` or `
|
191
|
+
In addition to the workflow definition setup, `AcidicJob` also provides a couple of methods to precisely control the workflow step execution. From within any step method, you can call either `repeat_step!` or `halt_workflow!`.
|
192
192
|
|
193
193
|
`repeat_step!` will cause the current step to be re-executed on the next iteration of the workflow. This is useful when you need to traverse a collection of items and perform the same operation on each item. For example, if you need to send an email to each user in a collection, you could do something like this:
|
194
194
|
|
@@ -218,7 +218,7 @@ end
|
|
218
218
|
|
219
219
|
This example demonstrates how you can leverage the basic building blocks provided by `AcidicJob` to orchestrate complex workflows. In this case, the `notify_users` step sends an email to each user in the collection, one at a time, and resiliently handles errors by storing a cursor in the `ctx` object to keep track of the current user being processed. If any error occurs while traversing the `@users` collection, the job will be retried, and the `notify_users` step will be re-executed from the last successful cursor position.
|
220
220
|
|
221
|
-
The `
|
221
|
+
The `halt_workflow!` method, on the other hand, stops not just the execution of the current step but the job as a whole. This is useful when you either need to conditionally stop the workflow based on some criteria or need to delay the job for some amount of time before being restarted. For example, if you need to send a follow-up email to a user 14 days after they sign up, you could do something like this:
|
222
222
|
|
223
223
|
```ruby
|
224
224
|
class Job < ActiveJob::Base
|
@@ -240,7 +240,7 @@ class Job < ActiveJob::Base
|
|
240
240
|
def send_welcome_email
|
241
241
|
if ctx[:halt]
|
242
242
|
ctx[:halt] = false
|
243
|
-
|
243
|
+
halt_workflow!
|
244
244
|
end
|
245
245
|
UserMailer.with(user: @user).welcome_email.deliver_later
|
246
246
|
end
|
@@ -252,7 +252,7 @@ In this example, the `delay` step creates a new instance of the job and enqueues
|
|
252
252
|
|
253
253
|
### Overview
|
254
254
|
|
255
|
-
`AcidicJob` is a library that provides a small yet powerful set of tools to build cohesive and resilient workflows in your Active Jobs. All of the tools are made available by `include`ing the `AcidicJob::Workflow` module. The primary and most important tool is the `execute_workflow` method, which you call within your `perform` method. Then, if you need to store any contextual data, you use the `ctx` objects setters and getters. Finally, within any step methods, you can call `repeat_step!` or `
|
255
|
+
`AcidicJob` is a library that provides a small yet powerful set of tools to build cohesive and resilient workflows in your Active Jobs. All of the tools are made available by `include`ing the `AcidicJob::Workflow` module. The primary and most important tool is the `execute_workflow` method, which you call within your `perform` method. Then, if you need to store any contextual data, you use the `ctx` objects setters and getters. Finally, within any step methods, you can call `repeat_step!` or `halt_workflow!` to control the execution of the workflow. If you need, you can also access the `execution` Active Record object to get information about the current execution of the workflow. With these lightweight tools, you can build complex workflows that are resilient to failures and can handle a wide range of use cases.
|
256
256
|
|
257
257
|
|
258
258
|
## Testing
|
@@ -4,8 +4,10 @@ module AcidicJob
|
|
4
4
|
class Entry < Record
|
5
5
|
belongs_to :execution, class_name: "AcidicJob::Execution"
|
6
6
|
|
7
|
-
|
8
|
-
|
7
|
+
serialize :data, coder: AcidicJob::Serializer
|
8
|
+
|
9
|
+
scope :for_step, -> (step) { where(step: step) }
|
10
|
+
scope :for_action, -> (action) { where(action: action) }
|
9
11
|
scope :ordered, -> { order(timestamp: :asc) }
|
10
12
|
|
11
13
|
def self.most_recent
|
@@ -2,25 +2,40 @@
|
|
2
2
|
|
3
3
|
module AcidicJob
|
4
4
|
class Execution < Record
|
5
|
-
has_many :entries, class_name: "AcidicJob::Entry"
|
6
|
-
has_many :values, class_name: "AcidicJob::Value"
|
5
|
+
has_many :entries, class_name: "AcidicJob::Entry", dependent: :destroy
|
6
|
+
has_many :values, class_name: "AcidicJob::Value", dependent: :destroy
|
7
|
+
|
8
|
+
serialize :definition, coder: AcidicJob::Serializer
|
7
9
|
|
8
10
|
validates :idempotency_key, presence: true # uniqueness constraint is enforced at the database level
|
9
11
|
validates :serialized_job, presence: true
|
10
12
|
|
11
|
-
scope :finished, -> {
|
12
|
-
|
13
|
-
|
14
|
-
|
13
|
+
scope :finished, -> {
|
14
|
+
where(recover_to: FINISHED_RECOVERY_POINT)
|
15
|
+
}
|
16
|
+
scope :outstanding, -> {
|
17
|
+
where.not(recover_to: FINISHED_RECOVERY_POINT).or(where(recover_to: [nil, ""]))
|
18
|
+
}
|
19
|
+
scope :clearable, -> (finished_before: AcidicJob.clear_finished_executions_after.ago) {
|
20
|
+
finished.where(last_run_at: ...finished_before)
|
21
|
+
}
|
22
|
+
|
23
|
+
def self.clear_finished_in_batches(batch_size: 500, finished_before: AcidicJob.clear_finished_executions_after.ago, sleep_between_batches: 0)
|
24
|
+
loop do
|
25
|
+
records_deleted = clearable(finished_before: finished_before).limit(batch_size).delete_all
|
26
|
+
sleep(sleep_between_batches) if sleep_between_batches > 0
|
27
|
+
break if records_deleted == 0
|
28
|
+
end
|
29
|
+
end
|
15
30
|
|
16
|
-
def record!(step:, action:, timestamp
|
31
|
+
def record!(step:, action:, timestamp: Time.current, **kwargs)
|
17
32
|
AcidicJob.instrument(:record_entry, step: step, action: action, timestamp: timestamp, data: kwargs) do
|
18
|
-
entries.
|
33
|
+
entries.insert!({
|
19
34
|
step: step,
|
20
35
|
action: action,
|
21
36
|
timestamp: timestamp,
|
22
|
-
data: kwargs.
|
23
|
-
)
|
37
|
+
data: kwargs.except(:ignored),
|
38
|
+
})
|
24
39
|
end
|
25
40
|
end
|
26
41
|
|
@@ -29,10 +44,8 @@ module AcidicJob
|
|
29
44
|
end
|
30
45
|
|
31
46
|
def finished?
|
32
|
-
# rubocop:disable Style/MultipleComparison
|
33
47
|
recover_to.to_s == FINISHED_RECOVERY_POINT ||
|
34
48
|
recover_to.to_s == "FINISHED" # old value pre-1.0, remove at v1.0
|
35
|
-
# rubocop:enable Style/MultipleComparison
|
36
49
|
end
|
37
50
|
|
38
51
|
def defined?(step)
|
data/lib/acidic_job/builder.rb
CHANGED
data/lib/acidic_job/context.rb
CHANGED
@@ -8,14 +8,31 @@ module AcidicJob
|
|
8
8
|
|
9
9
|
def set(hash)
|
10
10
|
AcidicJob.instrument(:set_context, **hash) do
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
|
18
|
-
|
11
|
+
records = hash.map do |key, value|
|
12
|
+
{
|
13
|
+
execution_id: @execution.id,
|
14
|
+
key: key,
|
15
|
+
value: value,
|
16
|
+
}
|
17
|
+
end
|
18
|
+
|
19
|
+
case AcidicJob::Value.connection.adapter_name.downcase.to_sym
|
20
|
+
when :postgresql, :sqlite
|
21
|
+
AcidicJob::Value.upsert_all(records, unique_by: [:execution_id, :key])
|
22
|
+
when :mysql2, :mysql, :trilogy
|
23
|
+
AcidicJob::Value.upsert_all(records)
|
24
|
+
else
|
25
|
+
# Fallback for other adapters - try with unique_by first, fall back without
|
26
|
+
begin
|
27
|
+
AcidicJob::Value.upsert_all(records, unique_by: [:execution_id, :key])
|
28
|
+
rescue ArgumentError => e
|
29
|
+
if e.message.include?("does not support :unique_by")
|
30
|
+
AcidicJob::Value.upsert_all(records)
|
31
|
+
else
|
32
|
+
raise
|
33
|
+
end
|
34
|
+
end
|
35
|
+
end
|
19
36
|
end
|
20
37
|
end
|
21
38
|
|
@@ -25,22 +42,21 @@ module AcidicJob
|
|
25
42
|
end
|
26
43
|
end
|
27
44
|
|
28
|
-
|
45
|
+
def fetch(key, default = nil)
|
46
|
+
result = get(key).first
|
47
|
+
return result if result
|
48
|
+
|
49
|
+
fallback = default || yield(key)
|
50
|
+
set(key => fallback)
|
51
|
+
fallback
|
52
|
+
end
|
53
|
+
|
29
54
|
def []=(key, value)
|
30
|
-
|
31
|
-
AcidicJob::Value.upsert(
|
32
|
-
{ execution_id: @execution.id,
|
33
|
-
key: key,
|
34
|
-
value: value },
|
35
|
-
unique_by: %i[execution_id key]
|
36
|
-
)
|
37
|
-
end
|
55
|
+
set(key => value)
|
38
56
|
end
|
39
57
|
|
40
58
|
def [](key)
|
41
|
-
|
42
|
-
@execution.values.select(:value).find_by(key: key)&.value
|
43
|
-
end
|
59
|
+
get(key).first
|
44
60
|
end
|
45
61
|
end
|
46
62
|
end
|
data/lib/acidic_job/errors.rb
CHANGED
@@ -28,7 +28,6 @@ module AcidicJob
|
|
28
28
|
end
|
29
29
|
end
|
30
30
|
|
31
|
-
# rubocop:disable Lint/MissingSuper
|
32
31
|
class ArgumentMismatchError < Error
|
33
32
|
def initialize(expected, existing)
|
34
33
|
@expected = expected
|
@@ -101,7 +100,7 @@ module AcidicJob
|
|
101
100
|
|
102
101
|
class DoublePluginCallError < Error
|
103
102
|
def initialize(plugin, step)
|
104
|
-
@plugin_name = Module === plugin ? plugin.name : plugin.class.name
|
103
|
+
@plugin_name = (Module === plugin) ? plugin.name : plugin.class.name
|
105
104
|
@step = step
|
106
105
|
end
|
107
106
|
|
@@ -112,7 +111,7 @@ module AcidicJob
|
|
112
111
|
|
113
112
|
class MissingPluginCallError < Error
|
114
113
|
def initialize(plugin, step)
|
115
|
-
@plugin_name = Module === plugin ? plugin.name : plugin.class.name
|
114
|
+
@plugin_name = (Module === plugin) ? plugin.name : plugin.class.name
|
116
115
|
@step = step
|
117
116
|
end
|
118
117
|
|
@@ -28,22 +28,20 @@ module AcidicJob
|
|
28
28
|
debug formatted_event(event, title: "Record entry", **event.payload.slice(:step, :action, :timestamp))
|
29
29
|
end
|
30
30
|
|
31
|
-
private
|
32
|
-
|
33
|
-
def formatted_event(event, title:, **attributes)
|
31
|
+
private def formatted_event(event, title:, **attributes)
|
34
32
|
"AcidicJob-#{AcidicJob::VERSION} #{title} (#{event.duration.round(1)}ms) #{formatted_attributes(**attributes)}"
|
35
33
|
end
|
36
34
|
|
37
|
-
def formatted_attributes(**attributes)
|
35
|
+
private def formatted_attributes(**attributes)
|
38
36
|
attributes.map { |attr, value| "#{attr}: #{value.inspect}" }.join(", ")
|
39
37
|
end
|
40
38
|
|
41
|
-
def formatted_error(error)
|
39
|
+
private def formatted_error(error)
|
42
40
|
[error.class, error.message].compact.join(" ")
|
43
41
|
end
|
44
42
|
|
45
43
|
# Use the logger configured for AcidicJob
|
46
|
-
def logger
|
44
|
+
private def logger
|
47
45
|
AcidicJob.logger
|
48
46
|
end
|
49
47
|
end
|
@@ -4,13 +4,22 @@ module AcidicJob
|
|
4
4
|
class PluginContext
|
5
5
|
PLUGIN_INACTIVE = :__ACIDIC_JOB_PLUGIN_INACTIVE__
|
6
6
|
|
7
|
-
def initialize(plugin, job, execution, step_definition)
|
7
|
+
def initialize(plugin, job, execution, context, step_definition)
|
8
8
|
@plugin = plugin
|
9
9
|
@job = job
|
10
10
|
@execution = execution
|
11
|
+
@context = context
|
11
12
|
@step_definition = step_definition
|
12
13
|
end
|
13
14
|
|
15
|
+
def set(hash)
|
16
|
+
@context.set(hash)
|
17
|
+
end
|
18
|
+
|
19
|
+
def get(*keys)
|
20
|
+
@context.get(*keys)
|
21
|
+
end
|
22
|
+
|
14
23
|
def definition
|
15
24
|
@step_definition.fetch(@plugin.keyword.to_s, PLUGIN_INACTIVE)
|
16
25
|
end
|
@@ -40,8 +49,22 @@ module AcidicJob
|
|
40
49
|
@job.enqueue(...)
|
41
50
|
end
|
42
51
|
|
43
|
-
def
|
44
|
-
@job.
|
52
|
+
def halt_workflow!
|
53
|
+
@job.halt_workflow!
|
54
|
+
end
|
55
|
+
|
56
|
+
def repeat_step!
|
57
|
+
@job.repeat_step!
|
58
|
+
end
|
59
|
+
|
60
|
+
def resolve_method(method_name)
|
61
|
+
begin
|
62
|
+
method_obj = @job.method(method_name)
|
63
|
+
rescue NameError
|
64
|
+
raise UndefinedMethodError.new(method_name)
|
65
|
+
end
|
66
|
+
|
67
|
+
method_obj
|
45
68
|
end
|
46
69
|
|
47
70
|
def plugin_action(action)
|
@@ -26,10 +26,10 @@ module AcidicJob
|
|
26
26
|
return yield if context.definition == false
|
27
27
|
|
28
28
|
model = if context.definition == true
|
29
|
-
|
30
|
-
|
31
|
-
|
32
|
-
|
29
|
+
AcidicJob::Execution
|
30
|
+
else
|
31
|
+
context.definition["on"].constantize
|
32
|
+
end
|
33
33
|
|
34
34
|
model.transaction(&block)
|
35
35
|
end
|
@@ -0,0 +1,28 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require "json"
|
4
|
+
require_relative "arguments"
|
5
|
+
|
6
|
+
module AcidicJob
|
7
|
+
# Used for `serialize` method in ActiveRecord
|
8
|
+
module Serializer
|
9
|
+
extend self
|
10
|
+
|
11
|
+
def load(json)
|
12
|
+
return if json.nil? || json.empty?
|
13
|
+
|
14
|
+
data = JSON.parse json
|
15
|
+
|
16
|
+
Arguments.__send__ :deserialize_argument, data
|
17
|
+
end
|
18
|
+
|
19
|
+
def dump(obj)
|
20
|
+
data = Arguments.__send__ :serialize_argument, obj
|
21
|
+
|
22
|
+
JSON.generate data, strict: true
|
23
|
+
rescue ActiveJob::SerializationError => e
|
24
|
+
e.message << " (`#{obj.inspect}`)"
|
25
|
+
raise e
|
26
|
+
end
|
27
|
+
end
|
28
|
+
end
|
data/lib/acidic_job/testing.rb
CHANGED
@@ -22,25 +22,23 @@ module AcidicJob
|
|
22
22
|
::DatabaseCleaner.cleaners = @original_cleaners
|
23
23
|
end
|
24
24
|
|
25
|
-
private
|
26
|
-
|
27
25
|
# Ensure that the system's original DatabaseCleaner configuration is maintained, options included,
|
28
26
|
# except that any `transaction` strategies for any ORMs are replaced with a `deletion` strategy.
|
29
|
-
def transaction_free_cleaners_for(original_cleaners)
|
27
|
+
private def transaction_free_cleaners_for(original_cleaners)
|
30
28
|
non_transaction_cleaners = original_cleaners.dup.to_h do |(orm, opts), cleaner|
|
31
29
|
[[orm, opts], ensure_no_transaction_strategies_for(cleaner)]
|
32
30
|
end
|
33
31
|
::DatabaseCleaner::Cleaners.new(non_transaction_cleaners)
|
34
32
|
end
|
35
33
|
|
36
|
-
def ensure_no_transaction_strategies_for(cleaner)
|
34
|
+
private def ensure_no_transaction_strategies_for(cleaner)
|
37
35
|
return cleaner unless strategy_name_for(cleaner) == "transaction"
|
38
36
|
|
39
37
|
cleaner.strategy = deletion_strategy_for(cleaner)
|
40
38
|
cleaner
|
41
39
|
end
|
42
40
|
|
43
|
-
def strategy_name_for(cleaner)
|
41
|
+
private def strategy_name_for(cleaner)
|
44
42
|
cleaner # <DatabaseCleaner::Cleaner>
|
45
43
|
.strategy # <DatabaseCleaner::ActiveRecord::Truncation>
|
46
44
|
.class # DatabaseCleaner::ActiveRecord::Truncation
|
@@ -50,19 +48,19 @@ module AcidicJob
|
|
50
48
|
.downcase # "truncation"
|
51
49
|
end
|
52
50
|
|
53
|
-
def deletion_strategy_for(cleaner)
|
51
|
+
private def deletion_strategy_for(cleaner)
|
54
52
|
strategy = cleaner.strategy
|
55
|
-
strategy_namespace = strategy
|
56
|
-
|
57
|
-
|
58
|
-
|
59
|
-
|
53
|
+
strategy_namespace = strategy # <DatabaseCleaner::ActiveRecord::Truncation>
|
54
|
+
.class # DatabaseCleaner::ActiveRecord::Truncation
|
55
|
+
.name # "DatabaseCleaner::ActiveRecord::Truncation"
|
56
|
+
.rpartition("::") # ["DatabaseCleaner::ActiveRecord", "::", "Truncation"]
|
57
|
+
.first # "DatabaseCleaner::ActiveRecord"
|
60
58
|
deletion_strategy_class_name = [strategy_namespace, "::", "Deletion"].join
|
61
59
|
deletion_strategy_class = deletion_strategy_class_name.constantize
|
62
60
|
instance_variable_hash = strategy.instance_variables.to_h do |var|
|
63
61
|
[
|
64
62
|
var.to_s.remove("@"),
|
65
|
-
strategy.instance_variable_get(var)
|
63
|
+
strategy.instance_variable_get(var),
|
66
64
|
]
|
67
65
|
end
|
68
66
|
options = instance_variable_hash.except("db", "connection_class")
|
data/lib/acidic_job/version.rb
CHANGED
data/lib/acidic_job/workflow.rb
CHANGED
@@ -8,7 +8,7 @@ module AcidicJob
|
|
8
8
|
HALT_STEP = :__ACIDIC_JOB_HALT_STEP_SIGNAL__
|
9
9
|
private_constant :REPEAT_STEP, :HALT_STEP
|
10
10
|
|
11
|
-
def execute_workflow(unique_by:, with:
|
11
|
+
def execute_workflow(unique_by:, with: AcidicJob.plugins, &block)
|
12
12
|
@__acidic_job_plugins__ = with
|
13
13
|
serialized_job = serialize
|
14
14
|
|
@@ -30,13 +30,13 @@ module AcidicJob
|
|
30
30
|
|
31
31
|
AcidicJob.instrument(:initialize_workflow, definition: workflow_definition) do
|
32
32
|
transaction_args = case ::ActiveRecord::Base.connection.adapter_name.downcase.to_sym
|
33
|
-
|
34
|
-
|
35
|
-
|
36
|
-
|
37
|
-
|
38
|
-
|
39
|
-
idempotency_key = Digest::SHA256.hexdigest(JSON.
|
33
|
+
# SQLite doesn't support `serializable` transactions
|
34
|
+
when :sqlite
|
35
|
+
{}
|
36
|
+
else
|
37
|
+
{ isolation: :serializable }
|
38
|
+
end
|
39
|
+
idempotency_key = Digest::SHA256.hexdigest(JSON.generate([self.class.name, unique_by], strict: true))
|
40
40
|
|
41
41
|
@__acidic_job_execution__ = ::ActiveRecord::Base.transaction(**transaction_args) do
|
42
42
|
record = Execution.find_by(idempotency_key: idempotency_key)
|
@@ -61,11 +61,11 @@ module AcidicJob
|
|
61
61
|
)
|
62
62
|
else
|
63
63
|
starting_point = if workflow_definition.key?("steps")
|
64
|
-
|
65
|
-
|
66
|
-
|
67
|
-
|
68
|
-
|
64
|
+
workflow_definition["steps"].keys.first
|
65
|
+
else
|
66
|
+
# TODO: add deprecation warning
|
67
|
+
workflow_definition.keys.first
|
68
|
+
end
|
69
69
|
|
70
70
|
record = Execution.create!(
|
71
71
|
idempotency_key: idempotency_key,
|
@@ -89,7 +89,7 @@ module AcidicJob
|
|
89
89
|
|
90
90
|
current_step = @__acidic_job_execution__.recover_to
|
91
91
|
|
92
|
-
if not @__acidic_job_execution__.defined?(current_step)
|
92
|
+
if not @__acidic_job_execution__.defined?(current_step)
|
93
93
|
raise UndefinedStepError.new(current_step)
|
94
94
|
end
|
95
95
|
|
@@ -101,11 +101,10 @@ module AcidicJob
|
|
101
101
|
@__acidic_job_execution__.record!(
|
102
102
|
step: step_definition.fetch("does"),
|
103
103
|
action: :halted,
|
104
|
-
timestamp: Time.now
|
105
104
|
)
|
106
105
|
return true
|
107
106
|
else
|
108
|
-
@__acidic_job_execution__.
|
107
|
+
@__acidic_job_execution__.update_column(:recover_to, recover_to)
|
109
108
|
end
|
110
109
|
end
|
111
110
|
end
|
@@ -116,14 +115,19 @@ module AcidicJob
|
|
116
115
|
throw :repeat, REPEAT_STEP
|
117
116
|
end
|
118
117
|
|
119
|
-
def
|
118
|
+
def halt_workflow!
|
120
119
|
throw :halt, HALT_STEP
|
121
120
|
end
|
122
121
|
|
122
|
+
def halt_step!
|
123
|
+
# TODO add deprecation warning
|
124
|
+
halt_workflow!
|
125
|
+
end
|
126
|
+
|
123
127
|
def step_retrying?
|
124
128
|
step_name = caller_locations.first.label
|
125
129
|
|
126
|
-
if not @__acidic_job_execution__.defined?(step_name)
|
130
|
+
if not @__acidic_job_execution__.defined?(step_name)
|
127
131
|
raise UndefinedStepError.new(step_name)
|
128
132
|
end
|
129
133
|
|
@@ -138,9 +142,7 @@ module AcidicJob
|
|
138
142
|
@__acidic_job_context__
|
139
143
|
end
|
140
144
|
|
141
|
-
private
|
142
|
-
|
143
|
-
def take_step(step_definition)
|
145
|
+
private def take_step(step_definition)
|
144
146
|
curr_step = step_definition.fetch("does")
|
145
147
|
next_step = step_definition.fetch("then")
|
146
148
|
|
@@ -148,7 +150,7 @@ module AcidicJob
|
|
148
150
|
|
149
151
|
rescued_error = nil
|
150
152
|
begin
|
151
|
-
@__acidic_job_execution__.record!(step: curr_step, action: :started
|
153
|
+
@__acidic_job_execution__.record!(step: curr_step, action: :started)
|
152
154
|
result = AcidicJob.instrument(:perform_step, **step_definition) do
|
153
155
|
perform_step_for(step_definition)
|
154
156
|
end
|
@@ -156,10 +158,16 @@ module AcidicJob
|
|
156
158
|
when REPEAT_STEP
|
157
159
|
curr_step
|
158
160
|
else
|
159
|
-
@__acidic_job_execution__.record!(
|
161
|
+
@__acidic_job_execution__.record!(
|
162
|
+
step: curr_step,
|
163
|
+
action: :succeeded,
|
164
|
+
ignored: {
|
165
|
+
result: result,
|
166
|
+
}
|
167
|
+
)
|
160
168
|
next_step
|
161
169
|
end
|
162
|
-
rescue
|
170
|
+
rescue => e
|
163
171
|
rescued_error = e
|
164
172
|
raise e
|
165
173
|
ensure
|
@@ -168,11 +176,10 @@ module AcidicJob
|
|
168
176
|
@__acidic_job_execution__.record!(
|
169
177
|
step: curr_step,
|
170
178
|
action: :errored,
|
171
|
-
timestamp: Time.now,
|
172
179
|
exception_class: rescued_error.class.name,
|
173
180
|
message: rescued_error.message
|
174
181
|
)
|
175
|
-
rescue
|
182
|
+
rescue => e
|
176
183
|
# We're already inside an error condition, so swallow any additional
|
177
184
|
# errors from here and just send them to logs.
|
178
185
|
logger.error(
|
@@ -183,7 +190,7 @@ module AcidicJob
|
|
183
190
|
end
|
184
191
|
end
|
185
192
|
|
186
|
-
def perform_step_for(step_definition)
|
193
|
+
private def perform_step_for(step_definition)
|
187
194
|
step_name = step_definition.fetch("does")
|
188
195
|
begin
|
189
196
|
step_method = method(step_name)
|
@@ -191,10 +198,10 @@ module AcidicJob
|
|
191
198
|
raise UndefinedMethodError.new(step_name)
|
192
199
|
end
|
193
200
|
|
194
|
-
raise InvalidMethodError.new(step_name) unless step_method.arity.zero?
|
201
|
+
# raise InvalidMethodError.new(step_name) unless step_method.arity.zero?
|
195
202
|
|
196
203
|
plugin_pipeline_callable = @__acidic_job_plugins__.reverse.reduce(step_method) do |callable, plugin|
|
197
|
-
context = PluginContext.new(plugin, self, @__acidic_job_execution__, step_definition)
|
204
|
+
context = PluginContext.new(plugin, self, @__acidic_job_execution__, @__acidic_job_context__, step_definition)
|
198
205
|
|
199
206
|
if context.inactive?
|
200
207
|
callable
|
@@ -202,14 +209,19 @@ module AcidicJob
|
|
202
209
|
proc do
|
203
210
|
called = false
|
204
211
|
|
205
|
-
result = plugin.around_step(context) do
|
212
|
+
result = plugin.around_step(context) do |*args, **kwargs|
|
206
213
|
raise DoublePluginCallError.new(plugin, step_name) if called
|
207
214
|
|
208
215
|
called = true
|
209
|
-
|
216
|
+
|
217
|
+
if callable.arity.zero?
|
218
|
+
callable.call
|
219
|
+
else
|
220
|
+
callable.call(*args, **kwargs)
|
221
|
+
end
|
210
222
|
end
|
211
223
|
|
212
|
-
raise MissingPluginCallError.new(plugin, step_name) unless called
|
224
|
+
# raise MissingPluginCallError.new(plugin, step_name) unless called
|
213
225
|
|
214
226
|
result
|
215
227
|
end
|