active_orchestrator 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/ADAPTERS.md +60 -0
- data/CHANGELOG.md +7 -0
- data/GUIDE.md +80 -0
- data/LICENSE +21 -0
- data/README.md +444 -0
- data/lib/active_workflow/configuration.rb +23 -0
- data/lib/active_workflow/context.rb +82 -0
- data/lib/active_workflow/dsl/options.rb +34 -0
- data/lib/active_workflow/dsl/signals.rb +24 -0
- data/lib/active_workflow/dsl/steps.rb +131 -0
- data/lib/active_workflow/errors.rb +18 -0
- data/lib/active_workflow/execution.rb +65 -0
- data/lib/active_workflow/jobs/runner_job.rb +20 -0
- data/lib/active_workflow/railtie.rb +17 -0
- data/lib/active_workflow/serializers/json.rb +20 -0
- data/lib/active_workflow/stores/active_record.rb +695 -0
- data/lib/active_workflow/stores/base.rb +58 -0
- data/lib/active_workflow/task.rb +32 -0
- data/lib/active_workflow/version.rb +5 -0
- data/lib/active_workflow/workflow.rb +136 -0
- data/lib/active_workflow.rb +94 -0
- data/lib/generators/active_workflow/install/install_generator.rb +43 -0
- data/lib/generators/active_workflow/install/templates/initializer.rb +8 -0
- data/lib/generators/active_workflow/install/templates/migrations/create_active_workflow_tables.rb +69 -0
- data/lib/generators/active_workflow/install/templates/sample_workflow.rb +29 -0
- data/lib/generators/active_workflow/workflow/templates/workflow.rb +18 -0
- data/lib/generators/active_workflow/workflow/workflow_generator.rb +17 -0
- metadata +173 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: ff02615a87a2bd821b74beb97c312b4c91f8f7de973f12bc99c0030b8b62c143
|
4
|
+
data.tar.gz: 3c2b13cbe51135bd60cf7113ea4e2958d743ec019ae907ee1160693916724f99
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: bbac23cc97b84e2e72a08a310150b34dfc15ba4abae3e40dfcc14a0a5c940bf2d3d63b37c1135c48ed7750a040a7c6a8cfba100f36f63901ebf4e68b49e9f830
|
7
|
+
data.tar.gz: 923ee72ded50cee24bf6a0a26cd8eca3034ec932f0bb87346b29c1b6ebe72d14935702125c01868628974b2f06cdb3dcfd53fdac5c05e1dc5a66c445627f9d42
|
data/ADAPTERS.md
ADDED
@@ -0,0 +1,60 @@
|
|
1
|
+
# Store Adapters
|
2
|
+
|
3
|
+
ActiveWorkflow persists workflow state through a strategy interface. This document explains the
|
4
|
+
contract adapters must implement and the behaviours the engine expects.
|
5
|
+
|
6
|
+
## Base Contract
|
7
|
+
|
8
|
+
All adapters inherit from `ActiveWorkflow::Stores::Base` and must implement:
|
9
|
+
|
10
|
+
- `start_execution(workflow_class:, context:, steps:, idempotency_key:, timeout:, metadata:)`
|
11
|
+
- `load_execution(id)`
|
12
|
+
- `process_execution(execution_id)`
|
13
|
+
- `complete_step!(execution_id, step_name, payload:, idempotency_key:)`
|
14
|
+
- `fail_step!(execution_id, step_name, error_class:, message:, details:, idempotency_key:)`
|
15
|
+
- `extend_timeout!(execution_id, step_name, by:)`
|
16
|
+
- `heartbeat!(execution_id, step_name, at:)`
|
17
|
+
- `signal!(execution_id, name, payload:)`
|
18
|
+
- `enqueue_runner(execution_id, run_at: nil)` – adapters may simply enqueue the internal runner job.
|
19
|
+
|
20
|
+
The methods must be **atomic**: never run business code outside a transaction or lock that protects
|
21
|
+
against concurrent runners.
|
22
|
+
|
23
|
+
## Step Lifecycle Responsibilities
|
24
|
+
|
25
|
+
Adapters are responsible for:
|
26
|
+
|
27
|
+
1. **Idempotency** – enforce unique workflow `idempotency_key` presence and step completion keys.
|
28
|
+
2. **Exactly-once** step completion – step must transition `pending → running → completed` without
|
29
|
+
double execution when jobs retry.
|
30
|
+
3. **Wait semantics** – async steps transition to `waiting` with timeout scheduling and resume only
|
31
|
+
via completion, failure, or timeout.
|
32
|
+
4. **Retry bookkeeping** – store attempts, backoff, jitter, and next scheduled run.
|
33
|
+
5. **Compensation triggering** – when execution fails terminally, invoke compensations in reverse
|
34
|
+
order.
|
35
|
+
6. **Observability** – emit `ActiveSupport::Notifications` listed in the README.
|
36
|
+
|
37
|
+
## Extending the ActiveRecord Store
|
38
|
+
|
39
|
+
The built-in adapter is designed for relational databases and uses three tables:
|
40
|
+
|
41
|
+
- `aw_executions` – workflow metadata and context.
|
42
|
+
- `aw_steps` – per-step state machine and retry/timeout metadata.
|
43
|
+
- `aw_events` – signal payloads awaiting consumption.
|
44
|
+
|
45
|
+
If you need to customise the schema (prefixed table names, auditing columns), pass your own models or
|
46
|
+
subclass the adapter. Ensure indexes that protect idempotency keys remain unique.
|
47
|
+
|
48
|
+
## Testing Adapters
|
49
|
+
|
50
|
+
Use the acceptance spec suite as a smoke test for new adapters. You can swap the store in
|
51
|
+
`spec_helper.rb` to your implementation and assert the scenarios continue to pass:
|
52
|
+
|
53
|
+
```ruby
|
54
|
+
ActiveWorkflow.configure do |config|
|
55
|
+
config.store = MyRedisStore.new
|
56
|
+
end
|
57
|
+
```
|
58
|
+
|
59
|
+
Because workflows rely on precise concurrency guarantees, add stress tests around jobs retrying,
|
60
|
+
timeouts, and simulated crashes before deploying alternative stores to production.
|
data/CHANGELOG.md
ADDED
data/GUIDE.md
ADDED
@@ -0,0 +1,80 @@
|
|
1
|
+
# ActiveWorkflow Guide
|
2
|
+
|
3
|
+
This guide expands on the README and focuses on advanced topics you will likely encounter when
|
4
|
+
operating ActiveWorkflow in production.
|
5
|
+
|
6
|
+
- [Security & Webhooks](#security--webhooks)
|
7
|
+
- [Correlation Patterns](#correlation-patterns)
|
8
|
+
- [Extending Stores](#extending-stores)
|
9
|
+
- [Observability](#observability)
|
10
|
+
|
11
|
+
## Security & Webhooks
|
12
|
+
|
13
|
+
Async workflows commonly rely on webhook callbacks to call `ActiveWorkflow.complete_step!` or
|
14
|
+
`ActiveWorkflow.fail_step!`. Treat these endpoints with the same rigor as payment webhooks:
|
15
|
+
|
16
|
+
1. **Authenticate** inbound requests. Sign responses from your external system and verify via HMAC
|
17
|
+
in the controller before mutating workflow state.
|
18
|
+
2. **Use correlation IDs**. The gem exposes `execution.id` and the step name. Pass a
|
19
|
+
deterministic correlation ID to your external service (for example `"#{execution.id}:export"`).
|
20
|
+
3. **Enforce idempotency**. Always forward a business-level idempotency key (event id) and supply it
|
21
|
+
when calling `complete_step!`. The ActiveRecord store keeps the last key and will reject
|
22
|
+
conflicting attempts.
|
23
|
+
4. **Rate-limit**. Protect the endpoints that mutate workflow state and monitor for abuse.
|
24
|
+
|
25
|
+
Example controller snippet:
|
26
|
+
|
27
|
+
```ruby
|
28
|
+
class ExportsController < ApplicationController
|
29
|
+
before_action :verify_hmac!
|
30
|
+
|
31
|
+
def ready
|
32
|
+
ActiveWorkflow.complete_step!(params[:execution_id], :export,
|
33
|
+
payload: params.require(:payload).permit(:url),
|
34
|
+
idempotency_key: params[:event_id])
|
35
|
+
head :ok
|
36
|
+
rescue ActiveWorkflow::Errors::AsyncCompletionConflict
|
37
|
+
head :unprocessable_entity
|
38
|
+
end
|
39
|
+
end
|
40
|
+
```
|
41
|
+
|
42
|
+
## Correlation Patterns
|
43
|
+
|
44
|
+
ActiveWorkflow does not impose a single correlation strategy. Some battle-tested options:
|
45
|
+
|
46
|
+
- **Execution scoped** – `"#{execution.id}:#{step_name}"` works well when the external system does
|
47
|
+
not need to know about your domain objects.
|
48
|
+
- **Business scoped** – `"export:#{ctx[:user_id]}"` if a user can only have one export at a time.
|
49
|
+
- **Composite** – combine execution and external ids to allow reconciliation jobs to find the
|
50
|
+
correct workflow.
|
51
|
+
|
52
|
+
Whatever you choose, store the value either in `ctx` or the step's `init_result` and reuse it for
|
53
|
+
timeouts, heartbeats, and compensations.
|
54
|
+
|
55
|
+
## Extending Stores
|
56
|
+
|
57
|
+
`ActiveWorkflow::Stores::Base` defines the contract for persistence. To write a custom adapter
|
58
|
+
(Redis, DynamoDB…), inherit from the base class and implement:
|
59
|
+
|
60
|
+
- `start_execution` – create an execution with initial steps and enqueue the runner job.
|
61
|
+
- `load_execution` – return an `ActiveWorkflow::Execution` snapshot.
|
62
|
+
- `process_execution` – pop and execute the next step atomically.
|
63
|
+
- Async APIs – `complete_step!`, `fail_step!`, `extend_timeout!`, `heartbeat!`, `signal!`.
|
64
|
+
|
65
|
+
The ActiveRecord store is heavily documented and can act as a reference implementation. When porting
|
66
|
+
it ensure you preserve transactional guarantees around step dedupe, retries, and compensations.
|
67
|
+
|
68
|
+
## Observability
|
69
|
+
|
70
|
+
The store emits `ActiveSupport::Notifications` for every significant transition. Subscribe in your
|
71
|
+
app and ship them to OpenTelemetry, Datadog, or logs. A simple logger subscriber looks like this:
|
72
|
+
|
73
|
+
```ruby
|
74
|
+
ActiveSupport::Notifications.subscribe(/active_workflow\./) do |event|
|
75
|
+
Rails.logger.info({ name: event.name, payload: event.payload, duration: event.duration }.to_json)
|
76
|
+
end
|
77
|
+
```
|
78
|
+
|
79
|
+
Use the events to drive SLIs (time-to-signal, retry depth, compensation rate) and to alert on stuck
|
80
|
+
workflows (no heartbeat before timeout).
|
data/LICENSE
ADDED
@@ -0,0 +1,21 @@
|
|
1
|
+
MIT License
|
2
|
+
|
3
|
+
Copyright (c) 2024 ActiveWorkflow
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
7
|
+
in the Software without restriction, including without limitation the rights
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
10
|
+
furnished to do so, subject to the following conditions:
|
11
|
+
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
13
|
+
copies or substantial portions of the Software.
|
14
|
+
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
21
|
+
SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,444 @@
|
|
1
|
+
# ActiveWorkflow
|
2
|
+
|
3
|
+
A workflow engine with durable steps, automatic retries with backoff and jitter, compensations, async waits, signals, timeouts, idempotency, and pluggable persistence (ActiveRecord included).
|
4
|
+
|
5
|
+
---
|
6
|
+
|
7
|
+
## Table of Contents
|
8
|
+
|
9
|
+
- [Why ActiveWorkflow?](#why-activeworkflow)
|
10
|
+
- [Features](#features)
|
11
|
+
- [Installation](#installation)
|
12
|
+
- [Quick Start](#quick-start)
|
13
|
+
- [Defining Workflows (3 equivalent styles)](#defining-workflows-3-equivalent-styles)
|
14
|
+
- [A) Method step](#a-method-step)
|
15
|
+
- [B) Task class](#b-task-class)
|
16
|
+
- [C) Inline block](#c-inline-block)
|
17
|
+
- [Async Steps (vs Sync) – Concept & Lifecycle](#async-steps-vs-sync--concept--lifecycle)
|
18
|
+
- [Declaring async steps](#declaring-async-steps)
|
19
|
+
- [Completion/Failure/Timeout APIs](#completionfailuretimeout-apis)
|
20
|
+
- [Timeouts, retries, and re-initiation](#timeouts-retries-and-re-initiation)
|
21
|
+
- [Security, idempotency & race-safety](#security-idempotency--race-safety)
|
22
|
+
- [Sync vs Async comparison](#sync-vs-async-comparison)
|
23
|
+
- [Signals & Waits](#signals--waits)
|
24
|
+
- [Retries, Backoff & Jitter](#retries-backoff--jitter)
|
25
|
+
- [Idempotency & Deduplication](#idempotency--deduplication)
|
26
|
+
- [Configuration](#configuration)
|
27
|
+
- [Persistence (Adapters)](#persistence-adapters)
|
28
|
+
- [Active Job Integration](#active-job-integration)
|
29
|
+
- [Observability (Events & Logging)](#observability-events--logging)
|
30
|
+
- [Generators](#generators)
|
31
|
+
- [Testing](#testing)
|
32
|
+
- [Examples](#examples)
|
33
|
+
- [Design Notes](#design-notes)
|
34
|
+
- [License](#license)
|
35
|
+
|
36
|
+
---
|
37
|
+
|
38
|
+
## Why ActiveWorkflow?
|
39
|
+
|
40
|
+
Most apps need reliable business workflows (sagas) with retries, compensations and “wait for an external thing (webhook/job)”—without operating a heavyweight orchestration service. **ActiveWorkflow** gives you that on plain Rails + Active Job.
|
41
|
+
|
42
|
+
---
|
43
|
+
|
44
|
+
## Features
|
45
|
+
|
46
|
+
- **Three step styles**: method, task class, or inline block – same options & semantics.
|
47
|
+
- **Async steps**: start external work, persist correlation, **wait** for completion via API.
|
48
|
+
- **Signals**: pause the flow until an external event arrives.
|
49
|
+
- **Retries**: fixed/exponential backoff with jitter, per-step timeouts.
|
50
|
+
- **Compensations**: reverse-order undo on cancel/failure.
|
51
|
+
- **Idempotency**: workflow-level keys + step dedupe + idempotent completions.
|
52
|
+
- **Pluggable stores**: strategy interface; ships with **ActiveRecord**.
|
53
|
+
- **Observability**: `ActiveSupport::Notifications` for steps, signals, retries, timeouts.
|
54
|
+
|
55
|
+
---
|
56
|
+
|
57
|
+
## Installation
|
58
|
+
|
59
|
+
```bash
|
60
|
+
# Gemfile
|
61
|
+
gem "active_workflow"
|
62
|
+
|
63
|
+
bundle install
|
64
|
+
rails g active_workflow:install
|
65
|
+
rails db:migrate
|
66
|
+
```
|
67
|
+
|
68
|
+
The installer creates:
|
69
|
+
|
70
|
+
- `config/initializers/active_workflow.rb`
|
71
|
+
- ActiveRecord migrations
|
72
|
+
- Optional sample workflow
|
73
|
+
|
74
|
+
---
|
75
|
+
|
76
|
+
## Quick Start
|
77
|
+
|
78
|
+
```ruby
|
79
|
+
# config/initializers/active_workflow.rb
|
80
|
+
ActiveWorkflow.configure do |c|
|
81
|
+
c.store = ActiveWorkflow::Stores::ActiveRecord.new
|
82
|
+
c.logger = Rails.logger
|
83
|
+
c.serializer = :json
|
84
|
+
c.clock = -> { Time.now.utc }
|
85
|
+
end
|
86
|
+
```
|
87
|
+
|
88
|
+
```ruby
|
89
|
+
# app/workflows/checkout_flow.rb
|
90
|
+
class CheckoutFlow < ActiveWorkflow::Workflow
|
91
|
+
idempotency_key { "checkout:#{ctx[:order_id]}" }
|
92
|
+
defaults retry: { max: 5, backoff: :exponential, jitter: true, first_delay: 1.second }
|
93
|
+
timeout 30.minutes
|
94
|
+
|
95
|
+
step :charge_card, compensate: :refund_payment, retry: { max: 6, first_delay: 2.seconds }
|
96
|
+
task :reserve_stock, ReserveStockTask, dedupe: true
|
97
|
+
task :send_receipt, fire_and_forget: true do |ctx|
|
98
|
+
Mailers::Receipt.deliver_later(ctx[:order_id])
|
99
|
+
end
|
100
|
+
|
101
|
+
def charge_card
|
102
|
+
payment = PSP.charge!(order_id: ctx[:order_id], token: ctx[:payment_token])
|
103
|
+
ctx[:payment_id] = payment.id
|
104
|
+
end
|
105
|
+
|
106
|
+
def refund_payment
|
107
|
+
PSP.refund!(ctx[:payment_id]) if ctx[:payment_id]
|
108
|
+
end
|
109
|
+
end
|
110
|
+
```
|
111
|
+
|
112
|
+
```ruby
|
113
|
+
# Run it
|
114
|
+
flow = CheckoutFlow.start(order_id: 42, payment_token: "tok_123")
|
115
|
+
result = flow.await # optional: block until terminal state
|
116
|
+
```
|
117
|
+
|
118
|
+
---
|
119
|
+
|
120
|
+
## Defining Workflows (3 equivalent styles)
|
121
|
+
|
122
|
+
All step forms accept the **same options**:
|
123
|
+
`retry:, compensate:, timeout:, async:, dedupe:, if:, unless:, args:, store_result_as:, fire_and_forget:`.
|
124
|
+
|
125
|
+
### A) Method step
|
126
|
+
|
127
|
+
```ruby
|
128
|
+
class ExampleFlow < ActiveWorkflow::Workflow
|
129
|
+
step :do_work, compensate: :undo_work
|
130
|
+
|
131
|
+
def do_work
|
132
|
+
res = Service.call!(ctx[:input])
|
133
|
+
ctx[:output] = res
|
134
|
+
end
|
135
|
+
|
136
|
+
def undo_work
|
137
|
+
Service.undo!(ctx[:output]) if ctx[:output]
|
138
|
+
end
|
139
|
+
end
|
140
|
+
```
|
141
|
+
|
142
|
+
### B) Task class
|
143
|
+
|
144
|
+
```ruby
|
145
|
+
class ReserveStockTask < ActiveWorkflow::Task
|
146
|
+
def call(ctx)
|
147
|
+
res = Inventory.reserve!(ctx[:order_id])
|
148
|
+
{ reservation_id: res.id } # merged into ctx
|
149
|
+
end
|
150
|
+
|
151
|
+
def compensate(ctx, result:)
|
152
|
+
Inventory.release!(result[:reservation_id]) if result&.dig(:reservation_id)
|
153
|
+
end
|
154
|
+
end
|
155
|
+
|
156
|
+
class CheckoutFlow < ActiveWorkflow::Workflow
|
157
|
+
task :reserve_stock, ReserveStockTask
|
158
|
+
end
|
159
|
+
```
|
160
|
+
|
161
|
+
### C) Inline block
|
162
|
+
|
163
|
+
```ruby
|
164
|
+
class NotifyFlow < ActiveWorkflow::Workflow
|
165
|
+
task :send_email do |ctx|
|
166
|
+
Mailers::Notice.deliver_later(ctx[:user_id])
|
167
|
+
nil
|
168
|
+
end
|
169
|
+
end
|
170
|
+
```
|
171
|
+
|
172
|
+
---
|
173
|
+
|
174
|
+
## Async Steps (vs Sync) – Concept & Lifecycle
|
175
|
+
|
176
|
+
**Sync step:** runs to completion within one worker execution. Returns value → advance; raises → retry/fail.
|
177
|
+
|
178
|
+
**Async step:** **initiates** external work and **returns immediately**. The engine persists the step in `waiting` state (with correlation/timeout). Later, an external caller resumes the step via `complete_step!` (or `fail_step!`). No worker thread is held while waiting.
|
179
|
+
|
180
|
+
### Declaring async steps
|
181
|
+
|
182
|
+
```ruby
|
183
|
+
# Per-step option
|
184
|
+
task :arrange_fulfillment, FulfillmentTask, async: true, timeout: 15.minutes
|
185
|
+
|
186
|
+
# Class-level declaration
|
187
|
+
class FulfillmentTask < ActiveWorkflow::Task
|
188
|
+
async! timeout: 15.minutes
|
189
|
+
def call(ctx)
|
190
|
+
job = FulfillmentAPI.create_job!(order_id: ctx[:order_id]) # initiate
|
191
|
+
{ fulfillment_job_id: job.id } # merge into ctx
|
192
|
+
end
|
193
|
+
end
|
194
|
+
```
|
195
|
+
|
196
|
+
**Engine behavior when async:**
|
197
|
+
|
198
|
+
- Runs `call` once to initiate work.
|
199
|
+
- If `call` returns, **persist** step → `waiting`, set `waiting_since`, compute `timeout_at`.
|
200
|
+
- If `call` raises, treat like sync failure (retry policy).
|
201
|
+
- Does **not** advance the cursor until a completion arrives.
|
202
|
+
|
203
|
+
### Completion/Failure/Timeout APIs
|
204
|
+
|
205
|
+
```ruby
|
206
|
+
# Mark an async step as completed and advance the flow
|
207
|
+
ActiveWorkflow.complete_step!(execution_id, :arrange_fulfillment,
|
208
|
+
payload: { tracking: "XYZ" }, idempotency_key: params[:event_id])
|
209
|
+
|
210
|
+
# Explicit failure path while waiting
|
211
|
+
ActiveWorkflow.fail_step!(execution_id, :arrange_fulfillment,
|
212
|
+
error_class: "RemoteError", message: "Timeout from vendor", details: { ... },
|
213
|
+
idempotency_key: params[:event_id])
|
214
|
+
|
215
|
+
# Optional: extend waiting timeout
|
216
|
+
ActiveWorkflow.extend_timeout!(execution_id, :arrange_fulfillment, by: 10.minutes)
|
217
|
+
|
218
|
+
# Optional: heartbeat for monitoring
|
219
|
+
ActiveWorkflow.heartbeat!(execution_id, :arrange_fulfillment)
|
220
|
+
```
|
221
|
+
|
222
|
+
**Payload storage:**
|
223
|
+
|
224
|
+
- Initiation return (from `call`) is merged into `ctx` immediately.
|
225
|
+
- Completion payload is stored under `ctx[step_name]` (or `store_result_as:` key).
|
226
|
+
|
227
|
+
### Timeouts, retries, and re-initiation
|
228
|
+
|
229
|
+
- `timeout:` caps **waiting** time. On timeout:
|
230
|
+
- If `retry` configured → **re-initiate** the step (re-run `call`) with backoff+jitter.
|
231
|
+
- Else mark step `failed`; apply flow failure/compensation policy.
|
232
|
+
- Re-initiation must be **idempotent**—use business idempotency keys (e.g., `execution_id:step_name`) for external calls.
|
233
|
+
|
234
|
+
### Security, idempotency & race-safety
|
235
|
+
|
236
|
+
- `complete_step!` / `fail_step!` are **idempotent** (accept `idempotency_key`).
|
237
|
+
- Optionally verify a **correlation_id** created during initiation before accepting completion.
|
238
|
+
- Use DB transactional updates and row locks so only one runner transitions a step.
|
239
|
+
- If a step isn’t `waiting`, completion/failure is a safe no-op.
|
240
|
+
|
241
|
+
### Sync vs Async comparison
|
242
|
+
|
243
|
+
| Aspect | Sync step | Async step |
|
244
|
+
| --------------------- | -------------------------------------- | ------------------------------------------------------------------------ |
|
245
|
+
| Worker time | Runs to completion in one job | Initiates work, returns immediately; later resumed via API |
|
246
|
+
| State progression | `pending → running → completed/failed` | `pending → running → waiting → (completed/failed/timed_out)` |
|
247
|
+
| Cursor advancement | Immediately after success | After `complete_step!` |
|
248
|
+
| Failure while waiting | N/A | `fail_step!` or timeout triggers retry/fail |
|
249
|
+
| Timeouts | Caps execution time | Caps waiting time; can `extend_timeout!` or `heartbeat!` |
|
250
|
+
| Idempotency | Step-level `dedupe` | Idempotent completions with `idempotency_key`; re-init safe with backoff |
|
251
|
+
|
252
|
+
---
|
253
|
+
|
254
|
+
## Signals & Waits
|
255
|
+
|
256
|
+
Pause the workflow until a signal arrives.
|
257
|
+
|
258
|
+
```ruby
|
259
|
+
class ApprovalFlow < ActiveWorkflow::Workflow
|
260
|
+
task :prepare, PrepareTask
|
261
|
+
wait_for_signal :approval, as: :approval
|
262
|
+
task :finalize, FinalizeTask, args: ->(ctx) { [ctx[:approval]] }
|
263
|
+
end
|
264
|
+
|
265
|
+
# Controller/webhook sending a signal:
|
266
|
+
ActiveWorkflow.signal!(params[:execution_id], :approval,
|
267
|
+
payload: { approved_by: current_user.id, decision: "approved" })
|
268
|
+
```
|
269
|
+
|
270
|
+
Signals are persisted events, consumed once, and idempotent.
|
271
|
+
|
272
|
+
---
|
273
|
+
|
274
|
+
## Retries, Backoff & Jitter
|
275
|
+
|
276
|
+
Specify per step or via `defaults`:
|
277
|
+
|
278
|
+
```ruby
|
279
|
+
defaults retry: { max: 5, backoff: :exponential, first_delay: 1.second, jitter: true }
|
280
|
+
step :slow_call, retry: { max: 10, backoff: :fixed, delay: 5.seconds }
|
281
|
+
```
|
282
|
+
|
283
|
+
- `backoff:` `:fixed` or `:exponential`
|
284
|
+
- `first_delay:` initial wait before first retry
|
285
|
+
- `delay:` (for fixed)
|
286
|
+
- `jitter: true` to randomize and reduce thundering herd
|
287
|
+
|
288
|
+
---
|
289
|
+
|
290
|
+
## Idempotency & Deduplication
|
291
|
+
|
292
|
+
- **Workflow level**: `idempotency_key { "checkout:#{ctx[:order_id]}" }` ensures only one logical execution per business key.
|
293
|
+
- **Step level**: `dedupe: true` skips re-executing a step that already completed successfully.
|
294
|
+
- **Async completion**: `complete_step!(..., idempotency_key:)` prevents duplicate completions.
|
295
|
+
|
296
|
+
---
|
297
|
+
|
298
|
+
## Configuration
|
299
|
+
|
300
|
+
```ruby
|
301
|
+
ActiveWorkflow.configure do |c|
|
302
|
+
c.store = ActiveWorkflow::Stores::ActiveRecord.new
|
303
|
+
c.logger = Rails.logger
|
304
|
+
c.serializer = :json # or a custom object responding to dump/load
|
305
|
+
c.clock = -> { Time.now.utc }
|
306
|
+
end
|
307
|
+
```
|
308
|
+
|
309
|
+
---
|
310
|
+
|
311
|
+
## Persistence (Adapters)
|
312
|
+
|
313
|
+
ActiveWorkflow uses a **Store** strategy.
|
314
|
+
|
315
|
+
**ActiveRecord store** (ships with gem):
|
316
|
+
|
317
|
+
- Tables (prefix `aw_`):
|
318
|
+
- `aw_executions`: workflow metadata (`workflow_class`, `state`, `ctx` jsonb, `cursor_step`, `idempotency_key`, timestamps)
|
319
|
+
- `aw_steps`: per-step status (`pending|running|waiting|completed|failed|timed_out|compensating|compensated`), attempts, backoff data, `init_result`, `completion_payload`, `timeout_at`, `waiting_since`, `heartbeat_at`, `correlation_id`, `completion_idempotency_key`
|
320
|
+
- `aw_events`: signals etc. (`name`, `payload`, `consumed_at`)
|
321
|
+
- (Optional) `aw_timers`: scheduled timeouts/backoffs (or embed in steps)
|
322
|
+
- Indexing: unique on `idempotency_key`; partial unique on `(execution_id, step_name, completion_idempotency_key) WHERE completion_idempotency_key IS NOT NULL`; JSONB GIN on payloads if needed.
|
323
|
+
- Concurrency: transactional updates + row locks / SKIP LOCKED.
|
324
|
+
|
325
|
+
You can implement other stores (e.g., Redis) by following the `ActiveWorkflow::Store` interface.
|
326
|
+
|
327
|
+
---
|
328
|
+
|
329
|
+
## Active Job Integration
|
330
|
+
|
331
|
+
- A single internal job (e.g., `ActiveWorkflow::Jobs::RunnerJob`) receives execution/step IDs and performs the next transition safely.
|
332
|
+
- **No dependency** on specific backends (Sidekiq, Solid Queue, etc.)—any Active Job adapter works.
|
333
|
+
- Timers/backoffs are scheduled via `set(wait_until: ...)`.
|
334
|
+
|
335
|
+
---
|
336
|
+
|
337
|
+
## Observability (Events & Logging)
|
338
|
+
|
339
|
+
Subscribe with `ActiveSupport::Notifications`:
|
340
|
+
|
341
|
+
Events include:
|
342
|
+
|
343
|
+
- `active_workflow.step.started`
|
344
|
+
- `active_workflow.step.completed`
|
345
|
+
- `active_workflow.step.failed`
|
346
|
+
- `active_workflow.step.waiting` _(async initiation successful)_
|
347
|
+
- `active_workflow.step.completed_async` _(after `complete_step!`)_
|
348
|
+
- `active_workflow.step.failed_async` _(after `fail_step!`)_
|
349
|
+
- `active_workflow.step.timeout`
|
350
|
+
- `active_workflow.retry.scheduled`
|
351
|
+
- `active_workflow.signal.received`
|
352
|
+
|
353
|
+
Each event carries identifiers (execution_id, workflow, step), timings, attempts, and error details (when applicable).
|
354
|
+
|
355
|
+
---
|
356
|
+
|
357
|
+
## Generators
|
358
|
+
|
359
|
+
```bash
|
360
|
+
rails g active_workflow:install # initializer + migrations + sample
|
361
|
+
rails g active_workflow:workflow CheckoutFlow
|
362
|
+
```
|
363
|
+
|
364
|
+
The workflow generator scaffolds a class under `app/workflows/`.
|
365
|
+
|
366
|
+
---
|
367
|
+
|
368
|
+
## Testing
|
369
|
+
|
370
|
+
- RSpec helpers (recommended):
|
371
|
+
- Start a flow and assert final state:
|
372
|
+
```ruby
|
373
|
+
flow = MyFlow.start(foo: 1)
|
374
|
+
expect(flow.await(timeout: 10.seconds).state).to eq("completed")
|
375
|
+
```
|
376
|
+
- Simulate async completion:
|
377
|
+
```ruby
|
378
|
+
ActiveWorkflow.complete_step!(flow.id, :export, payload: { url: "..." }, idempotency_key: "evt-1")
|
379
|
+
```
|
380
|
+
- Cover:
|
381
|
+
- retries/backoff behavior,
|
382
|
+
- dedupe (no double execution),
|
383
|
+
- signals,
|
384
|
+
- timeouts (including re-initiation),
|
385
|
+
- compensation reverse order,
|
386
|
+
- idempotent completions,
|
387
|
+
- crash-safety (re-enqueue after failure mid-step).
|
388
|
+
|
389
|
+
---
|
390
|
+
|
391
|
+
## Examples
|
392
|
+
|
393
|
+
**Signal workflow**
|
394
|
+
|
395
|
+
```ruby
|
396
|
+
class SignalWorkflow < ActiveWorkflow::Workflow
|
397
|
+
task :prepare, PrepareTask
|
398
|
+
wait_for_signal :approval, as: :approval
|
399
|
+
task :finalize, FinalizeTask, args: ->(ctx) { [ctx[:approval]] }
|
400
|
+
end
|
401
|
+
```
|
402
|
+
|
403
|
+
**Async export + notify**
|
404
|
+
|
405
|
+
```ruby
|
406
|
+
class ExportFlow < ActiveWorkflow::Workflow
|
407
|
+
task :request_export, ExporterTask, async: true, timeout: 30.minutes
|
408
|
+
task :notify_user do |ctx|
|
409
|
+
Mailers::ExportReady.deliver_later(ctx[:user_id], ctx[:request_export][:download_url])
|
410
|
+
end
|
411
|
+
end
|
412
|
+
|
413
|
+
class ExporterTask < ActiveWorkflow::Task
|
414
|
+
async! timeout: 30.minutes
|
415
|
+
def call(ctx)
|
416
|
+
req = ExportAPI.create!(user_id: ctx[:user_id], request_id: "#{ctx[:execution_id]}:request_export")
|
417
|
+
{ export_job_id: req.id, correlation: req.token }
|
418
|
+
end
|
419
|
+
end
|
420
|
+
```
|
421
|
+
|
422
|
+
Webhook:
|
423
|
+
|
424
|
+
```ruby
|
425
|
+
def export_ready
|
426
|
+
ActiveWorkflow.complete_step!(params[:exec_id], :request_export,
|
427
|
+
payload: { download_url: params[:url] }, idempotency_key: params[:event_id])
|
428
|
+
head :ok
|
429
|
+
end
|
430
|
+
```
|
431
|
+
|
432
|
+
---
|
433
|
+
|
434
|
+
## Design Notes
|
435
|
+
|
436
|
+
- Prefer **Task classes** for reusable/complex steps and DI; use **method** or **block** steps for quick wiring.
|
437
|
+
- Use **business idempotency keys** in external systems (`execution_id:step_name`) for safe re-init.
|
438
|
+
- Keep ctx small (IDs and small payloads). Store large blobs elsewhere and pass references.
|
439
|
+
|
440
|
+
---
|
441
|
+
|
442
|
+
## License
|
443
|
+
|
444
|
+
MIT © You and contributors.
|
@@ -0,0 +1,23 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require "active_support/logger"
|
4
|
+
|
5
|
+
module ActiveWorkflow
|
6
|
+
# Holds gem-level configuration.
|
7
|
+
class Configuration
|
8
|
+
attr_accessor :store, :serializer, :logger, :clock
|
9
|
+
|
10
|
+
def initialize
|
11
|
+
@serializer = ActiveWorkflow::Serializers::Json.new
|
12
|
+
@logger = ActiveSupport::Logger.new($stdout, level: :info)
|
13
|
+
@clock = -> { Time.now.utc }
|
14
|
+
end
|
15
|
+
|
16
|
+
# Ensures store is set before usage.
|
17
|
+
def store!
|
18
|
+
raise ActiveWorkflow::Errors::Configuration, "ActiveWorkflow.store is not configured" unless store
|
19
|
+
|
20
|
+
store
|
21
|
+
end
|
22
|
+
end
|
23
|
+
end
|