ajdc 0.0.1 → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +4 -0
- data/README.md +380 -13
- data/db/durable_schema.rb +57 -0
- data/lib/active_job/continuable/callbacks.rb +48 -0
- data/lib/active_job/continuable/configurable.rb +75 -0
- data/lib/active_job/continuation/callbacks.rb +18 -0
- data/lib/active_job/continuation/rescue_handlers_first.rb +29 -0
- data/lib/active_job/durable/args_mapper.rb +41 -0
- data/lib/active_job/durable/config.rb +100 -0
- data/lib/active_job/durable/continuation.rb +24 -0
- data/lib/active_job/durable/execution.rb +438 -0
- data/lib/active_job/durable/housekeeping_job.rb +16 -0
- data/lib/active_job/durable/record.rb +17 -0
- data/lib/active_job/durable/run.rb +165 -0
- data/lib/active_job/durable/step.rb +14 -0
- data/lib/active_job/durable/wake_job.rb +16 -0
- data/lib/active_job/durable.rb +251 -0
- data/lib/ajdc/railtie.rb +18 -0
- data/lib/ajdc/version.rb +1 -1
- data/lib/ajdc.rb +1 -0
- data/lib/generators/ajdc/install/USAGE +19 -0
- data/lib/generators/ajdc/install/install_generator.rb +66 -0
- data/lib/generators/ajdc/install/templates/create_active_job_durable_tables.rb.tt +56 -0
- data/lib/generators/ajdc/install/templates/initializer.rb.tt +4 -0
- data/skills/ajdc/SKILL.md +117 -0
- data/skills/ajdc/examples/README.md +11 -0
- data/skills/ajdc/examples/agent-loop.md +57 -0
- data/skills/ajdc/examples/halting.md +84 -0
- data/skills/ajdc/examples/pipeline.md +73 -0
- data/skills/ajdc/examples/signals.md +139 -0
- data/skills/ajdc/examples/timers.md +136 -0
- data/skills/ajdc/examples/uniqueness.md +87 -0
- data/skills/ajdc/references/api.md +157 -0
- data/skills/ajdc/references/decision-guide.md +130 -0
- data/skills/ajdc/references/installation.md +98 -0
- data/skills/ajdc/references/testing.md +152 -0
- data/skills/ajdc/references/transactions.md +68 -0
- metadata +83 -7
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 457acb0e1d1bf39fedf55009435838bdb2c571d23f84b2267824305182966dac
|
|
4
|
+
data.tar.gz: 27a48e7abba53b5dff2174a9ecc8e4f76cf545d3fabcfbf1a1e1c1e046e8bcec
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: f2efc59677a4fa9057e0bab2d4f08a224750574ecd001717ced02d414a87c8b04f8424e1747c07eff032acf8b3e6828222ba26e768f9d0a250c287bcf9fe571f
|
|
7
|
+
data.tar.gz: c8a7314a4c38ac7cb0b9d2414b6da85b6eb1963047701d8bc19cd8911fc620709ca1c7d253891de2f45d21d00e83c743af3e6f909df9df620d9a19b18ee76e23
|
data/CHANGELOG.md
CHANGED
data/README.md
CHANGED
|
@@ -1,37 +1,404 @@
|
|
|
1
1
|
[](https://rubygems.org/gems/ajdc)
|
|
2
2
|
[](https://github.com/palkan/ajdc/actions)
|
|
3
3
|
|
|
4
|
-
#
|
|
4
|
+
# AJ/DC: Active Job Durable Continuation
|
|
5
5
|
|
|
6
|
-
|
|
6
|
+
<img align="right" height="150" width="292"
|
|
7
|
+
title="AJ/DC logo" src="./assets/logo.png">
|
|
8
|
+
|
|
9
|
+
AJ/DC makes an `ActiveJob::Continuable` job's run and steps durable: they're recorded in the database, not only in the job payload, so progress survives a crash, not only a graceful restart.
|
|
7
10
|
|
|
8
11
|
## Installation
|
|
9
12
|
|
|
10
|
-
|
|
13
|
+
Add to your project's Gemfile:
|
|
11
14
|
|
|
12
15
|
```ruby
|
|
13
|
-
#
|
|
14
|
-
|
|
16
|
+
# Gemfile
|
|
17
|
+
gem "ajdc"
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
or run `bundle add ajdc`.
|
|
21
|
+
|
|
22
|
+
### Addinng agent skills
|
|
23
|
+
|
|
24
|
+
The gem ships a skill for coding agents in `skills/ajdc`. It follows the [skills.sh](https://www.skills.sh) layout, so it works with any agent that reads `SKILL.md`.
|
|
25
|
+
|
|
26
|
+
With [Rails Hyperdrive](https://github.com/rails-hyperdrive/rails-hyperdrive) it installs itself: `bin/rails hyperdrive:init` the first time, then `bundle add ajdc` or `bin/rails hyperdrive:sync` lands it in `.claude/skills/ajdc`.
|
|
27
|
+
|
|
28
|
+
Without it, install straight from the repository with the [skills.sh](https://www.skills.sh) CLI, or copy `skills/ajdc` into your agent's skills directory:
|
|
29
|
+
|
|
30
|
+
```sh
|
|
31
|
+
npx skills add palkan/ajdc
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
Now, you can ask your agent to continue the installation or do it yourself.
|
|
35
|
+
|
|
36
|
+
### Preparing the database
|
|
37
|
+
|
|
38
|
+
AJ/DC keeps its data in two tables (`active_job_durable_runs`, `active_job_durable_steps`). Generate the migration and run it:
|
|
39
|
+
|
|
40
|
+
```sh
|
|
41
|
+
bin/rails generate ajdc:install
|
|
42
|
+
bin/rails db:migrate
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
To keep the tables in a separate database (the Solid Queue way), declare it in `config/database.yml` with its own `migrations_paths`, then pass its name; the migration lands in that path and an initializer points AJ/DC at the database:
|
|
46
|
+
|
|
47
|
+
```sh
|
|
48
|
+
bin/rails generate ajdc:install --database=durable
|
|
49
|
+
bin/rails db:prepare
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
```ruby
|
|
53
|
+
# config/initializers/active_job_durable.rb (generated)
|
|
54
|
+
Rails.application.config.active_job_durable.connects_to = { database: { writing: :durable } }
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
### Requirements
|
|
58
|
+
|
|
59
|
+
- Ruby (MRI) >= 3.3
|
|
60
|
+
- Rails >= 8.1 (`ActiveJob::Attributes` needs Rails 8.2)
|
|
61
|
+
- SQLite / PostgreSQL / MySQL
|
|
62
|
+
|
|
63
|
+
## Usage
|
|
64
|
+
|
|
65
|
+
Include `ActiveJob::Durable` instead of `ActiveJob::Continuable`. `step`, cursors and `attribute` keep working exactly as they do today:
|
|
66
|
+
|
|
67
|
+
```ruby
|
|
68
|
+
# before
|
|
69
|
+
class ImportJob < ApplicationJob
|
|
70
|
+
include ActiveJob::Continuable
|
|
71
|
+
|
|
72
|
+
attribute :processed_count, :integer, default: 0
|
|
73
|
+
|
|
74
|
+
def perform(import)
|
|
75
|
+
step :validate
|
|
76
|
+
step :process do |step|
|
|
77
|
+
import.records.find_each(start: step.cursor) do |record|
|
|
78
|
+
record.process!
|
|
79
|
+
self.processed_count += 1
|
|
80
|
+
step.advance! from: record.id
|
|
81
|
+
end
|
|
82
|
+
end
|
|
83
|
+
end
|
|
84
|
+
end
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
```ruby
|
|
88
|
+
# after
|
|
89
|
+
class ImportJob < ApplicationJob
|
|
90
|
+
include ActiveJob::Durable
|
|
91
|
+
|
|
92
|
+
attribute :processed_count, :integer, default: 0
|
|
93
|
+
|
|
94
|
+
def perform(import)
|
|
95
|
+
step :validate
|
|
96
|
+
step :process do |step|
|
|
97
|
+
import.records.find_each(start: step.cursor) do |record|
|
|
98
|
+
record.process!
|
|
99
|
+
self.processed_count += 1
|
|
100
|
+
step.advance! from: record.id
|
|
101
|
+
end
|
|
102
|
+
end
|
|
103
|
+
end
|
|
104
|
+
end
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
Every checkpoint commits the run and the current step's cursor to the database, so a `SIGKILL` loses at most the work since the last checkpoint.
|
|
108
|
+
|
|
109
|
+
The run and its steps are plain Active Record models:
|
|
110
|
+
|
|
111
|
+
```ruby
|
|
112
|
+
run = ActiveJob::Durable::Run.last
|
|
113
|
+
run.status # => "completed"
|
|
114
|
+
run.current_step # => nil
|
|
115
|
+
run.completed_steps # => ["validate", "process"]
|
|
116
|
+
run.state # => {"processed_count" => 128}
|
|
117
|
+
run.steps.map(&:name) # => ["validate", "process"]
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
### Uniqueness
|
|
121
|
+
|
|
122
|
+
Durable state allows you to enforce wofkflow runs uniqueness. For that, use the `unique_by` macro in the workflow and specify the `perform` parameters that identify a run. Uniqueness is only enforced for runs that hasn't been terminated: either _live_ runs (with "enqueued", "running", "waiting", "awaiting" status) or _paused_ runs ("failed" or "halted"). Here is an example:
|
|
123
|
+
|
|
124
|
+
```ruby
|
|
125
|
+
class ImportJob < ApplicationJob
|
|
126
|
+
include ActiveJob::Durable
|
|
127
|
+
|
|
128
|
+
unique_by :import
|
|
129
|
+
|
|
130
|
+
def perform(import)
|
|
131
|
+
step :check
|
|
132
|
+
step :process
|
|
133
|
+
end
|
|
134
|
+
end
|
|
135
|
+
|
|
136
|
+
ImportJob.perform_later(import) # => the job
|
|
137
|
+
ImportJob.perform_later(import) # => false, nothing enqueued
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
You can provide and option `on_conflict` parameter to specify what to do in case of the uniqueness conflict:
|
|
141
|
+
|
|
142
|
+
- `:skip` (default) enqueues nothing; `perform_later` returns `false`.
|
|
143
|
+
- `:reject` raises `ActiveJob::Durable::RunAlreadyExists` (the currnent run is available as `error.run`).
|
|
144
|
+
- `:replace` cancels the current run and starts the new one. A renewal or a reschedule is `perform_later` again:
|
|
145
|
+
|
|
146
|
+
```ruby
|
|
147
|
+
class License::LifecycleJob < ApplicationJob
|
|
148
|
+
include ActiveJob::Durable
|
|
149
|
+
|
|
150
|
+
unique_by :license, on_conflict: :replace
|
|
15
151
|
# ...
|
|
16
|
-
|
|
152
|
+
end
|
|
153
|
+
```
|
|
154
|
+
|
|
155
|
+
### Reading runs
|
|
156
|
+
|
|
157
|
+
A job class knows its runs, newest first. `workflow_runs` is an Active Record relation, so the scopes below chain onto it:
|
|
158
|
+
|
|
159
|
+
```ruby
|
|
160
|
+
ImportJob.workflow_runs
|
|
161
|
+
ImportJob.workflow_runs.failed.at_step(:process)
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
`for` finds the runs `perform_later` would have created with the same arguments; it derives the key the same way. `for(workflow_key:)` matches a key verbatim:
|
|
165
|
+
|
|
166
|
+
```ruby
|
|
167
|
+
ImportJob.workflow_runs.for(import) # same key as ImportJob.perform_later(import)
|
|
168
|
+
ImportJob.workflow_runs.for(workflow_key: "imports/42")
|
|
169
|
+
ImportJob.workflow_runs.for(import).live.first # the run in progress, or nil
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
Without a declaration, every argument is part of the key: positional arguments in order, then keywords sorted by name as `name=value`. A record renders as `collection/id`; a value that is not a string, symbol, number or boolean becomes a short digest:
|
|
173
|
+
|
|
174
|
+
```ruby
|
|
175
|
+
ImportJob.perform_later(import, "csv", strict: true) # key "imports/42:csv:strict=true"
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
`identified_by` narrows the key to the named `perform` parameters, positional or keyword:
|
|
179
|
+
|
|
180
|
+
```ruby
|
|
181
|
+
class ImportJob < ApplicationJob
|
|
182
|
+
include ActiveJob::Durable
|
|
183
|
+
|
|
184
|
+
identified_by :import # key "imports/42", whatever the other arguments
|
|
185
|
+
|
|
186
|
+
def perform(import, format = "csv", strict: false)
|
|
187
|
+
# ...
|
|
188
|
+
end
|
|
189
|
+
end
|
|
190
|
+
```
|
|
191
|
+
|
|
192
|
+
The block form receives the `perform` arguments and returns one component or an array of them:
|
|
193
|
+
|
|
194
|
+
```ruby
|
|
195
|
+
identified_by { |import, **kwargs| [import, kwargs.fetch(:format, "csv")] } # key "imports/42:csv"
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
A name that is not a `perform` parameter raises `ArgumentError`: at the declaration when the class already defines `perform`, otherwise at the first `perform_later`.
|
|
199
|
+
|
|
200
|
+
`set(workflow_key:)` names one run's key verbatim, whatever the class derives. It is an enqueue option like `wait:` or `queue:`, so it works with `perform_later`, `perform_now` and `perform_all_later`:
|
|
201
|
+
|
|
202
|
+
```ruby
|
|
203
|
+
ImportJob.set(workflow_key: "imports/42/retry-3", queue: "low").perform_later(import)
|
|
204
|
+
ImportJob.workflow_runs.for(workflow_key: "imports/42/retry-3")
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
Other usefule scopes:
|
|
208
|
+
|
|
209
|
+
```ruby
|
|
210
|
+
MyJob.workflow_runs.live # enqueued, running, waiting, awaiting
|
|
211
|
+
MyJob.workflow_runs.at_step(:process)
|
|
212
|
+
MyJob.workflow_runs.stuck_for(1.hour)
|
|
213
|
+
MyJob.workflow_runs.newest_first
|
|
214
|
+
```
|
|
215
|
+
|
|
216
|
+
A run's `status` says where the job is. `enqueued`: the job is in the queue, written at the first enqueue and at every re-enqueue (an isolated step, a graceful-stop interrupt, a resume after an error, a `retry_on` retry). `running`: a worker is executing it. `started_at` is set once, at the first execution, so an `enqueued` run with no `started_at` never ran. `stuck_for` reads `enqueued` runs by `transitioned_at` and `running` runs by `last_heartbeat_at`.
|
|
217
|
+
|
|
218
|
+
### Halting
|
|
219
|
+
|
|
220
|
+
Halting allows you to pause the execution on an error that could be resolved by a human (or alike), so the run could be restarted later from the current step/cursor (e.g., a plan out of storage, a file to fix by hand). Use `halt_on` or `halt!` to stop the run but keep it resumeable (unlike `discard_on`):
|
|
221
|
+
|
|
222
|
+
```ruby
|
|
223
|
+
class ImportJob < ApplicationJob
|
|
224
|
+
include ActiveJob::Durable
|
|
225
|
+
|
|
226
|
+
discard_on ZipFile::InvalidFileError
|
|
227
|
+
halt_on InsufficientStorageSpaceError
|
|
228
|
+
|
|
229
|
+
def perform(import)
|
|
230
|
+
step :check
|
|
231
|
+
step :process
|
|
232
|
+
end
|
|
233
|
+
end
|
|
234
|
+
```
|
|
235
|
+
|
|
236
|
+
The run's status is `halted`; `error_class` and `error_message` hold the error, `current_step` names the step and its row keeps the cursor.
|
|
237
|
+
|
|
238
|
+
`halt!(reason)` does the same from inside a step, without an error; the reason lands in `halt_reason`:
|
|
239
|
+
|
|
240
|
+
```ruby
|
|
241
|
+
step :run do |step|
|
|
242
|
+
until chat.complete?
|
|
243
|
+
chat.step
|
|
244
|
+
halt!(:tool_approval) if chat.awaiting_approval?
|
|
245
|
+
step.checkpoint!
|
|
246
|
+
end
|
|
247
|
+
end
|
|
248
|
+
```
|
|
249
|
+
|
|
250
|
+
```ruby
|
|
251
|
+
ImportJob.workflow_runs.halted.first.halt_reason # => "tool_approval"
|
|
252
|
+
```
|
|
253
|
+
|
|
254
|
+
A halted (or failed) run is resumed in place: `resume!` puts the job back in the queue, the step that stopped re-runs from its cursor as a new attempt, and the earlier attempts keep their errors. Any other status raises `ActiveJob::Durable::NotResumable`.
|
|
255
|
+
|
|
256
|
+
```ruby
|
|
257
|
+
ImportJob.workflow_runs.halted.first.resume!
|
|
258
|
+
```
|
|
259
|
+
|
|
260
|
+
### Cancelling
|
|
261
|
+
|
|
262
|
+
`cancel!` ends a run that is not terminal: the row becomes `cancelled` and the queue is left alone. A queued job for a cancelled run performs nothing; a running job stops at its next checkpoint or step boundary, and the open step row is `cancelled` with its cursor. A terminal run raises `ActiveJob::Durable::NotCancellable`.
|
|
263
|
+
|
|
264
|
+
```ruby
|
|
265
|
+
ImportJob.workflow_runs.for(import).live.sole.cancel!
|
|
266
|
+
```
|
|
267
|
+
|
|
268
|
+
### Step callbacks
|
|
269
|
+
|
|
270
|
+
`before_step`, `after_step` and `around_step` are Active Job callbacks, like `before_perform` and friends, for every step that runs; a step skipped on resume triggers none. `after_step` runs only when the step completes. `current_step` is the running `ActiveJob::Continuation::Step`:
|
|
271
|
+
|
|
272
|
+
```ruby
|
|
273
|
+
class Cable::DiagnosticJob < ApplicationJob
|
|
274
|
+
include ActiveJob::Durable
|
|
275
|
+
|
|
276
|
+
after_step :broadcast_update
|
|
277
|
+
around_step { |job, block| Rails.logger.tagged(job.current_step.name, &block) }
|
|
278
|
+
|
|
279
|
+
def perform(cable)
|
|
280
|
+
@cable = cable
|
|
281
|
+
step :provider_status, isolated: true
|
|
282
|
+
step :websocket_status, isolated: true
|
|
283
|
+
step :admin_api_status, isolated: true
|
|
284
|
+
end
|
|
285
|
+
|
|
286
|
+
private
|
|
287
|
+
def broadcast_update
|
|
288
|
+
@cable.broadcast_replace(partial: "cables/diagnostic", locals: { step: current_step.name })
|
|
289
|
+
end
|
|
290
|
+
end
|
|
291
|
+
```
|
|
292
|
+
|
|
293
|
+
### Timers
|
|
294
|
+
|
|
295
|
+
A workflow run can _go to sleep_ (=pause) and _wake up_ (=resume) eaither at specific time (`wait_until`) or after a given time interval passed (`wait`). Example:
|
|
296
|
+
|
|
297
|
+
```ruby
|
|
298
|
+
class License::LifecycleJob < ApplicationJob
|
|
299
|
+
include ActiveJob::Durable
|
|
300
|
+
|
|
301
|
+
unique_by :license, on_conflict: :replace
|
|
302
|
+
|
|
303
|
+
def perform(license)
|
|
304
|
+
step :remind, wait_until: license.expires_at - 2.weeks
|
|
305
|
+
step :expire, wait_until: license.expires_at
|
|
306
|
+
step :revoke, wait: 2.weeks
|
|
307
|
+
end
|
|
308
|
+
|
|
17
309
|
# ...
|
|
18
310
|
end
|
|
19
311
|
```
|
|
20
312
|
|
|
21
|
-
|
|
313
|
+
When the workflow reaches the waiting step, it's status is changed to `waiting`, and it's no longer present in the job queue. To wake sleeping workflows up, you need to run a single recurring job, `ActiveJob::Durable::WakeJob`. For example, with Solid Queue:
|
|
314
|
+
|
|
315
|
+
```yaml
|
|
316
|
+
# config/recurring.yml
|
|
317
|
+
durable_wake:
|
|
318
|
+
class: ActiveJob::Durable::WakeJob
|
|
319
|
+
schedule: every minute
|
|
320
|
+
```
|
|
321
|
+
|
|
322
|
+
The job is one call to `ActiveJob::Durable.wake_up_due`, which wakes every due run once and returns how many. In tests, call it yourself inside `travel_to`:
|
|
22
323
|
|
|
23
324
|
```ruby
|
|
24
|
-
|
|
25
|
-
|
|
325
|
+
travel_to license.expires_at - 2.weeks do
|
|
326
|
+
perform_enqueued_jobs { ActiveJob::Durable.wake_up_due }
|
|
327
|
+
end
|
|
26
328
|
```
|
|
27
329
|
|
|
28
|
-
|
|
330
|
+
You can also wake up a single workflow manually by using the `#wake_up` method:
|
|
29
331
|
|
|
30
|
-
|
|
332
|
+
```ruby
|
|
333
|
+
License::LifecycleJob.workflow_runs.for(license).waiting.sole.wake_up # send the reminder now
|
|
334
|
+
```
|
|
31
335
|
|
|
32
|
-
|
|
336
|
+
### Signals
|
|
337
|
+
|
|
338
|
+
The durable job can sleep indifinitely waiting for an external signal to wake it up. For that, you can use the `#await` method. It defines a step with no body but a _signal handler_ instead:
|
|
339
|
+
|
|
340
|
+
```ruby
|
|
341
|
+
class BulkImportJob < ApplicationJob
|
|
342
|
+
include ActiveJob::Durable
|
|
343
|
+
|
|
344
|
+
attribute :confirmed, :boolean, default: false
|
|
345
|
+
|
|
346
|
+
def perform(import)
|
|
347
|
+
await :confirmation, wait: 10.minutes
|
|
348
|
+
return import.destroy! unless confirmed
|
|
349
|
+
|
|
350
|
+
step :apply do
|
|
351
|
+
BulkImportService.new.call(import)
|
|
352
|
+
end
|
|
353
|
+
end
|
|
354
|
+
|
|
355
|
+
private
|
|
356
|
+
def confirmation(signal) = self.confirmed = signal.presence
|
|
357
|
+
end
|
|
358
|
+
```
|
|
359
|
+
|
|
360
|
+
Then, you can use the `#wake_up` method to send a signal to the job:
|
|
361
|
+
|
|
362
|
+
```ruby
|
|
363
|
+
# from the controller
|
|
364
|
+
BulkImportJob.workflow_runs.for(import).live.sole.wake_up(:confirmation, true)
|
|
365
|
+
```
|
|
366
|
+
|
|
367
|
+
A signal sent before the `await` line is stored in the durable state and is replayed as soon as the job reaches this step. So, you can have multiple awaiting steps receiving signals in any order. A second signal for the same name overwrites the first.
|
|
368
|
+
|
|
369
|
+
If no signal received and the deadline is specified (`wait:`, `wait_until:`), the signal handler is called with the `nil` signal, so you can decide on how to continue.
|
|
370
|
+
|
|
371
|
+
You can find all the jobs waiting for a particular signal using the corresponding scope:
|
|
372
|
+
|
|
373
|
+
```ruby
|
|
374
|
+
CardGenerationJob.workflow_runs.awaiting.at_step(:review)
|
|
375
|
+
```
|
|
376
|
+
|
|
377
|
+
### Housekeeping
|
|
378
|
+
|
|
379
|
+
Terminal runs (`completed`, `discarded`, `cancelled`) and their steps are kept for 14 days after they end. To delete the older ones, schedule `ActiveJob::Durable::HousekeepingJob`, e.g., with Solid Queue:
|
|
380
|
+
|
|
381
|
+
```yaml
|
|
382
|
+
# config/recurring.yml
|
|
383
|
+
durable_housekeeping:
|
|
384
|
+
class: ActiveJob::Durable::HousekeepingJob
|
|
385
|
+
schedule: every hour
|
|
386
|
+
```
|
|
387
|
+
|
|
388
|
+
Configure the retention period (`nil` keeps runs forever):
|
|
389
|
+
|
|
390
|
+
```ruby
|
|
391
|
+
# config/application.rb
|
|
392
|
+
config.active_job_durable.keep_terminal_runs_for = 30.days
|
|
393
|
+
```
|
|
394
|
+
|
|
395
|
+
Runs that need attention (`failed`, `halted`) are never deleted. Stuck runs are left to the application: `stuck_for` finds `enqueued` runs whose message never arrived, `running` runs whose worker died (no heartbeat), and parked runs the clock missed, and `cancel!` ends any of them:
|
|
396
|
+
|
|
397
|
+
```ruby
|
|
398
|
+
ActiveJob::Durable::Run.stuck_for(1.hour).running.find_each(&:cancel!)
|
|
399
|
+
```
|
|
33
400
|
|
|
34
|
-
|
|
401
|
+
A heartbeat is written at every step boundary and checkpoint, so pick a duration longer than the longest stretch of a step without a checkpoint.
|
|
35
402
|
|
|
36
403
|
## Contributing
|
|
37
404
|
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
# Tables for ActiveJob::Durable: one row per run, one row per step attempt.
|
|
4
|
+
#
|
|
5
|
+
# JSON columns hold Active Job-serialized values (ActiveJob::Arguments.serialize),
|
|
6
|
+
# nothing in them is indexed. They carry no database defaults so that the same
|
|
7
|
+
# file loads on SQLite, PostgreSQL and MySQL; the models set the defaults.
|
|
8
|
+
ActiveRecord::Schema[8.1].define(version: 1) do
|
|
9
|
+
create_table "active_job_durable_runs", if_not_exists: true do |t|
|
|
10
|
+
t.string "job_class", null: false
|
|
11
|
+
t.string "key", null: false
|
|
12
|
+
t.string "active_key"
|
|
13
|
+
t.string "active_job_id", null: false
|
|
14
|
+
t.json "arguments", null: false
|
|
15
|
+
t.string "status", null: false
|
|
16
|
+
t.string "current_step"
|
|
17
|
+
t.json "completed_steps", null: false
|
|
18
|
+
t.json "state", null: false
|
|
19
|
+
t.datetime "wake_at"
|
|
20
|
+
t.json "pending_signals", null: false
|
|
21
|
+
t.json "parked_job"
|
|
22
|
+
t.integer "resumptions", default: 0, null: false
|
|
23
|
+
t.datetime "last_heartbeat_at"
|
|
24
|
+
t.string "error_class"
|
|
25
|
+
t.text "error_message"
|
|
26
|
+
t.string "halt_reason"
|
|
27
|
+
t.datetime "started_at"
|
|
28
|
+
t.datetime "finished_at"
|
|
29
|
+
t.datetime "transitioned_at"
|
|
30
|
+
t.timestamps
|
|
31
|
+
|
|
32
|
+
t.index ["job_class", "active_key"], name: "index_active_job_durable_runs_on_active_key", unique: true
|
|
33
|
+
t.index ["job_class", "key"], name: "index_active_job_durable_runs_on_key"
|
|
34
|
+
t.index ["active_job_id"], name: "index_active_job_durable_runs_on_active_job_id", unique: true
|
|
35
|
+
t.index ["status"], name: "index_active_job_durable_runs_on_status"
|
|
36
|
+
t.index ["status", "wake_at"], name: "index_active_job_durable_runs_on_status_and_wake_at"
|
|
37
|
+
t.index ["status", "transitioned_at"], name: "index_active_job_durable_runs_on_status_and_transitioned_at"
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
create_table "active_job_durable_steps", if_not_exists: true do |t|
|
|
41
|
+
t.references "run", null: false, index: false,
|
|
42
|
+
foreign_key: {to_table: "active_job_durable_runs", on_delete: :cascade}
|
|
43
|
+
t.string "name", null: false
|
|
44
|
+
t.integer "position", null: false
|
|
45
|
+
t.integer "attempt", default: 1, null: false
|
|
46
|
+
t.string "status", null: false
|
|
47
|
+
t.json "cursor"
|
|
48
|
+
t.boolean "isolated", default: false, null: false
|
|
49
|
+
t.string "error_class"
|
|
50
|
+
t.text "error_message"
|
|
51
|
+
t.datetime "started_at"
|
|
52
|
+
t.datetime "finished_at"
|
|
53
|
+
|
|
54
|
+
t.index ["run_id", "name", "attempt"], name: "index_active_job_durable_steps_on_attempt", unique: true
|
|
55
|
+
t.index ["run_id", "position"], name: "index_active_job_durable_steps_on_position"
|
|
56
|
+
end
|
|
57
|
+
end
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "active_job/continuable/configurable"
|
|
4
|
+
require "active_job/continuation/callbacks"
|
|
5
|
+
|
|
6
|
+
module ActiveJob
|
|
7
|
+
module Continuable
|
|
8
|
+
# = Step callbacks
|
|
9
|
+
#
|
|
10
|
+
# Callbacks around every step that runs (a step skipped on resume runs none),
|
|
11
|
+
# with the semantics of `before_perform` and friends; the step is `current_step`.
|
|
12
|
+
# `after_step` runs only when the step completes.
|
|
13
|
+
#
|
|
14
|
+
# class ProcessImportJob < ApplicationJob
|
|
15
|
+
# include ActiveJob::Continuable::Callbacks
|
|
16
|
+
#
|
|
17
|
+
# around_step :measure
|
|
18
|
+
# after_step { logger.info "#{current_step.name} done" }
|
|
19
|
+
# end
|
|
20
|
+
module Callbacks
|
|
21
|
+
extend ActiveSupport::Concern
|
|
22
|
+
include Configurable
|
|
23
|
+
|
|
24
|
+
included do
|
|
25
|
+
define_callbacks :step, skip_after_callbacks_if_terminated: true
|
|
26
|
+
|
|
27
|
+
unless continuation_class <= Continuation::Callbacks
|
|
28
|
+
self.continuation_class = Class.new(continuation_class) do
|
|
29
|
+
prepend Continuation::Callbacks
|
|
30
|
+
|
|
31
|
+
set_temporary_name "#{superclass.name}(WithCallbacks)"
|
|
32
|
+
end
|
|
33
|
+
end
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
class_methods do
|
|
37
|
+
def before_step(*filters, &blk) = set_callback(:step, :before, *filters, &blk)
|
|
38
|
+
|
|
39
|
+
def after_step(*filters, &blk) = set_callback(:step, :after, *filters, &blk)
|
|
40
|
+
|
|
41
|
+
def around_step(*filters, &blk) = set_callback(:step, :around, *filters, &blk)
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
# The `ActiveJob::Continuation::Step` that is running, `nil` between steps.
|
|
45
|
+
def current_step = continuation.running_step
|
|
46
|
+
end
|
|
47
|
+
end
|
|
48
|
+
end
|
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "active_job/continuable"
|
|
4
|
+
|
|
5
|
+
module ActiveJob
|
|
6
|
+
module Continuable
|
|
7
|
+
# = Configurable continuation
|
|
8
|
+
#
|
|
9
|
+
# Continuable with a pluggable continuation class, which receives any extra
|
|
10
|
+
# `step` options along with `start:` and `isolated:`:
|
|
11
|
+
#
|
|
12
|
+
# class TimedJob < ApplicationJob
|
|
13
|
+
# include ActiveJob::Continuable::Configurable
|
|
14
|
+
#
|
|
15
|
+
# self.continuation_class = TimedContinuation
|
|
16
|
+
#
|
|
17
|
+
# def perform
|
|
18
|
+
# step :remind, wait: 2.weeks # TimedContinuation#run_step(:remind, start: nil, isolated: false, wait: 2.weeks)
|
|
19
|
+
# end
|
|
20
|
+
# end
|
|
21
|
+
module Configurable
|
|
22
|
+
extend ActiveSupport::Concern
|
|
23
|
+
include ActiveJob::Continuable
|
|
24
|
+
|
|
25
|
+
# Continuable defines its constructor on the including class, so this one
|
|
26
|
+
# is prepended to the class to run after it.
|
|
27
|
+
module Initializer # :nodoc:
|
|
28
|
+
def initialize(...)
|
|
29
|
+
super
|
|
30
|
+
self.continuation = continuation_class.new(self, {})
|
|
31
|
+
end
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
included do
|
|
35
|
+
class_attribute :continuation_class, instance_writer: false, default: ActiveJob::Continuation
|
|
36
|
+
|
|
37
|
+
prepend Initializer
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
def step(step_name, start: nil, isolated: false, **options, &block)
|
|
41
|
+
block ||= step_method_block(step_name)
|
|
42
|
+
checkpoint! if continuation.advanced?
|
|
43
|
+
continuation.step(step_name, start:, isolated:, **options, &block)
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
if Continuable.private_method_defined?(:continuation_serialized?) # Rails 8.2+
|
|
47
|
+
private def deserialize_arguments_if_needed
|
|
48
|
+
serialized, @serialized_continuation = @serialized_continuation, nil
|
|
49
|
+
super
|
|
50
|
+
self.continuation = continuation_class.new(self, serialized) if serialized
|
|
51
|
+
end
|
|
52
|
+
else
|
|
53
|
+
def deserialize(job_data) # :nodoc:
|
|
54
|
+
super
|
|
55
|
+
self.continuation = continuation_class.new(self, job_data.fetch("continuation", {}))
|
|
56
|
+
end
|
|
57
|
+
end
|
|
58
|
+
|
|
59
|
+
private
|
|
60
|
+
|
|
61
|
+
# The method named after the step, as a step block.
|
|
62
|
+
def step_method_block(step_name)
|
|
63
|
+
step_method = method(step_name)
|
|
64
|
+
|
|
65
|
+
raise ArgumentError, "Step method '#{step_name}' must accept 0 or 1 arguments" if step_method.arity > 1
|
|
66
|
+
|
|
67
|
+
if step_method.parameters.any? { |type, _name| type == :key || type == :keyreq }
|
|
68
|
+
raise ArgumentError, "Step method '#{step_name}' must not accept keyword arguments"
|
|
69
|
+
end
|
|
70
|
+
|
|
71
|
+
(step_method.arity == 0) ? ->(_step) { step_method.call } : step_method
|
|
72
|
+
end
|
|
73
|
+
end
|
|
74
|
+
end
|
|
75
|
+
end
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "active_job/continuation"
|
|
4
|
+
|
|
5
|
+
module ActiveJob
|
|
6
|
+
class Continuation
|
|
7
|
+
# Runs the job's step callbacks around each step's body. Prepended to the
|
|
8
|
+
# job's `continuation_class` by `ActiveJob::Continuable::Callbacks`.
|
|
9
|
+
module Callbacks
|
|
10
|
+
# The step that is running, `nil` between steps.
|
|
11
|
+
def running_step = (current if running_step?)
|
|
12
|
+
|
|
13
|
+
private
|
|
14
|
+
|
|
15
|
+
def instrumenting_step(step, &block) = super { job.run_callbacks(:step, &block) }
|
|
16
|
+
end
|
|
17
|
+
end
|
|
18
|
+
end
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "active_job/continuation"
|
|
4
|
+
|
|
5
|
+
module ActiveJob
|
|
6
|
+
class Continuation
|
|
7
|
+
# = Rescue handlers first
|
|
8
|
+
#
|
|
9
|
+
# Continuable's `continue` rescues any `StandardError` raised after the job made
|
|
10
|
+
# progress and resumes the job, so `discard_on`, `retry_on` and `rescue_from`
|
|
11
|
+
# never see it. Prepended to `ActiveJob::Continuable`, this module gives the
|
|
12
|
+
# job's own handlers precedence: when a handler exists for the error,
|
|
13
|
+
# `resume_job` re-raises it, and `perform_now` runs the handler as it would
|
|
14
|
+
# for any other job. Errors without a handler are resumed as before, and a
|
|
15
|
+
# `Continuation::Interrupt` is never an error.
|
|
16
|
+
module RescueHandlersFirst
|
|
17
|
+
private
|
|
18
|
+
|
|
19
|
+
# Rails 8.1 passes the error-resume exception as `{exception: e}`; main
|
|
20
|
+
# passes it positionally. The argument goes to `super` unchanged.
|
|
21
|
+
def resume_job(exception)
|
|
22
|
+
error = exception.is_a?(Hash) ? exception[:exception] : exception
|
|
23
|
+
raise error if !error.is_a?(Continuation::Interrupt) && handler_for_rescue(error)
|
|
24
|
+
|
|
25
|
+
super
|
|
26
|
+
end
|
|
27
|
+
end
|
|
28
|
+
end
|
|
29
|
+
end
|