ajdc 0.0.1 → 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (39) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +4 -0
  3. data/README.md +380 -13
  4. data/db/durable_schema.rb +57 -0
  5. data/lib/active_job/continuable/callbacks.rb +48 -0
  6. data/lib/active_job/continuable/configurable.rb +75 -0
  7. data/lib/active_job/continuation/callbacks.rb +18 -0
  8. data/lib/active_job/continuation/rescue_handlers_first.rb +29 -0
  9. data/lib/active_job/durable/args_mapper.rb +41 -0
  10. data/lib/active_job/durable/config.rb +100 -0
  11. data/lib/active_job/durable/continuation.rb +24 -0
  12. data/lib/active_job/durable/execution.rb +438 -0
  13. data/lib/active_job/durable/housekeeping_job.rb +16 -0
  14. data/lib/active_job/durable/record.rb +17 -0
  15. data/lib/active_job/durable/run.rb +165 -0
  16. data/lib/active_job/durable/step.rb +14 -0
  17. data/lib/active_job/durable/wake_job.rb +16 -0
  18. data/lib/active_job/durable.rb +251 -0
  19. data/lib/ajdc/railtie.rb +18 -0
  20. data/lib/ajdc/version.rb +1 -1
  21. data/lib/ajdc.rb +1 -0
  22. data/lib/generators/ajdc/install/USAGE +19 -0
  23. data/lib/generators/ajdc/install/install_generator.rb +66 -0
  24. data/lib/generators/ajdc/install/templates/create_active_job_durable_tables.rb.tt +56 -0
  25. data/lib/generators/ajdc/install/templates/initializer.rb.tt +4 -0
  26. data/skills/ajdc/SKILL.md +117 -0
  27. data/skills/ajdc/examples/README.md +11 -0
  28. data/skills/ajdc/examples/agent-loop.md +57 -0
  29. data/skills/ajdc/examples/halting.md +84 -0
  30. data/skills/ajdc/examples/pipeline.md +73 -0
  31. data/skills/ajdc/examples/signals.md +139 -0
  32. data/skills/ajdc/examples/timers.md +136 -0
  33. data/skills/ajdc/examples/uniqueness.md +87 -0
  34. data/skills/ajdc/references/api.md +157 -0
  35. data/skills/ajdc/references/decision-guide.md +130 -0
  36. data/skills/ajdc/references/installation.md +98 -0
  37. data/skills/ajdc/references/testing.md +152 -0
  38. data/skills/ajdc/references/transactions.md +68 -0
  39. metadata +83 -7
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: d574c616f461f12c106a6aae83b1e4d661f0ef2379c37683e6568cc20fde3bfb
4
- data.tar.gz: 84dbb9b64741a0eab088b90cd25ec2ceb81ef0677ec28a7d978f76d2924e029d
3
+ metadata.gz: 457acb0e1d1bf39fedf55009435838bdb2c571d23f84b2267824305182966dac
4
+ data.tar.gz: 27a48e7abba53b5dff2174a9ecc8e4f76cf545d3fabcfbf1a1e1c1e046e8bcec
5
5
  SHA512:
6
- metadata.gz: f945272b9c66c101d06a62fb0c422129ed9e96fa7d764ca1b54d638f0cedc6f498181fddedd225312831d2d618734c85c534708eaa35e111cac8d39c26c39148
7
- data.tar.gz: 96692790799a6196a3d4cadda92ed737d77655236cad1aef4f5a4134bc3fe65e224bf8b64abec5c1c224bb6ca9806f07b95e360ff229cea03b9306f9708982d1
6
+ metadata.gz: f2efc59677a4fa9057e0bab2d4f08a224750574ecd001717ced02d414a87c8b04f8424e1747c07eff032acf8b3e6828222ba26e768f9d0a250c287bcf9fe571f
7
+ data.tar.gz: c8a7314a4c38ac7cb0b9d2414b6da85b6eb1963047701d8bc19cd8911fc620709ca1c7d253891de2f45d21d00e83c743af3e6f909df9df620d9a19b18ee76e23
data/CHANGELOG.md CHANGED
@@ -1,3 +1,7 @@
1
1
  # Change log
2
2
 
3
3
  ## master
4
+
5
+ ## 0.1.0
6
+
7
+ - Initial _Rails World_ release.
data/README.md CHANGED
@@ -1,37 +1,404 @@
1
1
  [![Gem Version](https://badge.fury.io/rb/ajdc.svg)](https://rubygems.org/gems/ajdc)
2
2
  [![Build](https://github.com/palkan/ajdc/workflows/Build/badge.svg)](https://github.com/palkan/ajdc/actions)
3
3
 
4
- # AJDC: Active Job Durable Continuation
4
+ # AJ/DC: Active Job Durable Continuation
5
5
 
6
- TBD
6
+ <img align="right" height="150" width="292"
7
+ title="AJ/DC logo" src="./assets/logo.png">
8
+
9
+ AJ/DC makes an `ActiveJob::Continuable` job's run and steps durable: they're recorded in the database, not only in the job payload, so progress survives a crash, not only a graceful restart.
7
10
 
8
11
  ## Installation
9
12
 
10
- Adding to a gem:
13
+ Add to your project's Gemfile:
11
14
 
12
15
  ```ruby
13
- # my-cool-gem.gemspec
14
- Gem::Specification.new do |spec|
16
+ # Gemfile
17
+ gem "ajdc"
18
+ ```
19
+
20
+ or run `bundle add ajdc`.
21
+
22
+ ### Addinng agent skills
23
+
24
+ The gem ships a skill for coding agents in `skills/ajdc`. It follows the [skills.sh](https://www.skills.sh) layout, so it works with any agent that reads `SKILL.md`.
25
+
26
+ With [Rails Hyperdrive](https://github.com/rails-hyperdrive/rails-hyperdrive) it installs itself: `bin/rails hyperdrive:init` the first time, then `bundle add ajdc` or `bin/rails hyperdrive:sync` lands it in `.claude/skills/ajdc`.
27
+
28
+ Without it, install straight from the repository with the [skills.sh](https://www.skills.sh) CLI, or copy `skills/ajdc` into your agent's skills directory:
29
+
30
+ ```sh
31
+ npx skills add palkan/ajdc
32
+ ```
33
+
34
+ Now, you can ask your agent to continue the installation or do it yourself.
35
+
36
+ ### Preparing the database
37
+
38
+ AJ/DC keeps its data in two tables (`active_job_durable_runs`, `active_job_durable_steps`). Generate the migration and run it:
39
+
40
+ ```sh
41
+ bin/rails generate ajdc:install
42
+ bin/rails db:migrate
43
+ ```
44
+
45
+ To keep the tables in a separate database (the Solid Queue way), declare it in `config/database.yml` with its own `migrations_paths`, then pass its name; the migration lands in that path and an initializer points AJ/DC at the database:
46
+
47
+ ```sh
48
+ bin/rails generate ajdc:install --database=durable
49
+ bin/rails db:prepare
50
+ ```
51
+
52
+ ```ruby
53
+ # config/initializers/active_job_durable.rb (generated)
54
+ Rails.application.config.active_job_durable.connects_to = { database: { writing: :durable } }
55
+ ```
56
+
57
+ ### Requirements
58
+
59
+ - Ruby (MRI) >= 3.3
60
+ - Rails >= 8.1 (`ActiveJob::Attributes` needs Rails 8.2)
61
+ - SQLite / PostgreSQL / MySQL
62
+
63
+ ## Usage
64
+
65
+ Include `ActiveJob::Durable` instead of `ActiveJob::Continuable`. `step`, cursors and `attribute` keep working exactly as they do today:
66
+
67
+ ```ruby
68
+ # before
69
+ class ImportJob < ApplicationJob
70
+ include ActiveJob::Continuable
71
+
72
+ attribute :processed_count, :integer, default: 0
73
+
74
+ def perform(import)
75
+ step :validate
76
+ step :process do |step|
77
+ import.records.find_each(start: step.cursor) do |record|
78
+ record.process!
79
+ self.processed_count += 1
80
+ step.advance! from: record.id
81
+ end
82
+ end
83
+ end
84
+ end
85
+ ```
86
+
87
+ ```ruby
88
+ # after
89
+ class ImportJob < ApplicationJob
90
+ include ActiveJob::Durable
91
+
92
+ attribute :processed_count, :integer, default: 0
93
+
94
+ def perform(import)
95
+ step :validate
96
+ step :process do |step|
97
+ import.records.find_each(start: step.cursor) do |record|
98
+ record.process!
99
+ self.processed_count += 1
100
+ step.advance! from: record.id
101
+ end
102
+ end
103
+ end
104
+ end
105
+ ```
106
+
107
+ Every checkpoint commits the run and the current step's cursor to the database, so a `SIGKILL` loses at most the work since the last checkpoint.
108
+
109
+ The run and its steps are plain Active Record models:
110
+
111
+ ```ruby
112
+ run = ActiveJob::Durable::Run.last
113
+ run.status # => "completed"
114
+ run.current_step # => nil
115
+ run.completed_steps # => ["validate", "process"]
116
+ run.state # => {"processed_count" => 128}
117
+ run.steps.map(&:name) # => ["validate", "process"]
118
+ ```
119
+
120
+ ### Uniqueness
121
+
122
+ Durable state allows you to enforce wofkflow runs uniqueness. For that, use the `unique_by` macro in the workflow and specify the `perform` parameters that identify a run. Uniqueness is only enforced for runs that hasn't been terminated: either _live_ runs (with "enqueued", "running", "waiting", "awaiting" status) or _paused_ runs ("failed" or "halted"). Here is an example:
123
+
124
+ ```ruby
125
+ class ImportJob < ApplicationJob
126
+ include ActiveJob::Durable
127
+
128
+ unique_by :import
129
+
130
+ def perform(import)
131
+ step :check
132
+ step :process
133
+ end
134
+ end
135
+
136
+ ImportJob.perform_later(import) # => the job
137
+ ImportJob.perform_later(import) # => false, nothing enqueued
138
+ ```
139
+
140
+ You can provide and option `on_conflict` parameter to specify what to do in case of the uniqueness conflict:
141
+
142
+ - `:skip` (default) enqueues nothing; `perform_later` returns `false`.
143
+ - `:reject` raises `ActiveJob::Durable::RunAlreadyExists` (the currnent run is available as `error.run`).
144
+ - `:replace` cancels the current run and starts the new one. A renewal or a reschedule is `perform_later` again:
145
+
146
+ ```ruby
147
+ class License::LifecycleJob < ApplicationJob
148
+ include ActiveJob::Durable
149
+
150
+ unique_by :license, on_conflict: :replace
15
151
  # ...
16
- spec.add_dependency "ajdc"
152
+ end
153
+ ```
154
+
155
+ ### Reading runs
156
+
157
+ A job class knows its runs, newest first. `workflow_runs` is an Active Record relation, so the scopes below chain onto it:
158
+
159
+ ```ruby
160
+ ImportJob.workflow_runs
161
+ ImportJob.workflow_runs.failed.at_step(:process)
162
+ ```
163
+
164
+ `for` finds the runs `perform_later` would have created with the same arguments; it derives the key the same way. `for(workflow_key:)` matches a key verbatim:
165
+
166
+ ```ruby
167
+ ImportJob.workflow_runs.for(import) # same key as ImportJob.perform_later(import)
168
+ ImportJob.workflow_runs.for(workflow_key: "imports/42")
169
+ ImportJob.workflow_runs.for(import).live.first # the run in progress, or nil
170
+ ```
171
+
172
+ Without a declaration, every argument is part of the key: positional arguments in order, then keywords sorted by name as `name=value`. A record renders as `collection/id`; a value that is not a string, symbol, number or boolean becomes a short digest:
173
+
174
+ ```ruby
175
+ ImportJob.perform_later(import, "csv", strict: true) # key "imports/42:csv:strict=true"
176
+ ```
177
+
178
+ `identified_by` narrows the key to the named `perform` parameters, positional or keyword:
179
+
180
+ ```ruby
181
+ class ImportJob < ApplicationJob
182
+ include ActiveJob::Durable
183
+
184
+ identified_by :import # key "imports/42", whatever the other arguments
185
+
186
+ def perform(import, format = "csv", strict: false)
187
+ # ...
188
+ end
189
+ end
190
+ ```
191
+
192
+ The block form receives the `perform` arguments and returns one component or an array of them:
193
+
194
+ ```ruby
195
+ identified_by { |import, **kwargs| [import, kwargs.fetch(:format, "csv")] } # key "imports/42:csv"
196
+ ```
197
+
198
+ A name that is not a `perform` parameter raises `ArgumentError`: at the declaration when the class already defines `perform`, otherwise at the first `perform_later`.
199
+
200
+ `set(workflow_key:)` names one run's key verbatim, whatever the class derives. It is an enqueue option like `wait:` or `queue:`, so it works with `perform_later`, `perform_now` and `perform_all_later`:
201
+
202
+ ```ruby
203
+ ImportJob.set(workflow_key: "imports/42/retry-3", queue: "low").perform_later(import)
204
+ ImportJob.workflow_runs.for(workflow_key: "imports/42/retry-3")
205
+ ```
206
+
207
+ Other usefule scopes:
208
+
209
+ ```ruby
210
+ MyJob.workflow_runs.live # enqueued, running, waiting, awaiting
211
+ MyJob.workflow_runs.at_step(:process)
212
+ MyJob.workflow_runs.stuck_for(1.hour)
213
+ MyJob.workflow_runs.newest_first
214
+ ```
215
+
216
+ A run's `status` says where the job is. `enqueued`: the job is in the queue, written at the first enqueue and at every re-enqueue (an isolated step, a graceful-stop interrupt, a resume after an error, a `retry_on` retry). `running`: a worker is executing it. `started_at` is set once, at the first execution, so an `enqueued` run with no `started_at` never ran. `stuck_for` reads `enqueued` runs by `transitioned_at` and `running` runs by `last_heartbeat_at`.
217
+
218
+ ### Halting
219
+
220
+ Halting allows you to pause the execution on an error that could be resolved by a human (or alike), so the run could be restarted later from the current step/cursor (e.g., a plan out of storage, a file to fix by hand). Use `halt_on` or `halt!` to stop the run but keep it resumeable (unlike `discard_on`):
221
+
222
+ ```ruby
223
+ class ImportJob < ApplicationJob
224
+ include ActiveJob::Durable
225
+
226
+ discard_on ZipFile::InvalidFileError
227
+ halt_on InsufficientStorageSpaceError
228
+
229
+ def perform(import)
230
+ step :check
231
+ step :process
232
+ end
233
+ end
234
+ ```
235
+
236
+ The run's status is `halted`; `error_class` and `error_message` hold the error, `current_step` names the step and its row keeps the cursor.
237
+
238
+ `halt!(reason)` does the same from inside a step, without an error; the reason lands in `halt_reason`:
239
+
240
+ ```ruby
241
+ step :run do |step|
242
+ until chat.complete?
243
+ chat.step
244
+ halt!(:tool_approval) if chat.awaiting_approval?
245
+ step.checkpoint!
246
+ end
247
+ end
248
+ ```
249
+
250
+ ```ruby
251
+ ImportJob.workflow_runs.halted.first.halt_reason # => "tool_approval"
252
+ ```
253
+
254
+ A halted (or failed) run is resumed in place: `resume!` puts the job back in the queue, the step that stopped re-runs from its cursor as a new attempt, and the earlier attempts keep their errors. Any other status raises `ActiveJob::Durable::NotResumable`.
255
+
256
+ ```ruby
257
+ ImportJob.workflow_runs.halted.first.resume!
258
+ ```
259
+
260
+ ### Cancelling
261
+
262
+ `cancel!` ends a run that is not terminal: the row becomes `cancelled` and the queue is left alone. A queued job for a cancelled run performs nothing; a running job stops at its next checkpoint or step boundary, and the open step row is `cancelled` with its cursor. A terminal run raises `ActiveJob::Durable::NotCancellable`.
263
+
264
+ ```ruby
265
+ ImportJob.workflow_runs.for(import).live.sole.cancel!
266
+ ```
267
+
268
+ ### Step callbacks
269
+
270
+ `before_step`, `after_step` and `around_step` are Active Job callbacks, like `before_perform` and friends, for every step that runs; a step skipped on resume triggers none. `after_step` runs only when the step completes. `current_step` is the running `ActiveJob::Continuation::Step`:
271
+
272
+ ```ruby
273
+ class Cable::DiagnosticJob < ApplicationJob
274
+ include ActiveJob::Durable
275
+
276
+ after_step :broadcast_update
277
+ around_step { |job, block| Rails.logger.tagged(job.current_step.name, &block) }
278
+
279
+ def perform(cable)
280
+ @cable = cable
281
+ step :provider_status, isolated: true
282
+ step :websocket_status, isolated: true
283
+ step :admin_api_status, isolated: true
284
+ end
285
+
286
+ private
287
+ def broadcast_update
288
+ @cable.broadcast_replace(partial: "cables/diagnostic", locals: { step: current_step.name })
289
+ end
290
+ end
291
+ ```
292
+
293
+ ### Timers
294
+
295
+ A workflow run can _go to sleep_ (=pause) and _wake up_ (=resume) eaither at specific time (`wait_until`) or after a given time interval passed (`wait`). Example:
296
+
297
+ ```ruby
298
+ class License::LifecycleJob < ApplicationJob
299
+ include ActiveJob::Durable
300
+
301
+ unique_by :license, on_conflict: :replace
302
+
303
+ def perform(license)
304
+ step :remind, wait_until: license.expires_at - 2.weeks
305
+ step :expire, wait_until: license.expires_at
306
+ step :revoke, wait: 2.weeks
307
+ end
308
+
17
309
  # ...
18
310
  end
19
311
  ```
20
312
 
21
- Or adding to your project:
313
+ When the workflow reaches the waiting step, it's status is changed to `waiting`, and it's no longer present in the job queue. To wake sleeping workflows up, you need to run a single recurring job, `ActiveJob::Durable::WakeJob`. For example, with Solid Queue:
314
+
315
+ ```yaml
316
+ # config/recurring.yml
317
+ durable_wake:
318
+ class: ActiveJob::Durable::WakeJob
319
+ schedule: every minute
320
+ ```
321
+
322
+ The job is one call to `ActiveJob::Durable.wake_up_due`, which wakes every due run once and returns how many. In tests, call it yourself inside `travel_to`:
22
323
 
23
324
  ```ruby
24
- # Gemfile
25
- gem "ajdc"
325
+ travel_to license.expires_at - 2.weeks do
326
+ perform_enqueued_jobs { ActiveJob::Durable.wake_up_due }
327
+ end
26
328
  ```
27
329
 
28
- ### Supported Ruby versions
330
+ You can also wake up a single workflow manually by using the `#wake_up` method:
29
331
 
30
- - Ruby (MRI) >= 3.3
332
+ ```ruby
333
+ License::LifecycleJob.workflow_runs.for(license).waiting.sole.wake_up # send the reminder now
334
+ ```
31
335
 
32
- ## Usage
336
+ ### Signals
337
+
338
+ The durable job can sleep indifinitely waiting for an external signal to wake it up. For that, you can use the `#await` method. It defines a step with no body but a _signal handler_ instead:
339
+
340
+ ```ruby
341
+ class BulkImportJob < ApplicationJob
342
+ include ActiveJob::Durable
343
+
344
+ attribute :confirmed, :boolean, default: false
345
+
346
+ def perform(import)
347
+ await :confirmation, wait: 10.minutes
348
+ return import.destroy! unless confirmed
349
+
350
+ step :apply do
351
+ BulkImportService.new.call(import)
352
+ end
353
+ end
354
+
355
+ private
356
+ def confirmation(signal) = self.confirmed = signal.presence
357
+ end
358
+ ```
359
+
360
+ Then, you can use the `#wake_up` method to send a signal to the job:
361
+
362
+ ```ruby
363
+ # from the controller
364
+ BulkImportJob.workflow_runs.for(import).live.sole.wake_up(:confirmation, true)
365
+ ```
366
+
367
+ A signal sent before the `await` line is stored in the durable state and is replayed as soon as the job reaches this step. So, you can have multiple awaiting steps receiving signals in any order. A second signal for the same name overwrites the first.
368
+
369
+ If no signal received and the deadline is specified (`wait:`, `wait_until:`), the signal handler is called with the `nil` signal, so you can decide on how to continue.
370
+
371
+ You can find all the jobs waiting for a particular signal using the corresponding scope:
372
+
373
+ ```ruby
374
+ CardGenerationJob.workflow_runs.awaiting.at_step(:review)
375
+ ```
376
+
377
+ ### Housekeeping
378
+
379
+ Terminal runs (`completed`, `discarded`, `cancelled`) and their steps are kept for 14 days after they end. To delete the older ones, schedule `ActiveJob::Durable::HousekeepingJob`, e.g., with Solid Queue:
380
+
381
+ ```yaml
382
+ # config/recurring.yml
383
+ durable_housekeeping:
384
+ class: ActiveJob::Durable::HousekeepingJob
385
+ schedule: every hour
386
+ ```
387
+
388
+ Configure the retention period (`nil` keeps runs forever):
389
+
390
+ ```ruby
391
+ # config/application.rb
392
+ config.active_job_durable.keep_terminal_runs_for = 30.days
393
+ ```
394
+
395
+ Runs that need attention (`failed`, `halted`) are never deleted. Stuck runs are left to the application: `stuck_for` finds `enqueued` runs whose message never arrived, `running` runs whose worker died (no heartbeat), and parked runs the clock missed, and `cancel!` ends any of them:
396
+
397
+ ```ruby
398
+ ActiveJob::Durable::Run.stuck_for(1.hour).running.find_each(&:cancel!)
399
+ ```
33
400
 
34
- TBD
401
+ A heartbeat is written at every step boundary and checkpoint, so pick a duration longer than the longest stretch of a step without a checkpoint.
35
402
 
36
403
  ## Contributing
37
404
 
@@ -0,0 +1,57 @@
1
+ # frozen_string_literal: true
2
+
3
+ # Tables for ActiveJob::Durable: one row per run, one row per step attempt.
4
+ #
5
+ # JSON columns hold Active Job-serialized values (ActiveJob::Arguments.serialize),
6
+ # nothing in them is indexed. They carry no database defaults so that the same
7
+ # file loads on SQLite, PostgreSQL and MySQL; the models set the defaults.
8
+ ActiveRecord::Schema[8.1].define(version: 1) do
9
+ create_table "active_job_durable_runs", if_not_exists: true do |t|
10
+ t.string "job_class", null: false
11
+ t.string "key", null: false
12
+ t.string "active_key"
13
+ t.string "active_job_id", null: false
14
+ t.json "arguments", null: false
15
+ t.string "status", null: false
16
+ t.string "current_step"
17
+ t.json "completed_steps", null: false
18
+ t.json "state", null: false
19
+ t.datetime "wake_at"
20
+ t.json "pending_signals", null: false
21
+ t.json "parked_job"
22
+ t.integer "resumptions", default: 0, null: false
23
+ t.datetime "last_heartbeat_at"
24
+ t.string "error_class"
25
+ t.text "error_message"
26
+ t.string "halt_reason"
27
+ t.datetime "started_at"
28
+ t.datetime "finished_at"
29
+ t.datetime "transitioned_at"
30
+ t.timestamps
31
+
32
+ t.index ["job_class", "active_key"], name: "index_active_job_durable_runs_on_active_key", unique: true
33
+ t.index ["job_class", "key"], name: "index_active_job_durable_runs_on_key"
34
+ t.index ["active_job_id"], name: "index_active_job_durable_runs_on_active_job_id", unique: true
35
+ t.index ["status"], name: "index_active_job_durable_runs_on_status"
36
+ t.index ["status", "wake_at"], name: "index_active_job_durable_runs_on_status_and_wake_at"
37
+ t.index ["status", "transitioned_at"], name: "index_active_job_durable_runs_on_status_and_transitioned_at"
38
+ end
39
+
40
+ create_table "active_job_durable_steps", if_not_exists: true do |t|
41
+ t.references "run", null: false, index: false,
42
+ foreign_key: {to_table: "active_job_durable_runs", on_delete: :cascade}
43
+ t.string "name", null: false
44
+ t.integer "position", null: false
45
+ t.integer "attempt", default: 1, null: false
46
+ t.string "status", null: false
47
+ t.json "cursor"
48
+ t.boolean "isolated", default: false, null: false
49
+ t.string "error_class"
50
+ t.text "error_message"
51
+ t.datetime "started_at"
52
+ t.datetime "finished_at"
53
+
54
+ t.index ["run_id", "name", "attempt"], name: "index_active_job_durable_steps_on_attempt", unique: true
55
+ t.index ["run_id", "position"], name: "index_active_job_durable_steps_on_position"
56
+ end
57
+ end
@@ -0,0 +1,48 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "active_job/continuable/configurable"
4
+ require "active_job/continuation/callbacks"
5
+
6
+ module ActiveJob
7
+ module Continuable
8
+ # = Step callbacks
9
+ #
10
+ # Callbacks around every step that runs (a step skipped on resume runs none),
11
+ # with the semantics of `before_perform` and friends; the step is `current_step`.
12
+ # `after_step` runs only when the step completes.
13
+ #
14
+ # class ProcessImportJob < ApplicationJob
15
+ # include ActiveJob::Continuable::Callbacks
16
+ #
17
+ # around_step :measure
18
+ # after_step { logger.info "#{current_step.name} done" }
19
+ # end
20
+ module Callbacks
21
+ extend ActiveSupport::Concern
22
+ include Configurable
23
+
24
+ included do
25
+ define_callbacks :step, skip_after_callbacks_if_terminated: true
26
+
27
+ unless continuation_class <= Continuation::Callbacks
28
+ self.continuation_class = Class.new(continuation_class) do
29
+ prepend Continuation::Callbacks
30
+
31
+ set_temporary_name "#{superclass.name}(WithCallbacks)"
32
+ end
33
+ end
34
+ end
35
+
36
+ class_methods do
37
+ def before_step(*filters, &blk) = set_callback(:step, :before, *filters, &blk)
38
+
39
+ def after_step(*filters, &blk) = set_callback(:step, :after, *filters, &blk)
40
+
41
+ def around_step(*filters, &blk) = set_callback(:step, :around, *filters, &blk)
42
+ end
43
+
44
+ # The `ActiveJob::Continuation::Step` that is running, `nil` between steps.
45
+ def current_step = continuation.running_step
46
+ end
47
+ end
48
+ end
@@ -0,0 +1,75 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "active_job/continuable"
4
+
5
+ module ActiveJob
6
+ module Continuable
7
+ # = Configurable continuation
8
+ #
9
+ # Continuable with a pluggable continuation class, which receives any extra
10
+ # `step` options along with `start:` and `isolated:`:
11
+ #
12
+ # class TimedJob < ApplicationJob
13
+ # include ActiveJob::Continuable::Configurable
14
+ #
15
+ # self.continuation_class = TimedContinuation
16
+ #
17
+ # def perform
18
+ # step :remind, wait: 2.weeks # TimedContinuation#run_step(:remind, start: nil, isolated: false, wait: 2.weeks)
19
+ # end
20
+ # end
21
+ module Configurable
22
+ extend ActiveSupport::Concern
23
+ include ActiveJob::Continuable
24
+
25
+ # Continuable defines its constructor on the including class, so this one
26
+ # is prepended to the class to run after it.
27
+ module Initializer # :nodoc:
28
+ def initialize(...)
29
+ super
30
+ self.continuation = continuation_class.new(self, {})
31
+ end
32
+ end
33
+
34
+ included do
35
+ class_attribute :continuation_class, instance_writer: false, default: ActiveJob::Continuation
36
+
37
+ prepend Initializer
38
+ end
39
+
40
+ def step(step_name, start: nil, isolated: false, **options, &block)
41
+ block ||= step_method_block(step_name)
42
+ checkpoint! if continuation.advanced?
43
+ continuation.step(step_name, start:, isolated:, **options, &block)
44
+ end
45
+
46
+ if Continuable.private_method_defined?(:continuation_serialized?) # Rails 8.2+
47
+ private def deserialize_arguments_if_needed
48
+ serialized, @serialized_continuation = @serialized_continuation, nil
49
+ super
50
+ self.continuation = continuation_class.new(self, serialized) if serialized
51
+ end
52
+ else
53
+ def deserialize(job_data) # :nodoc:
54
+ super
55
+ self.continuation = continuation_class.new(self, job_data.fetch("continuation", {}))
56
+ end
57
+ end
58
+
59
+ private
60
+
61
+ # The method named after the step, as a step block.
62
+ def step_method_block(step_name)
63
+ step_method = method(step_name)
64
+
65
+ raise ArgumentError, "Step method '#{step_name}' must accept 0 or 1 arguments" if step_method.arity > 1
66
+
67
+ if step_method.parameters.any? { |type, _name| type == :key || type == :keyreq }
68
+ raise ArgumentError, "Step method '#{step_name}' must not accept keyword arguments"
69
+ end
70
+
71
+ (step_method.arity == 0) ? ->(_step) { step_method.call } : step_method
72
+ end
73
+ end
74
+ end
75
+ end
@@ -0,0 +1,18 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "active_job/continuation"
4
+
5
+ module ActiveJob
6
+ class Continuation
7
+ # Runs the job's step callbacks around each step's body. Prepended to the
8
+ # job's `continuation_class` by `ActiveJob::Continuable::Callbacks`.
9
+ module Callbacks
10
+ # The step that is running, `nil` between steps.
11
+ def running_step = (current if running_step?)
12
+
13
+ private
14
+
15
+ def instrumenting_step(step, &block) = super { job.run_callbacks(:step, &block) }
16
+ end
17
+ end
18
+ end
@@ -0,0 +1,29 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "active_job/continuation"
4
+
5
+ module ActiveJob
6
+ class Continuation
7
+ # = Rescue handlers first
8
+ #
9
+ # Continuable's `continue` rescues any `StandardError` raised after the job made
10
+ # progress and resumes the job, so `discard_on`, `retry_on` and `rescue_from`
11
+ # never see it. Prepended to `ActiveJob::Continuable`, this module gives the
12
+ # job's own handlers precedence: when a handler exists for the error,
13
+ # `resume_job` re-raises it, and `perform_now` runs the handler as it would
14
+ # for any other job. Errors without a handler are resumed as before, and a
15
+ # `Continuation::Interrupt` is never an error.
16
+ module RescueHandlersFirst
17
+ private
18
+
19
+ # Rails 8.1 passes the error-resume exception as `{exception: e}`; main
20
+ # passes it positionally. The argument goes to `super` unchanged.
21
+ def resume_job(exception)
22
+ error = exception.is_a?(Hash) ? exception[:exception] : exception
23
+ raise error if !error.is_a?(Continuation::Interrupt) && handler_for_rescue(error)
24
+
25
+ super
26
+ end
27
+ end
28
+ end
29
+ end