postburner 1.0.0.rc.6 → 1.0.0.rc.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +31 -0
- data/README.md +23 -35
- data/app/concerns/postburner/insertion.rb +9 -1
- data/app/models/postburner/orphaned_job.rb +3 -3
- data/app/models/postburner/schedule.rb +2 -2
- data/app/models/postburner/schedule_execution.rb +14 -10
- data/lib/postburner/active_job/adapter.rb +87 -31
- data/lib/postburner/engine.rb +16 -0
- data/lib/postburner/strategies/default_queue.rb +7 -2
- data/lib/postburner/strategies/inline_test_queue.rb +1 -1
- data/lib/postburner/strategies/null_queue.rb +4 -1
- data/lib/postburner/strategies/strict_queue.rb +7 -2
- data/lib/postburner/version.rb +1 -1
- data/lib/postburner.rb +167 -27
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 9e547f2f387408e6f3abed4bafba9f6c737c49ec3d7cbc65bfd80dd2ba509451
|
|
4
|
+
data.tar.gz: 7aa5b40f8224bc79abcba8d63f423410ed86cb74c9db9a47a47c83027744a667
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 24aa9c290a6f34ea9630823da4849a96d28cf786bbfdac43e8e2ba45fc79d04fa49d2d99ef1917740baca2e282693ce80e928a010df653cb55b78f9f19d15f39
|
|
7
|
+
data.tar.gz: e39b5f65a04804282d5050115111c1906377ad9da2cc169c805235a43d6933687e01ecc2619c2869fb99cb004e6b7d6fed3009ccd2b720a4e13e9998eb83eae6
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,36 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## Unreleased
|
|
4
|
+
|
|
5
|
+
## v1.0.0.rc.8 - 2026-10-05
|
|
6
|
+
|
|
7
|
+
### Fixed
|
|
8
|
+
|
|
9
|
+
- **A testing strategy no longer drops ActiveJob enqueues outside the test environment.** `NullQueue`, `InlineTestQueue`, and `TimeTravelTestQueue` still keep `Postburner::Job` off Beanstalkd in every environment. `perform_later`, `deliver_later`, and `broadcast_*_later` skip the put only when `Rails.env.test?` and the strategy is a testing strategy. In production the adapter always puts, including after `Postburner.null_strategy!`.
|
|
10
|
+
- **The test strategy is chosen at boot.** The engine reads `config.active_job.queue_adapter` from `after_initialize`, so the choice is in place before the first test and does not wait for `ActiveJob::Base` to load. A strategy assigned in an initializer, or later in `test_helper.rb`, is kept. `:test` selects `InlineTestQueue`. `:postburner` selects `NullQueue`.
|
|
11
|
+
- **Boot no longer reads `config.active_job` unless the test environment has that railtie.** An application that requires Active Record, Action Controller, and Action View without the ActiveJob railtie has no `config.active_job`. The test-strategy hook returns before touching it outside test, and when `config` does not respond to `active_job`.
|
|
12
|
+
|
|
13
|
+
### Changed
|
|
14
|
+
|
|
15
|
+
- **`perform_later` waits for the surrounding transaction to commit.** Selecting the Postburner adapter sets `ActiveJob::Base.enqueue_after_transaction_commit = true`. A job enqueued inside a joinable transaction is put after commit and dropped on rollback. A Beanstalkd error on that put is raised after the data is committed. The flag lives on `ActiveJob::Base`, so a job class that uses another adapter in the same process inherits the same deferral.
|
|
16
|
+
- **In the test environment, `:postburner` does not write to Beanstalkd.** `Postburner::Job#queue!` keeps the row (`NullQueue`). ActiveJob enqueues still write a tracked row when the job is tracked, still emit the enqueue notification, and do not put. `Postburner.default_strategy!` or `Postburner.strict_strategy!` is the opt-in that puts.
|
|
17
|
+
|
|
18
|
+
### Upgrade notes
|
|
19
|
+
|
|
20
|
+
- A test that asserts a real tube job after `perform_later` or `queue!` calls `Postburner.default_strategy!` or `Postburner.strict_strategy!`.
|
|
21
|
+
- `Postburner.null_strategy!` in production still skips the tube for `Postburner::Job` only. ActiveJob continues to put.
|
|
22
|
+
|
|
23
|
+
## v1.0.0.rc.7 - 2026-08-03
|
|
24
|
+
|
|
25
|
+
### Fixed
|
|
26
|
+
|
|
27
|
+
- **Delayed jobs can release before their `run_at`.** Beanstalkd delays are whole seconds, so a fractional delay has to round. Insertions are now computed as `(at - now).ceil`.
|
|
28
|
+
- **Premature re-insert could spin.** `NiceQueue`/`DefaultQueue` re-inserted a premature job with a fractional second value. Re-insert delay is now `[early.ceil, 1].max`.
|
|
29
|
+
|
|
30
|
+
### Changed
|
|
31
|
+
|
|
32
|
+
- Premature-job messages now report how early the job arrived: the `PREMATURE; RE-INSERTED` audit log entry and the `Postburner::Job::PrematurePerform` message both include the delta in seconds, which makes sub-second scheduling drift visible instead of silent.
|
|
33
|
+
|
|
3
34
|
## v1.0.0.rc.6 - 2026-06-27
|
|
4
35
|
|
|
5
36
|
### Highlights
|
data/README.md
CHANGED
|
@@ -919,15 +919,15 @@ Pay attention to the following when writing jobs:
|
|
|
919
919
|
|
|
920
920
|
## Queue Strategies
|
|
921
921
|
|
|
922
|
-
Postburner uses different strategies to control
|
|
922
|
+
Postburner uses different strategies to control `Postburner::Job` execution. In the test environment a testing strategy also tells the ActiveJob adapter not to put to Beanstalkd. Outside test the adapter always puts. `DefaultQueue` and `StrictQueue` put in every environment.
|
|
923
923
|
|
|
924
924
|
| Strategy | When to Use | Behavior | Requires Beanstalkd |
|
|
925
925
|
|----------|-------------|----------|---------------------|
|
|
926
|
-
| **DefaultQueue** | Production | Async, gracefully requeues premature jobs | Yes |
|
|
927
|
-
| **StrictQueue** | Production | Async via Beanstalkd, raises error on premature execution | Yes |
|
|
928
|
-
| **NullQueue** | Test
|
|
929
|
-
| **InlineTestQueue** | Test | Inline execution (error raised for delayed jobs) | No |
|
|
930
|
-
| **TimeTravelTestQueue** | Test | Inline execution, (auto time-travels for delayed jobs) | No |
|
|
926
|
+
| **DefaultQueue** | Production, or a test that needs a real tube | Async, gracefully requeues premature jobs. The ActiveJob adapter puts. | Yes |
|
|
927
|
+
| **StrictQueue** | Production, or a test that needs a real tube | Async via Beanstalkd, raises error on premature execution. The ActiveJob adapter puts. | Yes |
|
|
928
|
+
| **NullQueue** | Test default when the adapter is `:postburner`. Also deferred or manual `Postburner::Job` execution | Jobs created but not inserted into Beanstalkd. In test, the ActiveJob adapter does not put. Outside test, the adapter still puts. | No |
|
|
929
|
+
| **InlineTestQueue** | Test default when the adapter is `:test` | Inline execution (error raised for delayed jobs). In test, the ActiveJob adapter does not put. | No |
|
|
930
|
+
| **TimeTravelTestQueue** | Test | Inline execution, (auto time-travels for delayed jobs). In test, the ActiveJob adapter does not put. | No |
|
|
931
931
|
|
|
932
932
|
```ruby
|
|
933
933
|
# Switch strategies
|
|
@@ -950,45 +950,33 @@ There are 4 hook methods that are used with inheritance:
|
|
|
950
950
|
|
|
951
951
|
### Automatic Test Mode
|
|
952
952
|
|
|
953
|
-
|
|
953
|
+
`config.active_job.queue_adapter` is the only setup. At the end of boot the test environment selects a strategy that does not write to Beanstalkd. A strategy assigned in an initializer is already set by then and is kept. A strategy assigned in `test_helper.rb` runs after boot and is kept:
|
|
954
|
+
|
|
955
|
+
| `queue_adapter` | `Postburner::Job#queue!` | ActiveJob `perform_later` |
|
|
956
|
+
|-----------------|--------------------------|---------------------------|
|
|
957
|
+
| `:test` | `InlineTestQueue` (runs in process) | Rails keeps the job in memory |
|
|
958
|
+
| `:postburner` | `NullQueue` (row kept, run it with `Postburner::Job.perform`) | Tracked jobs still write their row. Nothing is put. |
|
|
954
959
|
|
|
955
960
|
```ruby
|
|
956
|
-
#
|
|
957
|
-
|
|
961
|
+
Postburner.default_strategy! # this example needs a real tube
|
|
962
|
+
SendEmail.perform_later(123) # now the adapter puts
|
|
958
963
|
```
|
|
959
964
|
|
|
960
|
-
Postburner
|
|
961
|
-
|
|
962
|
-
### Testing Default Jobs (ActiveJob)
|
|
963
|
-
|
|
964
|
-
Use standard ActiveJob test helpers:
|
|
965
|
-
|
|
966
|
-
```ruby
|
|
967
|
-
class MyTest < ActiveSupport::TestCase
|
|
968
|
-
test "job executes" do
|
|
969
|
-
SendEmail.perform_later(123)
|
|
970
|
-
# Job executes immediately in test mode
|
|
971
|
-
end
|
|
965
|
+
`Postburner.strict_strategy!` is the same opt-in with strict premature handling. `assert_enqueued_jobs` still needs the `:test` adapter, via `config/environments/test.rb` or `queue_adapter_for_test`.
|
|
972
966
|
|
|
973
|
-
|
|
974
|
-
travel 1.hour do
|
|
975
|
-
SendEmail.set(wait: 1.hour).perform_later(123)
|
|
976
|
-
end
|
|
977
|
-
end
|
|
978
|
-
end
|
|
979
|
-
```
|
|
967
|
+
An application that does not load the ActiveJob railtie has no `config.active_job`, so test boot leaves `DefaultQueue` in place and `Postburner::Job#queue!` puts. Call `Postburner.null_strategy!` in that app. `:inline` and `:async` leave `DefaultQueue` for the same reason: only `:test` and `:postburner` select a strategy that does not put.
|
|
980
968
|
|
|
981
969
|
### Testing Tracked Jobs
|
|
982
970
|
|
|
983
|
-
|
|
971
|
+
With the `:postburner` adapter in test, `perform_later` writes the `TrackedJob` row and leaves `bkid` nil:
|
|
984
972
|
|
|
985
973
|
```ruby
|
|
986
|
-
test "tracked job
|
|
974
|
+
test "tracked job records the enqueue" do
|
|
987
975
|
ProcessPayment.perform_later(456)
|
|
988
976
|
|
|
989
977
|
job = Postburner::TrackedJob.last
|
|
990
|
-
assert job.
|
|
991
|
-
|
|
978
|
+
assert job.queued_at.present?
|
|
979
|
+
assert_nil job.bkid
|
|
992
980
|
end
|
|
993
981
|
```
|
|
994
982
|
|
|
@@ -1014,16 +1002,16 @@ class EmailTest < ActiveSupport::TestCase
|
|
|
1014
1002
|
end
|
|
1015
1003
|
```
|
|
1016
1004
|
|
|
1017
|
-
**How this works:** `assert_enqueued_emails` and `assert_enqueued_jobs`
|
|
1005
|
+
**How this works:** `assert_enqueued_emails` and `assert_enqueued_jobs` read Rails' `:test` adapter. They do not see jobs enqueued through `:postburner`. Set the test adapter when a test needs those assertions. With `:postburner` left in place, test enqueues stay out of Beanstalkd until the test calls `Postburner.default_strategy!` or `Postburner.strict_strategy!`.
|
|
1018
1006
|
|
|
1019
|
-
|
|
1007
|
+
Rails uses the `:test` adapter in the test environment unless the application sets another one:
|
|
1020
1008
|
|
|
1021
1009
|
```ruby
|
|
1022
1010
|
# config/environments/test.rb
|
|
1023
1011
|
config.active_job.queue_adapter = :test
|
|
1024
1012
|
```
|
|
1025
1013
|
|
|
1026
|
-
|
|
1014
|
+
Without the `:test` adapter, `assert_enqueued_emails` does not see the enqueue. Postburner does not run the job in its place, and it does not put one on a tube either.
|
|
1027
1015
|
|
|
1028
1016
|
**`Postburner::Mailer`** bypasses ActiveJob entirely, so `assert_enqueued_emails` cannot see these jobs regardless of adapter. Assert on side effects instead:
|
|
1029
1017
|
|
|
@@ -85,7 +85,15 @@ module Postburner
|
|
|
85
85
|
self.run_at = case
|
|
86
86
|
when at.present?
|
|
87
87
|
# this is rudimentary, add error handling
|
|
88
|
-
|
|
88
|
+
#
|
|
89
|
+
# Ceil, don't floor. Beanstalkd delays are whole seconds,
|
|
90
|
+
# so some rounding is unavoidable -- but it must round
|
|
91
|
+
# *away* from now. Flooring both sides (`at.to_i -
|
|
92
|
+
# now.to_i`) drops frac(at) and frac(now) independently,
|
|
93
|
+
# releasing the job early by frac(at) - frac(now)
|
|
94
|
+
# whenever that is positive. Consumers then see a job
|
|
95
|
+
# arrive before its own run_at.
|
|
96
|
+
options[:delay] ||= [(at.to_time - now).ceil, 0].max
|
|
89
97
|
at
|
|
90
98
|
when options[:delay].present?
|
|
91
99
|
now + options[:delay].seconds
|
|
@@ -33,12 +33,12 @@ module Postburner
|
|
|
33
33
|
# application code.
|
|
34
34
|
#
|
|
35
35
|
# @example Typical lifecycle
|
|
36
|
-
# # A DB row exists with type: '
|
|
36
|
+
# # A DB row exists with type: 'MyApp::RemovedJob' (class deleted)
|
|
37
37
|
# job = Postburner::Job.find(42)
|
|
38
38
|
# job.class # => Postburner::OrphanedJob
|
|
39
|
-
# job.type # => '
|
|
39
|
+
# job.type # => 'MyApp::RemovedJob'
|
|
40
40
|
# job.orphaned? # => true
|
|
41
|
-
# job.missing_class_name # => '
|
|
41
|
+
# job.missing_class_name # => 'MyApp::RemovedJob'
|
|
42
42
|
# job.perform # => raises Postburner::Job::OrphanedJobError
|
|
43
43
|
# job.remove! # soft-deletes fine despite readonly?
|
|
44
44
|
#
|
|
@@ -895,7 +895,7 @@ module Postburner
|
|
|
895
895
|
|
|
896
896
|
# Calculate next time for month-based intervals.
|
|
897
897
|
#
|
|
898
|
-
# Handles variable month lengths
|
|
898
|
+
# Handles variable month lengths, as monthly billing cycles require.
|
|
899
899
|
# Always snaps to the anchor's day-of-month (or end of month if shorter).
|
|
900
900
|
#
|
|
901
901
|
# Uses Rails Duration for month arithmetic with day adjustment.
|
|
@@ -944,7 +944,7 @@ module Postburner
|
|
|
944
944
|
next_time = anchor_time + (n * interval).months
|
|
945
945
|
|
|
946
946
|
# Adjust day to match anchor's day (or end of month if shorter)
|
|
947
|
-
#
|
|
947
|
+
# "Snap back" behavior: a short month never permanently shifts the day
|
|
948
948
|
if next_time.end_of_month.day >= requested_day
|
|
949
949
|
next_time.change(day: requested_day)
|
|
950
950
|
else
|
|
@@ -144,7 +144,9 @@ module Postburner
|
|
|
144
144
|
# BOTH enqueue paths defer the actual Beanstalkd insertion until this
|
|
145
145
|
# transaction commits:
|
|
146
146
|
# - Postburner::Job#queue! inserts via an after-commit hook;
|
|
147
|
-
# - the
|
|
147
|
+
# - selecting the Postburner adapter sets
|
|
148
|
+
# ActiveJob::Base.enqueue_after_transaction_commit, so Rails holds
|
|
149
|
+
# the adapter enqueue until this transaction commits.
|
|
148
150
|
# So the real Beanstalkd id (and, for tracked ActiveJobs, the TrackedJob
|
|
149
151
|
# row) only exists AFTER the transaction below. We capture the job here and
|
|
150
152
|
# link/record the Beanstalkd id afterward.
|
|
@@ -276,9 +278,8 @@ module Postburner
|
|
|
276
278
|
|
|
277
279
|
# Validate that the next execution matches the cached next_run_at.
|
|
278
280
|
#
|
|
279
|
-
#
|
|
280
|
-
#
|
|
281
|
-
# of the next execution created.
|
|
281
|
+
# Ensures that the pre-calculated next_run_at in this execution matches
|
|
282
|
+
# the actual run_at of the next execution created.
|
|
282
283
|
#
|
|
283
284
|
# This validation helps detect calculation bugs or manual tampering with
|
|
284
285
|
# execution records.
|
|
@@ -379,11 +380,12 @@ module Postburner
|
|
|
379
380
|
# Creates and enqueues an ActiveJob instance with the scheduled run_at time.
|
|
380
381
|
# Handles both Hash args (passed as keyword arguments) and Array args (splatted).
|
|
381
382
|
#
|
|
382
|
-
# IMPORTANT: the Postburner
|
|
383
|
-
# enqueue_after_transaction_commit
|
|
384
|
-
# until the surrounding transaction commits. This
|
|
385
|
-
# *registers* the enqueue; the resulting
|
|
386
|
-
# ActiveJobs, the TrackedJob record)
|
|
383
|
+
# IMPORTANT: selecting the Postburner adapter sets
|
|
384
|
+
# ActiveJob::Base.enqueue_after_transaction_commit, so Rails defers the
|
|
385
|
+
# actual Beanstalkd put until the surrounding transaction commits. This
|
|
386
|
+
# method therefore only *registers* the enqueue; the resulting
|
|
387
|
+
# beanstalk_job_id (and, for tracked ActiveJobs, the TrackedJob record)
|
|
388
|
+
# are not available until after commit.
|
|
387
389
|
# Linking is the caller's responsibility via {#link_activejob!}.
|
|
388
390
|
#
|
|
389
391
|
# @return [ActiveJob::Base] the enqueued job, to be linked after commit
|
|
@@ -407,7 +409,9 @@ module Postburner
|
|
|
407
409
|
job.priority = cached_schedule['priority'] if cached_schedule['priority']
|
|
408
410
|
|
|
409
411
|
# Enqueue with scheduled time (use instance method enqueue, not class method perform_later)
|
|
410
|
-
|
|
412
|
+
# ceil so the job is never released before run_at; see
|
|
413
|
+
# Postburner::Insertion#queue!
|
|
414
|
+
delay = (run_at - Time.current).ceil
|
|
411
415
|
delay = 0 if delay < 0
|
|
412
416
|
|
|
413
417
|
if delay > 0
|
|
@@ -53,6 +53,17 @@ module ActiveJob
|
|
|
53
53
|
# end
|
|
54
54
|
#
|
|
55
55
|
class PostburnerAdapter
|
|
56
|
+
# Rails 8 reads `ActiveJob::Base.enqueue_after_transaction_commit` and
|
|
57
|
+
# never calls {#enqueue_after_transaction_commit?}. Setting it here, when
|
|
58
|
+
# the adapter is selected, covers jobs that inherit `ActiveJob::Base`
|
|
59
|
+
# directly (Turbo broadcasts, Action Mailer). A job class that uses a
|
|
60
|
+
# different adapter in the same process inherits the same deferral.
|
|
61
|
+
# The put then runs after commit, so a Beanstalkd failure raises after
|
|
62
|
+
# the surrounding data is committed and the enqueue is dropped on rollback.
|
|
63
|
+
def initialize(*)
|
|
64
|
+
ActiveJob::Base.enqueue_after_transaction_commit = true
|
|
65
|
+
end
|
|
66
|
+
|
|
56
67
|
# Enqueues a job for immediate execution.
|
|
57
68
|
#
|
|
58
69
|
# @param job [ActiveJob::Base] The job to enqueue
|
|
@@ -71,19 +82,17 @@ module ActiveJob
|
|
|
71
82
|
# @return [void]
|
|
72
83
|
#
|
|
73
84
|
def enqueue_at(job, timestamp)
|
|
74
|
-
if
|
|
85
|
+
if Postburner.suppress_active_job_enqueue?
|
|
86
|
+
enqueue_for_test(job, timestamp)
|
|
87
|
+
elsif tracked?(job)
|
|
75
88
|
enqueue_tracked(job, timestamp)
|
|
76
89
|
else
|
|
77
90
|
enqueue_default(job, timestamp)
|
|
78
91
|
end
|
|
79
92
|
end
|
|
80
93
|
|
|
81
|
-
#
|
|
82
|
-
#
|
|
83
|
-
# Returns true because Postburner uses Beanstalkd (external queue),
|
|
84
|
-
# which means jobs are safely enqueued outside the database transaction.
|
|
85
|
-
# This allows Rails to automatically defer job enqueuing until after
|
|
86
|
-
# the current database transaction commits.
|
|
94
|
+
# @deprecated Rails 8 does not call this. {#initialize} sets
|
|
95
|
+
# `ActiveJob::Base.enqueue_after_transaction_commit` instead.
|
|
87
96
|
#
|
|
88
97
|
# @return [Boolean] Always returns true
|
|
89
98
|
#
|
|
@@ -147,6 +156,32 @@ module ActiveJob
|
|
|
147
156
|
hook.call(job) || {}
|
|
148
157
|
end
|
|
149
158
|
|
|
159
|
+
# Test enqueue. Writes a tracked row when the job is tracked, emits the
|
|
160
|
+
# enqueue notification, and does not put to Beanstalkd.
|
|
161
|
+
#
|
|
162
|
+
# Reached only from {Postburner.suppress_active_job_enqueue?}, which is
|
|
163
|
+
# true in the test environment while a testing strategy is active.
|
|
164
|
+
# Outside test the adapter puts even when the strategy is {Postburner::NullQueue}.
|
|
165
|
+
# {Postburner.default_strategy!} and {Postburner.strict_strategy!} are
|
|
166
|
+
# the opt-in that puts in test.
|
|
167
|
+
#
|
|
168
|
+
# @param job [ActiveJob::Base] The job instance
|
|
169
|
+
# @param timestamp [Time, nil] When to execute the job
|
|
170
|
+
#
|
|
171
|
+
# @return [void]
|
|
172
|
+
#
|
|
173
|
+
def enqueue_for_test(job, timestamp)
|
|
174
|
+
tracked_job = tracked?(job) ? create_tracked_record(job, timestamp) : nil
|
|
175
|
+
job.provider_job_id = nil
|
|
176
|
+
instrument_enqueue(
|
|
177
|
+
job,
|
|
178
|
+
timestamp,
|
|
179
|
+
tracked: !tracked_job.nil?,
|
|
180
|
+
postburner_job_id: tracked_job&.id,
|
|
181
|
+
beanstalk_job_id: nil
|
|
182
|
+
)
|
|
183
|
+
end
|
|
184
|
+
|
|
150
185
|
# Enqueues a tracked job (with PostgreSQL audit trail).
|
|
151
186
|
#
|
|
152
187
|
# Creates a Postburner::TrackedJob record, then queues minimal payload
|
|
@@ -162,15 +197,12 @@ module ActiveJob
|
|
|
162
197
|
# @return [void]
|
|
163
198
|
#
|
|
164
199
|
def enqueue_tracked(job, timestamp)
|
|
165
|
-
|
|
166
|
-
tracked_job = Postburner::TrackedJob.create!(
|
|
167
|
-
args: Postburner::ActiveJob::Payload.serialize_for_tracked(job),
|
|
168
|
-
run_at: timestamp ? Time.zone.at(timestamp) : nil,
|
|
169
|
-
queued_at: Time.current
|
|
170
|
-
)
|
|
200
|
+
tracked_job = create_tracked_record(job, timestamp)
|
|
171
201
|
|
|
172
202
|
# Calculate delay for Beanstalkd
|
|
173
|
-
|
|
203
|
+
# ceil so the job is never released before its timestamp; see
|
|
204
|
+
# Postburner::Insertion#queue!
|
|
205
|
+
delay = timestamp ? [(Time.zone.at(timestamp) - Time.current).ceil, 0].max : 0
|
|
174
206
|
|
|
175
207
|
# Queue to Beanstalkd with minimal payload
|
|
176
208
|
bkid = nil
|
|
@@ -196,25 +228,13 @@ module ActiveJob
|
|
|
196
228
|
# callers (e.g. ScheduleExecution) can capture it after enqueue.
|
|
197
229
|
job.provider_job_id = bkid
|
|
198
230
|
|
|
199
|
-
|
|
200
|
-
job_payload = Postburner::Instrumentation.job_payload_from_activejob(
|
|
231
|
+
instrument_enqueue(
|
|
201
232
|
job,
|
|
233
|
+
timestamp,
|
|
202
234
|
tracked: true,
|
|
203
235
|
postburner_job_id: tracked_job.id,
|
|
204
236
|
beanstalk_job_id: bkid
|
|
205
237
|
)
|
|
206
|
-
scheduled_at = timestamp ? Time.zone.at(timestamp) : nil
|
|
207
|
-
|
|
208
|
-
if scheduled_at && scheduled_at > Time.current
|
|
209
|
-
ActiveSupport::Notifications.instrument('enqueue_at.job.postburner', {
|
|
210
|
-
job: job_payload,
|
|
211
|
-
scheduled_at: scheduled_at
|
|
212
|
-
})
|
|
213
|
-
else
|
|
214
|
-
ActiveSupport::Notifications.instrument('enqueue.job.postburner', {
|
|
215
|
-
job: job_payload
|
|
216
|
-
})
|
|
217
|
-
end
|
|
218
238
|
end
|
|
219
239
|
|
|
220
240
|
# Enqueues a default job (Beanstalkd only, no PostgreSQL).
|
|
@@ -232,7 +252,9 @@ module ActiveJob
|
|
|
232
252
|
# @return [void]
|
|
233
253
|
#
|
|
234
254
|
def enqueue_default(job, timestamp)
|
|
235
|
-
|
|
255
|
+
# ceil so the job is never released before its timestamp; see
|
|
256
|
+
# Postburner::Insertion#queue!
|
|
257
|
+
delay = timestamp ? [(Time.zone.at(timestamp) - Time.current).ceil, 0].max : 0
|
|
236
258
|
bkid = nil
|
|
237
259
|
|
|
238
260
|
Postburner.connected do |conn|
|
|
@@ -254,13 +276,47 @@ module ActiveJob
|
|
|
254
276
|
# callers (e.g. ScheduleExecution) can capture it after enqueue.
|
|
255
277
|
job.provider_job_id = bkid
|
|
256
278
|
|
|
257
|
-
|
|
258
|
-
job_payload = Postburner::Instrumentation.job_payload_from_activejob(
|
|
279
|
+
instrument_enqueue(
|
|
259
280
|
job,
|
|
281
|
+
timestamp,
|
|
260
282
|
tracked: false,
|
|
261
283
|
postburner_job_id: nil,
|
|
262
284
|
beanstalk_job_id: bkid
|
|
263
285
|
)
|
|
286
|
+
end
|
|
287
|
+
|
|
288
|
+
# Creates the PostgreSQL row for a tracked ActiveJob.
|
|
289
|
+
#
|
|
290
|
+
# @param job [ActiveJob::Base] The job instance
|
|
291
|
+
# @param timestamp [Numeric, nil] Scheduled time as a unix timestamp
|
|
292
|
+
#
|
|
293
|
+
# @return [Postburner::TrackedJob]
|
|
294
|
+
#
|
|
295
|
+
def create_tracked_record(job, timestamp)
|
|
296
|
+
Postburner::TrackedJob.create!(
|
|
297
|
+
args: Postburner::ActiveJob::Payload.serialize_for_tracked(job),
|
|
298
|
+
run_at: timestamp ? Time.zone.at(timestamp) : nil,
|
|
299
|
+
queued_at: Time.current
|
|
300
|
+
)
|
|
301
|
+
end
|
|
302
|
+
|
|
303
|
+
# Emits enqueue or enqueue_at for an ActiveJob.
|
|
304
|
+
#
|
|
305
|
+
# @param job [ActiveJob::Base]
|
|
306
|
+
# @param timestamp [Numeric, nil]
|
|
307
|
+
# @param tracked [Boolean]
|
|
308
|
+
# @param postburner_job_id [Integer, nil]
|
|
309
|
+
# @param beanstalk_job_id [Integer, nil]
|
|
310
|
+
#
|
|
311
|
+
# @return [void]
|
|
312
|
+
#
|
|
313
|
+
def instrument_enqueue(job, timestamp, tracked:, postburner_job_id:, beanstalk_job_id:)
|
|
314
|
+
job_payload = Postburner::Instrumentation.job_payload_from_activejob(
|
|
315
|
+
job,
|
|
316
|
+
tracked: tracked,
|
|
317
|
+
postburner_job_id: postburner_job_id,
|
|
318
|
+
beanstalk_job_id: beanstalk_job_id
|
|
319
|
+
)
|
|
264
320
|
scheduled_at = timestamp ? Time.zone.at(timestamp) : nil
|
|
265
321
|
|
|
266
322
|
if scheduled_at && scheduled_at > Time.current
|
data/lib/postburner/engine.rb
CHANGED
|
@@ -13,5 +13,21 @@ module Postburner
|
|
|
13
13
|
g.skip_routes true
|
|
14
14
|
end
|
|
15
15
|
|
|
16
|
+
# Choose the test strategy once boot has finished applying configuration.
|
|
17
|
+
# after_initialize runs after config/initializers and before a test helper
|
|
18
|
+
# continues. The configured adapter is already a name here, so this does
|
|
19
|
+
# not load ActiveJob::Base. An application that already assigned a
|
|
20
|
+
# strategy keeps that choice.
|
|
21
|
+
#
|
|
22
|
+
# The hook runs in every environment. {Postburner.install_test_queue_strategy_for}
|
|
23
|
+
# returns before reading +config.active_job+ outside test, and when the
|
|
24
|
+
# app did not load the ActiveJob railtie. That config namespace does not
|
|
25
|
+
# exist unless the railtie is required.
|
|
26
|
+
initializer "postburner.active_job_test_strategy", after: "active_job.set_configs" do |app|
|
|
27
|
+
app.config.after_initialize do
|
|
28
|
+
Postburner.install_test_queue_strategy_for(app)
|
|
29
|
+
end
|
|
30
|
+
end
|
|
31
|
+
|
|
16
32
|
end
|
|
17
33
|
end
|
|
@@ -77,8 +77,13 @@ module Postburner
|
|
|
77
77
|
# @note Logs "PREMATURE; RE-INSERTED" message to job's audit trail
|
|
78
78
|
#
|
|
79
79
|
def handle_premature_perform(job)
|
|
80
|
-
|
|
81
|
-
|
|
80
|
+
early = job.run_at - Time.current
|
|
81
|
+
# Never re-insert with a delay under a second. A fractional delay
|
|
82
|
+
# truncates to 0, so the job comes straight back, is judged premature
|
|
83
|
+
# again, and burns reserve/delete/put plus a DB write per pass until
|
|
84
|
+
# the clock catches up.
|
|
85
|
+
response = job.insert! delay: [early.ceil, 1].max
|
|
86
|
+
job.log! "PREMATURE (#{early.round(3)}s early); RE-INSERTED: #{response}"
|
|
82
87
|
end
|
|
83
88
|
end
|
|
84
89
|
end
|
|
@@ -7,7 +7,7 @@ module Postburner
|
|
|
7
7
|
# tests fast and predictable. It raises {Postburner::Job::PrematurePerform} if
|
|
8
8
|
# a job has a future run_at, forcing you to use explicit time travel in tests.
|
|
9
9
|
#
|
|
10
|
-
# @note
|
|
10
|
+
# @note Selected in the Rails test environment when the ActiveJob adapter is `:test`
|
|
11
11
|
#
|
|
12
12
|
# ## When to Use InlineTestQueue
|
|
13
13
|
#
|
|
@@ -9,7 +9,10 @@ module Postburner
|
|
|
9
9
|
# to Beanstalkd. Jobs can be manually executed later using {.handle_perform!},
|
|
10
10
|
# which includes automatic time travel for scheduled jobs.
|
|
11
11
|
#
|
|
12
|
-
# @note This is a test-mode strategy (returns true for testing)
|
|
12
|
+
# @note This is a test-mode strategy (returns true for testing).
|
|
13
|
+
# Selected in the Rails test environment when the ActiveJob adapter is `:postburner`.
|
|
14
|
+
# Outside the test environment it still keeps {Postburner::Job} off
|
|
15
|
+
# Beanstalkd. ActiveJob `perform_later` continues to put.
|
|
13
16
|
#
|
|
14
17
|
# ## When to Use NullQueue
|
|
15
18
|
#
|
|
@@ -81,7 +81,10 @@ module Postburner
|
|
|
81
81
|
|
|
82
82
|
# Get priority, TTR from job instance (respects instance overrides) or options
|
|
83
83
|
pri = options[:pri] || job.priority || Postburner.configuration.default_priority
|
|
84
|
-
|
|
84
|
+
# ceil, not to_i: callers may pass a Float delay (see
|
|
85
|
+
# DefaultQueue#handle_premature_perform), and truncating it would
|
|
86
|
+
# re-introduce the early release this guards against.
|
|
87
|
+
delay = options[:delay].present? ? [0, options[:delay].to_f.ceil].max : 0
|
|
85
88
|
ttr = options[:ttr] || job.ttr || Postburner.configuration.default_ttr
|
|
86
89
|
payload = JSON.generate(data)
|
|
87
90
|
|
|
@@ -119,7 +122,9 @@ module Postburner
|
|
|
119
122
|
# @raise [Postburner::Job::PrematurePerform] Always raises for this strategy
|
|
120
123
|
#
|
|
121
124
|
def handle_premature_perform(job)
|
|
122
|
-
|
|
125
|
+
early = job.run_at - Time.current
|
|
126
|
+
raise Postburner::Job::PrematurePerform,
|
|
127
|
+
"Job has future run_at: #{job.run_at} (#{early.round(3)}s early)"
|
|
123
128
|
end
|
|
124
129
|
end
|
|
125
130
|
end
|
data/lib/postburner/version.rb
CHANGED
data/lib/postburner.rb
CHANGED
|
@@ -45,9 +45,19 @@ require "postburner/test_helpers"
|
|
|
45
45
|
#
|
|
46
46
|
# ## Auto-Detection
|
|
47
47
|
#
|
|
48
|
-
#
|
|
49
|
-
#
|
|
50
|
-
#
|
|
48
|
+
# During `after_initialize`, {#install_test_queue_strategy!} reads
|
|
49
|
+
# `config.active_job.queue_adapter` and selects the test strategy. It does
|
|
50
|
+
# not wait for `ActiveJob::Base` to load, and it leaves a strategy the
|
|
51
|
+
# application already assigned in place.
|
|
52
|
+
#
|
|
53
|
+
# - `:test` → {InlineTestQueue}. `Postburner::Job#queue!` runs in process.
|
|
54
|
+
# - `:postburner` → {NullQueue}. The row is kept. In the test environment
|
|
55
|
+
# the ActiveJob adapter does not put. {#default_strategy!} or
|
|
56
|
+
# {#strict_strategy!} is the opt-in that writes to Beanstalkd.
|
|
57
|
+
# - Any other adapter leaves the current strategy unchanged.
|
|
58
|
+
#
|
|
59
|
+
# Outside the test environment the adapter always puts. A testing strategy
|
|
60
|
+
# still keeps {Postburner::Job} off Beanstalkd.
|
|
51
61
|
#
|
|
52
62
|
# ## Usage
|
|
53
63
|
#
|
|
@@ -67,17 +77,17 @@ require "postburner/test_helpers"
|
|
|
67
77
|
# job.queue!(delay: 1.hour)
|
|
68
78
|
#
|
|
69
79
|
# @example Switching queue strategies
|
|
70
|
-
# # Production
|
|
71
|
-
# Postburner.
|
|
80
|
+
# # Production default. Premature jobs are re-inserted with a delay.
|
|
81
|
+
# Postburner.default_strategy!
|
|
72
82
|
#
|
|
73
83
|
# # Production with strict premature errors
|
|
74
|
-
# Postburner.
|
|
84
|
+
# Postburner.strict_strategy!
|
|
75
85
|
#
|
|
76
86
|
# # Test mode with explicit time control
|
|
77
87
|
# Postburner.inline_test_strategy!
|
|
78
88
|
#
|
|
79
89
|
# # Test mode with automatic time travel
|
|
80
|
-
# Postburner.
|
|
90
|
+
# Postburner.time_travel_test_strategy!
|
|
81
91
|
#
|
|
82
92
|
# @example Using Beanstalkd connection
|
|
83
93
|
# Postburner.connected do |conn|
|
|
@@ -87,7 +97,8 @@ require "postburner/test_helpers"
|
|
|
87
97
|
#
|
|
88
98
|
# @example Checking if in test mode
|
|
89
99
|
# if Postburner.testing?
|
|
90
|
-
# #
|
|
100
|
+
# # Postburner::Job stays off Beanstalkd. In the test environment the
|
|
101
|
+
# # ActiveJob adapter does not put either.
|
|
91
102
|
# else
|
|
92
103
|
# # Jobs queued to Beanstalkd
|
|
93
104
|
# end
|
|
@@ -99,13 +110,15 @@ require "postburner/test_helpers"
|
|
|
99
110
|
# @see TimeTravelTestQueue
|
|
100
111
|
#
|
|
101
112
|
module Postburner
|
|
102
|
-
# @!attribute [
|
|
113
|
+
# @!attribute [r] queue_strategy
|
|
103
114
|
# The current queue strategy class used for job execution.
|
|
104
115
|
#
|
|
105
|
-
# Defaults to {DefaultQueue}
|
|
106
|
-
#
|
|
116
|
+
# Defaults to {DefaultQueue}. In the Rails test environment,
|
|
117
|
+
# {#install_test_queue_strategy!} switches to {InlineTestQueue} when the
|
|
118
|
+
# ActiveJob adapter is `:test`, and to {NullQueue} when it is `:postburner`,
|
|
119
|
+
# unless the application has already assigned a strategy.
|
|
107
120
|
#
|
|
108
|
-
# @return [Class] One of StrictQueue, DefaultQueue, InlineTestQueue, or
|
|
121
|
+
# @return [Class] One of StrictQueue, DefaultQueue, InlineTestQueue, TimeTravelTestQueue, or NullQueue
|
|
109
122
|
#
|
|
110
123
|
# @example
|
|
111
124
|
# Postburner.queue_strategy
|
|
@@ -116,18 +129,143 @@ module Postburner
|
|
|
116
129
|
# @see #inline_test_strategy!
|
|
117
130
|
# @see #time_travel_test_strategy!
|
|
118
131
|
#
|
|
119
|
-
|
|
132
|
+
mattr_reader :queue_strategy
|
|
133
|
+
|
|
134
|
+
# Assigns the queue strategy and records it as the application's choice.
|
|
135
|
+
#
|
|
136
|
+
# {#install_test_queue_strategy!} will not replace a strategy assigned
|
|
137
|
+
# through this writer or through a `*_strategy!` setter. The load-time
|
|
138
|
+
# default is applied without marking that choice.
|
|
139
|
+
#
|
|
140
|
+
# @param strategy [Class] A queue strategy class
|
|
141
|
+
# @return [Class]
|
|
142
|
+
#
|
|
143
|
+
def self.queue_strategy=(strategy)
|
|
144
|
+
@queue_strategy_explicit = true unless @applying_queue_strategy
|
|
145
|
+
@@queue_strategy = strategy
|
|
146
|
+
end
|
|
147
|
+
|
|
148
|
+
# True when application code assigned {#queue_strategy}, rather than the
|
|
149
|
+
# load-time default or {#install_test_queue_strategy!}.
|
|
150
|
+
#
|
|
151
|
+
# @return [Boolean]
|
|
152
|
+
#
|
|
153
|
+
def self.queue_strategy_explicit?
|
|
154
|
+
@queue_strategy_explicit == true
|
|
155
|
+
end
|
|
156
|
+
|
|
157
|
+
# Selects the queue strategy for the Rails test environment from the
|
|
158
|
+
# configured ActiveJob adapter. See the Auto-Detection section above.
|
|
159
|
+
#
|
|
160
|
+
# Called from {#install_test_queue_strategy_for} during `after_initialize`,
|
|
161
|
+
# which runs after `config/initializers` and before a test helper continues.
|
|
162
|
+
# The argument is `config.active_job.queue_adapter`. This method does not
|
|
163
|
+
# read `ActiveJob::Base.queue_adapter_name`, because that loads ActiveJob
|
|
164
|
+
# and forces `:async` when the adapter is still unset.
|
|
165
|
+
#
|
|
166
|
+
# No-op outside the test environment. No-op when {#queue_strategy_explicit?}
|
|
167
|
+
# is true. An adapter other than `:test` or `:postburner` leaves the
|
|
168
|
+
# current strategy unchanged.
|
|
169
|
+
#
|
|
170
|
+
# @param adapter [Symbol, String, Object, nil] Adapter name or instance.
|
|
171
|
+
# `nil` reads `Rails.application.config.active_job.queue_adapter`.
|
|
172
|
+
# @return [void]
|
|
173
|
+
#
|
|
174
|
+
def self.install_test_queue_strategy!(adapter = nil)
|
|
175
|
+
return unless defined?(Rails) && Rails.env.test?
|
|
176
|
+
return if queue_strategy_explicit?
|
|
177
|
+
|
|
178
|
+
case adapter_name_for_install(adapter)
|
|
179
|
+
when "test"
|
|
180
|
+
apply_queue_strategy(InlineTestQueue)
|
|
181
|
+
when "postburner"
|
|
182
|
+
apply_queue_strategy(NullQueue)
|
|
183
|
+
end
|
|
184
|
+
end
|
|
185
|
+
|
|
186
|
+
# Engine entry point for {#install_test_queue_strategy!}.
|
|
187
|
+
#
|
|
188
|
+
# Returns before reading +config.active_job+ unless this is the test
|
|
189
|
+
# environment and the application loaded the ActiveJob railtie. Without
|
|
190
|
+
# that railtie, +Rails::Application::Configuration+ has no +active_job+
|
|
191
|
+
# method, and reading it raises during boot.
|
|
192
|
+
#
|
|
193
|
+
# @param app [Rails::Application]
|
|
194
|
+
# @return [void]
|
|
195
|
+
#
|
|
196
|
+
def self.install_test_queue_strategy_for(app)
|
|
197
|
+
return unless defined?(Rails) && Rails.env.test?
|
|
198
|
+
return unless app.config.respond_to?(:active_job)
|
|
199
|
+
|
|
200
|
+
install_test_queue_strategy!(app.config.active_job.queue_adapter)
|
|
201
|
+
end
|
|
202
|
+
|
|
203
|
+
# True when the ActiveJob adapter should record the enqueue and skip the
|
|
204
|
+
# Beanstalkd put.
|
|
205
|
+
#
|
|
206
|
+
# Limited to the test environment. A testing strategy in production still
|
|
207
|
+
# only governs {Postburner::Job}: `perform_later`, `deliver_later`, and
|
|
208
|
+
# `broadcast_*_later` keep writing to the tube.
|
|
209
|
+
#
|
|
210
|
+
# @return [Boolean]
|
|
211
|
+
#
|
|
212
|
+
def self.suppress_active_job_enqueue?
|
|
213
|
+
return false unless defined?(Rails) && Rails.env.test?
|
|
214
|
+
testing?
|
|
215
|
+
end
|
|
216
|
+
|
|
217
|
+
# Assigns a strategy without recording an application choice.
|
|
218
|
+
#
|
|
219
|
+
# @param strategy [Class]
|
|
220
|
+
# @return [Class]
|
|
221
|
+
# @api private
|
|
222
|
+
#
|
|
223
|
+
def self.apply_queue_strategy(strategy)
|
|
224
|
+
previous = @applying_queue_strategy
|
|
225
|
+
@applying_queue_strategy = true
|
|
226
|
+
self.queue_strategy = strategy
|
|
227
|
+
ensure
|
|
228
|
+
@applying_queue_strategy = previous
|
|
229
|
+
end
|
|
230
|
+
private_class_method :apply_queue_strategy
|
|
231
|
+
|
|
232
|
+
# @param adapter [Symbol, String, Object, nil]
|
|
233
|
+
# @return [String, nil]
|
|
234
|
+
# @api private
|
|
235
|
+
#
|
|
236
|
+
def self.adapter_name_for_install(adapter)
|
|
237
|
+
adapter = configured_active_job_adapter if adapter.nil?
|
|
238
|
+
return if adapter.nil?
|
|
120
239
|
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
240
|
+
case adapter
|
|
241
|
+
when Symbol, String
|
|
242
|
+
adapter.to_s
|
|
243
|
+
else
|
|
244
|
+
return unless defined?(::ActiveJob) && ::ActiveJob.respond_to?(:adapter_name)
|
|
124
245
|
|
|
125
|
-
|
|
126
|
-
if defined?(Rails) && Rails.env.test? && defined?(::ActiveJob::Base)
|
|
127
|
-
if ::ActiveJob::Base.queue_adapter_name == :test
|
|
128
|
-
self.queue_strategy = Postburner::InlineTestQueue
|
|
246
|
+
::ActiveJob.adapter_name(adapter).to_s.underscore
|
|
129
247
|
end
|
|
130
248
|
end
|
|
249
|
+
private_class_method :adapter_name_for_install
|
|
250
|
+
|
|
251
|
+
# @return [Object, nil]
|
|
252
|
+
# @api private
|
|
253
|
+
#
|
|
254
|
+
def self.configured_active_job_adapter
|
|
255
|
+
return unless defined?(Rails) && Rails.respond_to?(:application) && Rails.application
|
|
256
|
+
|
|
257
|
+
config = Rails.application.config
|
|
258
|
+
return unless config.respond_to?(:active_job)
|
|
259
|
+
|
|
260
|
+
config.active_job.queue_adapter
|
|
261
|
+
end
|
|
262
|
+
private_class_method :configured_active_job_adapter
|
|
263
|
+
|
|
264
|
+
# DefaultQueue re-inserts premature jobs with a delay. The test strategy is
|
|
265
|
+
# applied later by {#install_test_queue_strategy!}. Assigning it here does
|
|
266
|
+
# not count as an application choice. Reading the adapter name at require
|
|
267
|
+
# time would force :async before the application sets it.
|
|
268
|
+
apply_queue_strategy(Postburner::DefaultQueue)
|
|
131
269
|
|
|
132
270
|
# Activates strict test mode with inline job execution.
|
|
133
271
|
#
|
|
@@ -154,7 +292,7 @@ module Postburner
|
|
|
154
292
|
# # Raises PrematurePerform - use travel_to instead
|
|
155
293
|
#
|
|
156
294
|
# @see InlineTestQueue
|
|
157
|
-
# @see #
|
|
295
|
+
# @see #time_travel_test_strategy!
|
|
158
296
|
#
|
|
159
297
|
def self.inline_test_strategy!
|
|
160
298
|
self.queue_strategy = Postburner::InlineTestQueue
|
|
@@ -282,16 +420,18 @@ module Postburner
|
|
|
282
420
|
self.queue_strategy = Postburner::NullQueue
|
|
283
421
|
end
|
|
284
422
|
|
|
285
|
-
# Checks if
|
|
423
|
+
# Checks if the current queue strategy is a testing strategy.
|
|
286
424
|
#
|
|
287
|
-
#
|
|
288
|
-
# {
|
|
289
|
-
#
|
|
425
|
+
# True for {NullQueue}, {InlineTestQueue}, and {TimeTravelTestQueue}.
|
|
426
|
+
# Those strategies keep {Postburner::Job} off Beanstalkd. In the test
|
|
427
|
+
# environment they also tell the ActiveJob adapter not to put. Outside
|
|
428
|
+
# test, the adapter puts even when this is true. {DefaultQueue} and
|
|
429
|
+
# {StrictQueue} put in every environment.
|
|
290
430
|
#
|
|
291
|
-
# @return [Boolean] true if using
|
|
431
|
+
# @return [Boolean] true if using a testing strategy
|
|
292
432
|
#
|
|
293
433
|
# @example
|
|
294
|
-
# Postburner.
|
|
434
|
+
# Postburner.default_strategy!
|
|
295
435
|
# Postburner.testing? # => false
|
|
296
436
|
#
|
|
297
437
|
# Postburner.inline_test_strategy!
|