activejob 4.2.11.3 → 5.0.0.beta1
Sign up to get free protection for your applications and to get access to all the features.
Potentially problematic release.
This version of activejob might be problematic. Click here for more details.
- checksums.yaml +5 -5
- data/CHANGELOG.md +88 -54
- data/MIT-LICENSE +1 -1
- data/README.md +5 -5
- data/lib/active_job.rb +2 -1
- data/lib/active_job/arguments.rb +38 -19
- data/lib/active_job/async_job.rb +77 -0
- data/lib/active_job/base.rb +3 -1
- data/lib/active_job/callbacks.rb +2 -2
- data/lib/active_job/core.rb +42 -5
- data/lib/active_job/enqueuing.rb +5 -0
- data/lib/active_job/gem_version.rb +4 -4
- data/lib/active_job/logging.rb +17 -3
- data/lib/active_job/queue_adapter.rb +44 -16
- data/lib/active_job/queue_adapters.rb +86 -0
- data/lib/active_job/queue_adapters/async_adapter.rb +23 -0
- data/lib/active_job/queue_adapters/backburner_adapter.rb +6 -8
- data/lib/active_job/queue_adapters/delayed_job_adapter.rb +9 -7
- data/lib/active_job/queue_adapters/inline_adapter.rb +5 -7
- data/lib/active_job/queue_adapters/qu_adapter.rb +11 -9
- data/lib/active_job/queue_adapters/que_adapter.rb +9 -7
- data/lib/active_job/queue_adapters/queue_classic_adapter.rb +22 -20
- data/lib/active_job/queue_adapters/resque_adapter.rb +8 -10
- data/lib/active_job/queue_adapters/sidekiq_adapter.rb +15 -17
- data/lib/active_job/queue_adapters/sneakers_adapter.rb +11 -11
- data/lib/active_job/queue_adapters/sucker_punch_adapter.rb +5 -7
- data/lib/active_job/queue_adapters/test_adapter.rb +25 -16
- data/lib/active_job/queue_name.rb +1 -1
- data/lib/active_job/queue_priority.rb +44 -0
- data/lib/active_job/test_helper.rb +144 -46
- data/lib/rails/generators/job/job_generator.rb +1 -2
- data/lib/rails/generators/job/templates/job.rb +1 -1
- metadata +15 -10
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
|
-
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 3bfdadd9d05eeb7c784d6916260b2ee3d1494d0d
|
4
|
+
data.tar.gz: 089b206f85a6a27a8b23b699c64bd7227dee50d7
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 6195c91d63855ba8eee3f6aedccdd0093a393ef462d566d1b936c299841e6469724f6819c0dd1f1046fae2b72036d2465e2e283f3b423264630c34a1ef1c6f47
|
7
|
+
data.tar.gz: 18c352ac26d1739716698165a8ce4a8c9a2c36fba7b4177692a27d42295f5b74ff5a63fdcbcec10ccd5510bffb899761d086fdc72d0cf70aaea2c368cef6d406
|
data/CHANGELOG.md
CHANGED
@@ -1,103 +1,137 @@
|
|
1
|
-
## Rails
|
1
|
+
## Rails 5.0.0.beta1 (December 18, 2015) ##
|
2
2
|
|
3
3
|
* No changes.
|
4
4
|
|
5
5
|
|
6
|
-
|
7
|
-
|
8
|
-
* No changes.
|
9
|
-
|
10
|
-
|
11
|
-
## Rails 4.2.11.1 (March 11, 2019) ##
|
12
|
-
|
13
|
-
* No changes.
|
6
|
+
* Fixed serializing `:at` option for `assert_enqueued_with`
|
7
|
+
and `assert_performed_with`.
|
14
8
|
|
9
|
+
*Wojciech Wnętrzak*
|
15
10
|
|
16
|
-
|
11
|
+
* Support passing array to `assert_enqueued_jobs` in `:only` option.
|
17
12
|
|
18
|
-
*
|
13
|
+
*Wojciech Wnętrzak*
|
19
14
|
|
20
|
-
|
21
|
-
information that should not be accessible to them.
|
15
|
+
* Add job priorities to Active Job.
|
22
16
|
|
23
|
-
|
17
|
+
*wvengen*
|
24
18
|
|
25
|
-
|
19
|
+
* Implement a simple `AsyncJob` processor and associated `AsyncAdapter` that
|
20
|
+
queue jobs to a `concurrent-ruby` thread pool.
|
26
21
|
|
22
|
+
*Jerry D'Antonio*
|
27
23
|
|
28
|
-
|
24
|
+
* Implement `provider_job_id` for `queue_classic` adapter. This requires the
|
25
|
+
latest, currently unreleased, version of queue_classic.
|
29
26
|
|
30
|
-
*
|
31
|
-
|
32
|
-
|
33
|
-
## Rails 4.2.9 (June 26, 2017) ##
|
34
|
-
|
35
|
-
* No changes.
|
27
|
+
*Yves Senn*
|
36
28
|
|
29
|
+
* `assert_enqueued_with` and `assert_performed_with` now returns the matched
|
30
|
+
job instance for further assertions.
|
37
31
|
|
38
|
-
|
32
|
+
*Jean Boussier*
|
39
33
|
|
40
|
-
*
|
41
|
-
|
42
|
-
|
43
|
-
## Rails 4.2.7 (July 12, 2016) ##
|
34
|
+
* Include I18n.locale into job serialization/deserialization and use it around
|
35
|
+
`perform`.
|
44
36
|
|
45
|
-
|
37
|
+
Fixes #20799.
|
46
38
|
|
39
|
+
*Johannes Opper*
|
47
40
|
|
48
|
-
|
41
|
+
* Allow `DelayedJob`, `Sidekiq`, `qu`, and `que` to report the job id back to
|
42
|
+
`ActiveJob::Base` as `provider_job_id`.
|
49
43
|
|
50
|
-
|
44
|
+
Fixes #18821.
|
51
45
|
|
46
|
+
*Kevin Deisz*, *Jeroen van Baarsen*
|
52
47
|
|
53
|
-
|
48
|
+
* `assert_enqueued_jobs` and `assert_performed_jobs` in block form use the
|
49
|
+
given number as expected value. This makes the error message much easier to
|
50
|
+
understand.
|
54
51
|
|
55
|
-
*
|
52
|
+
*y-yagi*
|
56
53
|
|
54
|
+
* A generated job now inherits from `app/jobs/application_job.rb` by default.
|
57
55
|
|
58
|
-
|
56
|
+
*Jeroen van Baarsen*
|
59
57
|
|
60
|
-
*
|
58
|
+
* Add an `:only` option to `perform_enqueued_jobs` to filter jobs based on
|
59
|
+
type.
|
61
60
|
|
61
|
+
This allows specific jobs to be tested, while preventing others from
|
62
|
+
being performed unnecessarily.
|
62
63
|
|
63
|
-
|
64
|
+
Example:
|
64
65
|
|
65
|
-
|
66
|
+
def test_hello_job
|
67
|
+
assert_performed_jobs 1, only: HelloJob do
|
68
|
+
HelloJob.perform_later('jeremy')
|
69
|
+
LoggingJob.perform_later
|
70
|
+
end
|
71
|
+
end
|
66
72
|
|
73
|
+
An array may also be specified, to support testing multiple jobs.
|
67
74
|
|
68
|
-
|
75
|
+
Example:
|
69
76
|
|
70
|
-
|
71
|
-
|
77
|
+
def test_hello_and_logging_jobs
|
78
|
+
assert_nothing_raised do
|
79
|
+
assert_performed_jobs 2, only: [HelloJob, LoggingJob] do
|
80
|
+
HelloJob.perform_later('jeremy')
|
81
|
+
LoggingJob.perform_later('stewie')
|
82
|
+
RescueJob.perform_later('david')
|
83
|
+
end
|
84
|
+
end
|
85
|
+
end
|
72
86
|
|
73
|
-
Fixes #
|
87
|
+
Fixes #18802.
|
74
88
|
|
75
|
-
*
|
89
|
+
*Michael Ryan*
|
76
90
|
|
91
|
+
* Allow keyword arguments to be used with Active Job.
|
77
92
|
|
78
|
-
|
93
|
+
Fixes #18741.
|
79
94
|
|
80
|
-
*
|
81
|
-
given number as expected value. This makes the error message much easier to
|
82
|
-
understand.
|
95
|
+
*Sean Griffin*
|
83
96
|
|
84
|
-
|
97
|
+
* Add `:only` option to `assert_enqueued_jobs`, to check the number of times
|
98
|
+
a specific kind of job is enqueued.
|
85
99
|
|
100
|
+
Example:
|
86
101
|
|
87
|
-
|
102
|
+
def test_logging_job
|
103
|
+
assert_enqueued_jobs 1, only: LoggingJob do
|
104
|
+
LoggingJob.perform_later
|
105
|
+
HelloJob.perform_later('jeremy')
|
106
|
+
end
|
107
|
+
end
|
88
108
|
|
89
|
-
*
|
109
|
+
*George Claghorn*
|
90
110
|
|
111
|
+
* `ActiveJob::Base.deserialize` delegates to the job class.
|
91
112
|
|
92
|
-
|
113
|
+
Since `ActiveJob::Base#deserialize` can be overridden by subclasses (like
|
114
|
+
`ActiveJob::Base#serialize`) this allows jobs to attach arbitrary metadata
|
115
|
+
when they get serialized and read it back when they get performed.
|
93
116
|
|
94
|
-
|
117
|
+
Example:
|
95
118
|
|
96
|
-
|
119
|
+
class DeliverWebhookJob < ActiveJob::Base
|
120
|
+
def serialize
|
121
|
+
super.merge('attempt_number' => (@attempt_number || 0) + 1)
|
122
|
+
end
|
97
123
|
|
98
|
-
|
124
|
+
def deserialize(job_data)
|
125
|
+
super
|
126
|
+
@attempt_number = job_data['attempt_number']
|
127
|
+
end
|
99
128
|
|
129
|
+
rescue_from(TimeoutError) do |exception|
|
130
|
+
raise exception if @attempt_number > 5
|
131
|
+
retry_job(wait: 10)
|
132
|
+
end
|
133
|
+
end
|
100
134
|
|
101
|
-
|
135
|
+
*Isaac Seymour*
|
102
136
|
|
103
|
-
|
137
|
+
Please check [4-2-stable](https://github.com/rails/rails/blob/4-2-stable/activejob/CHANGELOG.md) for previous changes.
|
data/MIT-LICENSE
CHANGED
data/README.md
CHANGED
@@ -7,7 +7,7 @@ small units of work and run in parallel, really.
|
|
7
7
|
|
8
8
|
It also serves as the backend for Action Mailer's #deliver_later functionality
|
9
9
|
that makes it easy to turn any mailing into a job for running later. That's
|
10
|
-
one of the most common jobs in a modern web application:
|
10
|
+
one of the most common jobs in a modern web application: sending emails outside
|
11
11
|
of the request-response cycle, so the user doesn't have to wait on it.
|
12
12
|
|
13
13
|
The main point is to ensure that all Rails apps will have a job infrastructure
|
@@ -44,7 +44,7 @@ end
|
|
44
44
|
Enqueue a job like so:
|
45
45
|
|
46
46
|
```ruby
|
47
|
-
MyJob.perform_later record # Enqueue a job to be performed as soon the queueing system is free.
|
47
|
+
MyJob.perform_later record # Enqueue a job to be performed as soon as the queueing system is free.
|
48
48
|
```
|
49
49
|
|
50
50
|
```ruby
|
@@ -102,12 +102,12 @@ see the API Documentation for [ActiveJob::QueueAdapters](http://api.rubyonrails.
|
|
102
102
|
The latest version of Active Job can be installed with RubyGems:
|
103
103
|
|
104
104
|
```
|
105
|
-
|
105
|
+
$ gem install activejob
|
106
106
|
```
|
107
107
|
|
108
108
|
Source code can be downloaded as part of the Rails project on GitHub
|
109
109
|
|
110
|
-
* https://github.com/rails/rails/tree/
|
110
|
+
* https://github.com/rails/rails/tree/master/activejob
|
111
111
|
|
112
112
|
## License
|
113
113
|
|
@@ -118,7 +118,7 @@ Active Job is released under the MIT license:
|
|
118
118
|
|
119
119
|
## Support
|
120
120
|
|
121
|
-
API documentation is at
|
121
|
+
API documentation is at:
|
122
122
|
|
123
123
|
* http://api.rubyonrails.org
|
124
124
|
|
data/lib/active_job.rb
CHANGED
@@ -1,5 +1,5 @@
|
|
1
1
|
#--
|
2
|
-
# Copyright (c) 2014 David Heinemeier Hansson
|
2
|
+
# Copyright (c) 2014-2015 David Heinemeier Hansson
|
3
3
|
#
|
4
4
|
# Permission is hereby granted, free of charge, to any person obtaining
|
5
5
|
# a copy of this software and associated documentation files (the
|
@@ -32,6 +32,7 @@ module ActiveJob
|
|
32
32
|
autoload :Base
|
33
33
|
autoload :QueueAdapters
|
34
34
|
autoload :ConfiguredJob
|
35
|
+
autoload :AsyncJob
|
35
36
|
autoload :TestCase
|
36
37
|
autoload :TestHelper
|
37
38
|
end
|
data/lib/active_job/arguments.rb
CHANGED
@@ -3,30 +3,38 @@ require 'active_support/core_ext/hash'
|
|
3
3
|
module ActiveJob
|
4
4
|
# Raised when an exception is raised during job arguments deserialization.
|
5
5
|
#
|
6
|
-
# Wraps the original exception raised as +
|
6
|
+
# Wraps the original exception raised as +cause+.
|
7
7
|
class DeserializationError < StandardError
|
8
|
-
|
8
|
+
def initialize(e = nil) #:nodoc:
|
9
|
+
if e
|
10
|
+
ActiveSupport::Deprecation.warn("Passing #original_exception is deprecated and has no effect. " \
|
11
|
+
"Exceptions will automatically capture the original exception.", caller)
|
12
|
+
end
|
9
13
|
|
10
|
-
|
11
|
-
|
12
|
-
@original_exception = e
|
13
|
-
set_backtrace e.backtrace
|
14
|
+
super("Error while trying to deserialize arguments: #{$!.message}")
|
15
|
+
set_backtrace $!.backtrace
|
14
16
|
end
|
15
|
-
end
|
16
17
|
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
|
22
|
-
|
18
|
+
# The original exception that was raised during deserialization of job
|
19
|
+
# arguments.
|
20
|
+
def original_exception
|
21
|
+
ActiveSupport::Deprecation.warn("#original_exception is deprecated. Use #cause instead.", caller)
|
22
|
+
cause
|
23
|
+
end
|
23
24
|
end
|
24
25
|
|
26
|
+
# Raised when an unsupported argument type is set as a job argument. We
|
27
|
+
# currently support NilClass, Fixnum, Float, String, TrueClass, FalseClass,
|
28
|
+
# Bignum and objects that can be represented as GlobalIDs (ex: Active Record).
|
29
|
+
# Raised if you set the key for a Hash something else than a string or
|
30
|
+
# a symbol. Also raised when trying to serialize an object which can't be
|
31
|
+
# identified with a Global ID - such as an unpersisted Active Record model.
|
32
|
+
class SerializationError < ArgumentError; end
|
33
|
+
|
25
34
|
module Arguments
|
26
35
|
extend self
|
27
36
|
# :nodoc:
|
28
|
-
TYPE_WHITELIST = [ NilClass,
|
29
|
-
TYPE_WHITELIST.push(Fixnum, Bignum) unless 1.class == Integer
|
37
|
+
TYPE_WHITELIST = [ NilClass, Fixnum, Float, String, TrueClass, FalseClass, Bignum ]
|
30
38
|
|
31
39
|
# Serializes a set of arguments. Whitelisted types are returned
|
32
40
|
# as-is. Arrays/Hashes are serialized element by element.
|
@@ -40,13 +48,16 @@ module ActiveJob
|
|
40
48
|
# All other types are deserialized using GlobalID.
|
41
49
|
def deserialize(arguments)
|
42
50
|
arguments.map { |argument| deserialize_argument(argument) }
|
43
|
-
rescue
|
44
|
-
raise DeserializationError
|
51
|
+
rescue
|
52
|
+
raise DeserializationError
|
45
53
|
end
|
46
54
|
|
47
55
|
private
|
56
|
+
# :nodoc:
|
48
57
|
GLOBALID_KEY = '_aj_globalid'.freeze
|
58
|
+
# :nodoc:
|
49
59
|
SYMBOL_KEYS_KEY = '_aj_symbol_keys'.freeze
|
60
|
+
# :nodoc:
|
50
61
|
WITH_INDIFFERENT_ACCESS_KEY = '_aj_hash_with_indifferent_access'.freeze
|
51
62
|
private_constant :GLOBALID_KEY, :SYMBOL_KEYS_KEY, :WITH_INDIFFERENT_ACCESS_KEY
|
52
63
|
|
@@ -55,7 +66,7 @@ module ActiveJob
|
|
55
66
|
when *TYPE_WHITELIST
|
56
67
|
argument
|
57
68
|
when GlobalID::Identification
|
58
|
-
|
69
|
+
convert_to_global_id_hash(argument)
|
59
70
|
when Array
|
60
71
|
argument.map { |arg| serialize_argument(arg) }
|
61
72
|
when ActiveSupport::HashWithIndifferentAccess
|
@@ -75,7 +86,7 @@ module ActiveJob
|
|
75
86
|
def deserialize_argument(argument)
|
76
87
|
case argument
|
77
88
|
when String
|
78
|
-
argument
|
89
|
+
GlobalID::Locator.locate(argument) || argument
|
79
90
|
when *TYPE_WHITELIST
|
80
91
|
argument
|
81
92
|
when Array
|
@@ -115,6 +126,7 @@ module ActiveJob
|
|
115
126
|
result
|
116
127
|
end
|
117
128
|
|
129
|
+
# :nodoc:
|
118
130
|
RESERVED_KEYS = [
|
119
131
|
GLOBALID_KEY, GLOBALID_KEY.to_sym,
|
120
132
|
SYMBOL_KEYS_KEY, SYMBOL_KEYS_KEY.to_sym,
|
@@ -142,5 +154,12 @@ module ActiveJob
|
|
142
154
|
end
|
143
155
|
end
|
144
156
|
end
|
157
|
+
|
158
|
+
def convert_to_global_id_hash(argument)
|
159
|
+
{ GLOBALID_KEY => argument.to_global_id.to_s }
|
160
|
+
rescue URI::GID::MissingModelIdError
|
161
|
+
raise SerializationError, "Unable to serialize #{argument.class} " \
|
162
|
+
"without an id. (Maybe you forgot to call save?)"
|
163
|
+
end
|
145
164
|
end
|
146
165
|
end
|
@@ -0,0 +1,77 @@
|
|
1
|
+
require 'concurrent/map'
|
2
|
+
require 'concurrent/scheduled_task'
|
3
|
+
require 'concurrent/executor/thread_pool_executor'
|
4
|
+
require 'concurrent/utility/processor_counter'
|
5
|
+
|
6
|
+
module ActiveJob
|
7
|
+
# == Active Job Async Job
|
8
|
+
#
|
9
|
+
# When enqueueing jobs with Async Job each job will be executed asynchronously
|
10
|
+
# on a +concurrent-ruby+ thread pool. All job data is retained in memory.
|
11
|
+
# Because job data is not saved to a persistent datastore there is no
|
12
|
+
# additional infrastructure needed and jobs process quickly. The lack of
|
13
|
+
# persistence, however, means that all unprocessed jobs will be lost on
|
14
|
+
# application restart. Therefore in-memory queue adapters are unsuitable for
|
15
|
+
# most production environments but are excellent for development and testing.
|
16
|
+
#
|
17
|
+
# Read more about Concurrent Ruby {here}[https://github.com/ruby-concurrency/concurrent-ruby].
|
18
|
+
#
|
19
|
+
# To use Async Job set the queue_adapter config to +:async+.
|
20
|
+
#
|
21
|
+
# Rails.application.config.active_job.queue_adapter = :async
|
22
|
+
#
|
23
|
+
# Async Job supports job queues specified with +queue_as+. Queues are created
|
24
|
+
# automatically as needed and each has its own thread pool.
|
25
|
+
class AsyncJob
|
26
|
+
|
27
|
+
DEFAULT_EXECUTOR_OPTIONS = {
|
28
|
+
min_threads: [2, Concurrent.processor_count].max,
|
29
|
+
max_threads: Concurrent.processor_count * 10,
|
30
|
+
auto_terminate: true,
|
31
|
+
idletime: 60, # 1 minute
|
32
|
+
max_queue: 0, # unlimited
|
33
|
+
fallback_policy: :caller_runs # shouldn't matter -- 0 max queue
|
34
|
+
}.freeze
|
35
|
+
|
36
|
+
QUEUES = Concurrent::Map.new do |hash, queue_name| #:nodoc:
|
37
|
+
hash.compute_if_absent(queue_name) { ActiveJob::AsyncJob.create_thread_pool }
|
38
|
+
end
|
39
|
+
|
40
|
+
class << self
|
41
|
+
# Forces jobs to process immediately when testing the Active Job gem.
|
42
|
+
# This should only be called from within unit tests.
|
43
|
+
def perform_immediately! #:nodoc:
|
44
|
+
@perform_immediately = true
|
45
|
+
end
|
46
|
+
|
47
|
+
# Allows jobs to run asynchronously when testing the Active Job gem.
|
48
|
+
# This should only be called from within unit tests.
|
49
|
+
def perform_asynchronously! #:nodoc:
|
50
|
+
@perform_immediately = false
|
51
|
+
end
|
52
|
+
|
53
|
+
def create_thread_pool #:nodoc:
|
54
|
+
if @perform_immediately
|
55
|
+
Concurrent::ImmediateExecutor.new
|
56
|
+
else
|
57
|
+
Concurrent::ThreadPoolExecutor.new(DEFAULT_EXECUTOR_OPTIONS)
|
58
|
+
end
|
59
|
+
end
|
60
|
+
|
61
|
+
def enqueue(job_data, queue: 'default') #:nodoc:
|
62
|
+
QUEUES[queue].post(job_data) { |job| ActiveJob::Base.execute(job) }
|
63
|
+
end
|
64
|
+
|
65
|
+
def enqueue_at(job_data, timestamp, queue: 'default') #:nodoc:
|
66
|
+
delay = timestamp - Time.current.to_f
|
67
|
+
if delay > 0
|
68
|
+
Concurrent::ScheduledTask.execute(delay, args: [job_data], executor: QUEUES[queue]) do |job|
|
69
|
+
ActiveJob::Base.execute(job)
|
70
|
+
end
|
71
|
+
else
|
72
|
+
enqueue(job_data, queue: queue)
|
73
|
+
end
|
74
|
+
end
|
75
|
+
end
|
76
|
+
end
|
77
|
+
end
|