backburner 0.1.2 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/CHANGELOG.md +5 -1
- data/HOOKS.md +56 -0
- data/README.md +13 -5
- data/TODO +0 -2
- data/examples/hooked.rb +87 -0
- data/lib/backburner.rb +1 -0
- data/lib/backburner/hooks.rb +51 -0
- data/lib/backburner/job.rb +12 -1
- data/lib/backburner/queue.rb +2 -1
- data/lib/backburner/version.rb +1 -1
- data/lib/backburner/worker.rb +4 -0
- data/test/fixtures/hooked.rb +114 -0
- data/test/fixtures/test_jobs.rb +27 -0
- data/test/hooks_test.rb +90 -0
- data/test/worker_test.rb +75 -28
- metadata +10 -1
data/CHANGELOG.md
CHANGED
data/HOOKS.md
ADDED
@@ -0,0 +1,56 @@
|
|
1
|
+
# Backburner Hooks
|
2
|
+
|
3
|
+
You can customize Backburner or write plugins using its hook API.
|
4
|
+
In many cases you can use a hook rather than mess around with Backburner's internals.
|
5
|
+
|
6
|
+
## Job Hooks
|
7
|
+
|
8
|
+
Hooks are transparently adapted from [Resque](https://github.com/defunkt/resque/blob/master/docs/HOOKS.md), so
|
9
|
+
if you are familiar with their hook API, now you can use nearly the same ones with beanstalkd and backburner!
|
10
|
+
|
11
|
+
There are a variety of hooks available that are triggered during the lifecycle of a job:
|
12
|
+
|
13
|
+
* `before_enqueue`: Called with the job args before a job is placed on the queue.
|
14
|
+
If the hook returns `false`, the job will not be placed on the queue.
|
15
|
+
|
16
|
+
* `after_enqueue`: Called with the job args after a job is placed on the queue.
|
17
|
+
Any exception raised propagates up to the code which queued the job.
|
18
|
+
|
19
|
+
* `before_perform`: Called with the job args before perform. If a hook returns false,
|
20
|
+
the job is aborted. Other exceptions are treated like regular job exceptions.
|
21
|
+
|
22
|
+
* `after_perform`: Called with the job args after it performs. Uncaught
|
23
|
+
exceptions will be treated like regular job exceptions.
|
24
|
+
|
25
|
+
* `around_perform`: Called with the job args. It is expected to yield in order
|
26
|
+
to perform the job (but is not required to do so). It may handle exceptions
|
27
|
+
thrown by perform, but uncaught exceptions will be treated like regular job exceptions.
|
28
|
+
|
29
|
+
* `on_failure`: Called with the exception and job args if any exception occurs
|
30
|
+
while performing the job (or hooks).
|
31
|
+
|
32
|
+
Hooks are just methods prefixed with the hook type. For example:
|
33
|
+
|
34
|
+
```ruby
|
35
|
+
class SomeJob
|
36
|
+
def self.before_perform_log_job(*args)
|
37
|
+
Logger.info "About to perform #{self} with #{args.inspect}"
|
38
|
+
end
|
39
|
+
|
40
|
+
def self.on_failure_bury(e, *args)
|
41
|
+
Logger.info "Performing #{self} caused an exception (#{e})"
|
42
|
+
self.bury
|
43
|
+
end
|
44
|
+
|
45
|
+
def self.perform(*args)
|
46
|
+
# ...
|
47
|
+
end
|
48
|
+
end
|
49
|
+
```
|
50
|
+
|
51
|
+
You can also setup modules to create compose-able and reusable hooks for your jobs.
|
52
|
+
|
53
|
+
## Worker Hooks
|
54
|
+
|
55
|
+
Coming soon. What do you need here? Just let me know!
|
56
|
+
|
data/README.md
CHANGED
@@ -83,20 +83,20 @@ Backburner is extremely simple to setup. Just configure basic settings for backb
|
|
83
83
|
|
84
84
|
```ruby
|
85
85
|
Backburner.configure do |config|
|
86
|
-
config.beanstalk_url
|
87
|
-
config.tube_namespace
|
88
|
-
config.on_error
|
86
|
+
config.beanstalk_url = ["beanstalk://127.0.0.1", "beanstalk://127.0.0.1:11301"]
|
87
|
+
config.tube_namespace = "some.app.production"
|
88
|
+
config.on_error = lambda { |e| puts e }
|
89
89
|
config.max_job_retries = 3 # default 0 retries
|
90
90
|
config.retry_delay = 2 # default 5 seconds
|
91
91
|
config.default_priority = 65536
|
92
92
|
config.respond_timeout = 120
|
93
|
-
config.logger
|
93
|
+
config.logger = Logger.new(STDOUT)
|
94
94
|
end
|
95
95
|
```
|
96
96
|
|
97
97
|
* The `beanstalk_url` supports a string such as 'beanstalk://127.0.0.1' or an array of addresses.
|
98
98
|
* The `tube_namespace` is the prefix used for all tubes related to this backburner queue.
|
99
|
-
* The `on_error` is a callback that gets invoked with the error whenever
|
99
|
+
* The `on_error` is a callback that gets invoked with the error whenever any job in the system fails.
|
100
100
|
* The `max_job_retries` determines how many times to retry a job before burying
|
101
101
|
* The `retry_delay` determines the base time to wait (in secs) between retries
|
102
102
|
* The `logger` is the logger object written to when backburner wants to report info or errors.
|
@@ -231,6 +231,14 @@ Backburner.default_queues << NewsletterJob.queue
|
|
231
231
|
|
232
232
|
The `default_queues` stores the specific list of queues that should be processed by default by a worker.
|
233
233
|
|
234
|
+
### Hooks
|
235
|
+
|
236
|
+
Backburner is highly extensible and can be tailored to your needs by using various hooks that
|
237
|
+
can be triggered across the job processing lifecycle.
|
238
|
+
Often using hooks is much easier then trying to monkey patch the externals.
|
239
|
+
|
240
|
+
Check out [HOOKS.md](HOOKS.md) for a detailed overview on using hooks.
|
241
|
+
|
234
242
|
### Failures
|
235
243
|
|
236
244
|
When a job fails in backburner (usually because an exception was raised), the job will be released
|
data/TODO
CHANGED
@@ -1,6 +1,4 @@
|
|
1
1
|
- Custom front-end in sinatra for viewing beanstalk jobs
|
2
2
|
- Refer to https://github.com/denniskuczynski/beanstalkd_view
|
3
3
|
- Fork jobs to control memory
|
4
|
-
- https://github.com/michaeldwan/stalker/commit/386267690a7c03e11d1a8b7b6f08b7c9c7cd2c0d
|
5
|
-
- Add hooks
|
6
4
|
- https://github.com/michaeldwan/stalker/commit/386267690a7c03e11d1a8b7b6f08b7c9c7cd2c0d
|
data/examples/hooked.rb
ADDED
@@ -0,0 +1,87 @@
|
|
1
|
+
$:.unshift "lib"
|
2
|
+
require 'backburner'
|
3
|
+
|
4
|
+
$fail = 0
|
5
|
+
class User
|
6
|
+
include Backburner::Performable
|
7
|
+
|
8
|
+
# Called with the job args before a job is placed on the queue.
|
9
|
+
# !! If the hook returns `false`, the job will not be placed on the queue.
|
10
|
+
def self.before_enqueue_foo(*args)
|
11
|
+
puts "[before_enqueue] Just about to enqueue #{self} with #{args.inspect}"
|
12
|
+
end
|
13
|
+
|
14
|
+
# Called with the job args after a job is placed on the queue.
|
15
|
+
# !! Any exception raised propagates up to the code which queued the job.
|
16
|
+
def self.after_enqueue_foo(*args)
|
17
|
+
puts "[after_enqueue] Finished enqueuing #{self} with #{args.inspect}"
|
18
|
+
end
|
19
|
+
|
20
|
+
# Called with the job args before perform. If it raises
|
21
|
+
# `Backburner::Job::DontPerform`, the job is aborted. Other exceptions
|
22
|
+
# are treated like regular job exceptions.
|
23
|
+
def self.before_perform_foo(*args)
|
24
|
+
puts "[before_perform] Just about to perform #{self} with #{args.inspect}"
|
25
|
+
end
|
26
|
+
|
27
|
+
# Called with the job args after it performs. Uncaught
|
28
|
+
# exceptions will be treated like regular job exceptions.
|
29
|
+
def self.after_perform_foo(*args)
|
30
|
+
puts "[after_perform] Just finished performing #{self} with #{args.inspect}"
|
31
|
+
end
|
32
|
+
|
33
|
+
# Called with the job args. It is expected to yield in order
|
34
|
+
# to perform the job (but is not required to do so). It may handle exceptions
|
35
|
+
# thrown by perform, but uncaught exceptions will be treated like regular job exceptions.
|
36
|
+
def self.around_perform_bar(*args)
|
37
|
+
puts "[around_perform_bar before] About to perform #{self} with #{args.inspect}"
|
38
|
+
yield
|
39
|
+
puts "[around_perform_bar after] Just after performing #{self} with #{args.inspect}"
|
40
|
+
end
|
41
|
+
|
42
|
+
# Called with the job args. It is expected to yield in order
|
43
|
+
# to perform the job (but is not required to do so). It may handle exceptions
|
44
|
+
# thrown by perform, but uncaught exceptions will be treated like regular job exceptions.
|
45
|
+
def self.around_perform_cat(*args)
|
46
|
+
puts "[around_perform_cat before] About to perform #{self} with #{args.inspect}"
|
47
|
+
yield
|
48
|
+
puts "[around_perform_cat after] Just after performing #{self} with #{args.inspect}"
|
49
|
+
end
|
50
|
+
|
51
|
+
# Called with the job args. It is expected to yield in order
|
52
|
+
# to perform the job (but is not required to do so). It may handle exceptions
|
53
|
+
# thrown by perform, but uncaught exceptions will be treated like regular job exceptions.
|
54
|
+
def self.around_perform_foo(*args)
|
55
|
+
puts "[around_perform_foo before] About to perform #{self} with #{args.inspect}"
|
56
|
+
yield
|
57
|
+
puts "[around_perform_foo after] Just after performing #{self} with #{args.inspect}"
|
58
|
+
end
|
59
|
+
|
60
|
+
# Called with the exception and job args if any exception occurs
|
61
|
+
# while performing the job (or hooks).
|
62
|
+
def self.on_failure_foo(ex, *args)
|
63
|
+
puts "[on_failure] Failure #{ex.inspect} occurred for job #{self} with #{args.inspect}"
|
64
|
+
end
|
65
|
+
|
66
|
+
def self.foo
|
67
|
+
$fail += 1
|
68
|
+
raise "Fail!" if $fail == 1
|
69
|
+
puts "This is the job running successfully!!"
|
70
|
+
end
|
71
|
+
end
|
72
|
+
|
73
|
+
# Configure Backburner
|
74
|
+
Backburner.configure do |config|
|
75
|
+
config.beanstalk_url = "beanstalk://127.0.0.1"
|
76
|
+
config.tube_namespace = "demo.production"
|
77
|
+
config.on_error = lambda { |e| puts "HEY!!! #{e.class}" }
|
78
|
+
config.max_job_retries = 1
|
79
|
+
config.retry_delay = 0
|
80
|
+
end
|
81
|
+
|
82
|
+
# Enqueue tasks
|
83
|
+
User.async.foo
|
84
|
+
|
85
|
+
# Run work
|
86
|
+
# Backburner.default_queues << "user"
|
87
|
+
Backburner.work
|
data/lib/backburner.rb
CHANGED
@@ -0,0 +1,51 @@
|
|
1
|
+
module Backburner
|
2
|
+
module Hooks
|
3
|
+
# Triggers all method hooks that match the given event type with specified arguments.
|
4
|
+
#
|
5
|
+
# @example
|
6
|
+
# invoke_hook_events(:before_enqueue, 'some', 'args')
|
7
|
+
# invoke_hook_events(:after_perform, 5)
|
8
|
+
#
|
9
|
+
def invoke_hook_events(event, *args)
|
10
|
+
res = find_hook_events(event).map { |e| send(e, *args) }
|
11
|
+
return false if res.any? { |result| result == false }
|
12
|
+
res
|
13
|
+
end
|
14
|
+
|
15
|
+
# Triggers all method hooks that match given around event type. Used for 'around' hooks
|
16
|
+
# that stack over the original task cumulatively onto one another.
|
17
|
+
#
|
18
|
+
# The final block will be the one that actually invokes the
|
19
|
+
# original task after calling all other around blocks.
|
20
|
+
#
|
21
|
+
# @example
|
22
|
+
# around_hook_events(:around_perform) { job.perform }
|
23
|
+
#
|
24
|
+
def around_hook_events(event, *args, &block)
|
25
|
+
raise "Please pass a block to hook events!" unless block_given?
|
26
|
+
around_hooks = find_hook_events(event).reverse
|
27
|
+
aggregate_filter = Proc.new { |&blk| blk.call }
|
28
|
+
around_hooks.each do |ah|
|
29
|
+
prior_around_filter = aggregate_filter
|
30
|
+
aggregate_filter = Proc.new do |&blk|
|
31
|
+
method(ah).call(*args) do
|
32
|
+
prior_around_filter.call(&blk)
|
33
|
+
end
|
34
|
+
end
|
35
|
+
end
|
36
|
+
aggregate_filter.call(&block)
|
37
|
+
end
|
38
|
+
|
39
|
+
protected
|
40
|
+
|
41
|
+
# Returns all methods that match given hook type
|
42
|
+
#
|
43
|
+
# @example
|
44
|
+
# find_hook_events(:before_enqueue)
|
45
|
+
# # => ['before_enqueue_foo', 'before_enqueue_bar']
|
46
|
+
#
|
47
|
+
def find_hook_events(event)
|
48
|
+
(self.methods - Object.methods).grep(/^#{event}/).sort
|
49
|
+
end
|
50
|
+
end # Hooks
|
51
|
+
end # Backburner
|
data/lib/backburner/job.rb
CHANGED
@@ -40,8 +40,19 @@ module Backburner
|
|
40
40
|
# @task.process
|
41
41
|
#
|
42
42
|
def process
|
43
|
-
|
43
|
+
# Invoke before hook and stop if false
|
44
|
+
res = job_class.invoke_hook_events(:before_perform, *args)
|
45
|
+
return false unless res
|
46
|
+
# Execute the job
|
47
|
+
job_class.around_hook_events(:around_perform, *args) do
|
48
|
+
timeout_job_after(task.ttr - 1) { job_class.perform(*args) }
|
49
|
+
end
|
44
50
|
task.delete
|
51
|
+
# Invoke after perform hook
|
52
|
+
job_class.invoke_hook_events(:after_perform, *args)
|
53
|
+
rescue => e
|
54
|
+
job_class.invoke_hook_events(:on_failure, e, *args)
|
55
|
+
raise e
|
45
56
|
end
|
46
57
|
|
47
58
|
protected
|
data/lib/backburner/queue.rb
CHANGED
@@ -2,6 +2,7 @@ module Backburner
|
|
2
2
|
module Queue
|
3
3
|
def self.included(base)
|
4
4
|
base.send(:extend, Backburner::Helpers)
|
5
|
+
base.send(:extend, Backburner::Hooks)
|
5
6
|
base.extend ClassMethods
|
6
7
|
Backburner::Worker.known_queue_classes << base
|
7
8
|
end
|
@@ -35,5 +36,5 @@ module Backburner
|
|
35
36
|
end
|
36
37
|
end
|
37
38
|
end # ClassMethods
|
38
|
-
end #
|
39
|
+
end # Queue
|
39
40
|
end # Backburner
|
data/lib/backburner/version.rb
CHANGED
data/lib/backburner/worker.rb
CHANGED
@@ -24,8 +24,12 @@ module Backburner
|
|
24
24
|
delay = [0, opts[:delay].to_i].max
|
25
25
|
ttr = opts[:ttr] || Backburner.configuration.respond_timeout
|
26
26
|
tube = connection.tubes[expand_tube_name(opts[:queue] || job_class)]
|
27
|
+
res = job_class.invoke_hook_events(:before_enqueue, *args)
|
28
|
+
return false unless res # stop if hook is false
|
27
29
|
data = { :class => job_class.name, :args => args }
|
28
30
|
tube.put data.to_json, :pri => pri, :delay => delay, :ttr => ttr
|
31
|
+
job_class.invoke_hook_events(:after_enqueue, *args)
|
32
|
+
return true
|
29
33
|
end
|
30
34
|
|
31
35
|
# Starts processing jobs in the specified tube_names
|
@@ -0,0 +1,114 @@
|
|
1
|
+
$hooked_fail_count = 0
|
2
|
+
class HookFailError < RuntimeError; end
|
3
|
+
|
4
|
+
class HookedObjectBeforeEnqueueFail
|
5
|
+
include Backburner::Performable
|
6
|
+
|
7
|
+
def self.before_enqueue_abe(*args)
|
8
|
+
puts "!!before_enqueue_foo!! #{args.inspect}"
|
9
|
+
end
|
10
|
+
|
11
|
+
def self.before_enqueue_bar(*args)
|
12
|
+
return false
|
13
|
+
end
|
14
|
+
end
|
15
|
+
|
16
|
+
|
17
|
+
class HookedObjectAfterEnqueueFail
|
18
|
+
extend Backburner::Hooks
|
19
|
+
|
20
|
+
def self.after_enqueue_abe(*args)
|
21
|
+
puts "!!after_enqueue_foo!! #{args.inspect}"
|
22
|
+
end
|
23
|
+
|
24
|
+
def self.after_enqueue_bar(*args)
|
25
|
+
raise HookFailError, "Fail HookedObjectAfterEnqueueFail"
|
26
|
+
end
|
27
|
+
end
|
28
|
+
|
29
|
+
class HookedObjectBeforePerformFail
|
30
|
+
include Backburner::Performable
|
31
|
+
|
32
|
+
def self.before_perform_abe(*args)
|
33
|
+
puts "!!before_perform_foo!! #{args.inspect}"
|
34
|
+
end
|
35
|
+
|
36
|
+
def self.before_perform_foo(*args)
|
37
|
+
return false
|
38
|
+
end
|
39
|
+
|
40
|
+
def self.foo(x)
|
41
|
+
puts "Fail ran!!"
|
42
|
+
raise HookFailError, "HookedObjectJobFailure on foo!"
|
43
|
+
end
|
44
|
+
end
|
45
|
+
|
46
|
+
class HookedObjectAfterPerformFail
|
47
|
+
extend Backburner::Hooks
|
48
|
+
|
49
|
+
def self.after_perform_abe(*args)
|
50
|
+
puts "!!after_perform_foo!! #{args.inspect}"
|
51
|
+
end
|
52
|
+
|
53
|
+
def self.after_perform_bar(*args)
|
54
|
+
raise HookFailError, "Fail HookedObjectAfterEnqueueFail"
|
55
|
+
end
|
56
|
+
end
|
57
|
+
|
58
|
+
class HookedObjectJobFailure
|
59
|
+
extend Backburner::Hooks
|
60
|
+
|
61
|
+
def self.foo(x)
|
62
|
+
raise HookFailError, "HookedObjectJobFailure on foo!"
|
63
|
+
end
|
64
|
+
end
|
65
|
+
|
66
|
+
class HookedObjectSuccess
|
67
|
+
include Backburner::Performable
|
68
|
+
|
69
|
+
def self.before_enqueue_foo(*args)
|
70
|
+
puts "!!before_enqueue_foo!! #{args.inspect}"
|
71
|
+
end
|
72
|
+
|
73
|
+
def self.before_enqueue_bar(*args)
|
74
|
+
puts "!!before_enqueue_bar!! #{args.inspect}"
|
75
|
+
end
|
76
|
+
|
77
|
+
def self.after_enqueue_foo(*args)
|
78
|
+
puts "!!after_enqueue_foo!! #{args.inspect}"
|
79
|
+
end
|
80
|
+
|
81
|
+
def self.after_enqueue_bar(*args)
|
82
|
+
puts "!!after_enqueue_bar!! #{args.inspect}"
|
83
|
+
end
|
84
|
+
|
85
|
+
def self.before_perform_foo(*args)
|
86
|
+
puts "!!before_perform_foo!! #{args.inspect}"
|
87
|
+
end
|
88
|
+
|
89
|
+
def self.after_perform_foo(*args)
|
90
|
+
puts "!!after_perform_foo!! #{args.inspect}"
|
91
|
+
end
|
92
|
+
|
93
|
+
def self.around_perform_bar(*args)
|
94
|
+
puts "!!BEGIN around_perform_bar!! #{args.inspect}"
|
95
|
+
yield
|
96
|
+
puts "!!END around_perform_bar!! #{args.inspect}"
|
97
|
+
end
|
98
|
+
|
99
|
+
def self.around_perform_cat(*args)
|
100
|
+
puts "!!BEGIN around_perform_cat!! #{args.inspect}"
|
101
|
+
yield
|
102
|
+
puts "!!END around_perform_cat!! #{args.inspect}"
|
103
|
+
end
|
104
|
+
|
105
|
+
def self.on_failure_foo(ex, *args)
|
106
|
+
puts "!!on_failure_foo!! #{ex.inspect} #{args.inspect}"
|
107
|
+
end
|
108
|
+
|
109
|
+
def self.foo(x)
|
110
|
+
$hooked_fail_count += 1
|
111
|
+
raise HookFailError, "Fail!" if $hooked_fail_count == 1
|
112
|
+
puts "This is the job running successfully!! #{x.inspect}"
|
113
|
+
end
|
114
|
+
end # HookedObjectSuccess
|
@@ -0,0 +1,27 @@
|
|
1
|
+
$worker_test_count = 0
|
2
|
+
$worker_success = false
|
3
|
+
|
4
|
+
class TestJob
|
5
|
+
include Backburner::Queue
|
6
|
+
queue_priority 1000
|
7
|
+
def self.perform(x, y); $worker_test_count += x + y; end
|
8
|
+
end
|
9
|
+
|
10
|
+
class TestFailJob
|
11
|
+
include Backburner::Queue
|
12
|
+
def self.perform(x, y); raise RuntimeError; end
|
13
|
+
end
|
14
|
+
|
15
|
+
class TestRetryJob
|
16
|
+
include Backburner::Queue
|
17
|
+
def self.perform(x, y)
|
18
|
+
$worker_test_count += 1
|
19
|
+
raise RuntimeError unless $worker_test_count > 2
|
20
|
+
$worker_success = true
|
21
|
+
end
|
22
|
+
end
|
23
|
+
|
24
|
+
class TestAsyncJob
|
25
|
+
include Backburner::Performable
|
26
|
+
def self.foo(x, y); $worker_test_count = x * y; end
|
27
|
+
end
|
data/test/hooks_test.rb
ADDED
@@ -0,0 +1,90 @@
|
|
1
|
+
require File.expand_path('../test_helper', __FILE__)
|
2
|
+
require File.expand_path('../fixtures/hooked', __FILE__)
|
3
|
+
|
4
|
+
describe "Backburner::Hooks module" do
|
5
|
+
before do
|
6
|
+
$hooked_fail_count = 0
|
7
|
+
end
|
8
|
+
|
9
|
+
describe "for invoke_hook_events method" do
|
10
|
+
describe "with before_enqueue" do
|
11
|
+
it "should support successful invocation" do
|
12
|
+
out = silenced { @res = HookedObjectSuccess.invoke_hook_events(:before_enqueue, 5, 6) }
|
13
|
+
assert_equal [nil, nil], @res
|
14
|
+
assert_match /!!before_enqueue_foo!! \[5\, 6\]/, out
|
15
|
+
assert_match /!!before_enqueue_bar!! \[5\, 6\]/, out
|
16
|
+
end
|
17
|
+
|
18
|
+
it "should support fail case" do
|
19
|
+
out = silenced { @res = HookedObjectBeforeEnqueueFail.invoke_hook_events(:before_enqueue, 5, 6) }
|
20
|
+
assert_equal false, @res
|
21
|
+
assert_match /!!before_enqueue_foo!! \[5\, 6\]/, out
|
22
|
+
end
|
23
|
+
end # before_enqueue
|
24
|
+
|
25
|
+
describe "with after_enqueue" do
|
26
|
+
it "should support successful invocation" do
|
27
|
+
out = silenced { HookedObjectSuccess.invoke_hook_events(:after_enqueue, 7, 8) }
|
28
|
+
assert_match /!!after_enqueue_foo!! \[7\, 8\]/, out
|
29
|
+
assert_match /!!after_enqueue_bar!! \[7\, 8\]/, out
|
30
|
+
end
|
31
|
+
|
32
|
+
it "should support fail case" do
|
33
|
+
assert_raises(HookFailError) do
|
34
|
+
silenced { @res = HookedObjectAfterEnqueueFail.invoke_hook_events(:after_enqueue, 5, 6) }
|
35
|
+
end
|
36
|
+
end
|
37
|
+
end # after_enqueue
|
38
|
+
|
39
|
+
describe "with before_perform" do
|
40
|
+
it "should support successful invocation" do
|
41
|
+
out = silenced { HookedObjectSuccess.invoke_hook_events(:before_perform, 1, 2) }
|
42
|
+
assert_match /!!before_perform_foo!! \[1\, 2\]/, out
|
43
|
+
end
|
44
|
+
|
45
|
+
it "should support fail case" do
|
46
|
+
out = silenced { @res = HookedObjectBeforePerformFail.invoke_hook_events(:before_perform, 5, 6) }
|
47
|
+
assert_equal false, @res
|
48
|
+
assert_match /!!before_perform_foo!! \[5\, 6\]/, out
|
49
|
+
end
|
50
|
+
end # before_perform
|
51
|
+
|
52
|
+
describe "with after_perform" do
|
53
|
+
it "should support successful invocation" do
|
54
|
+
out = silenced { HookedObjectSuccess.invoke_hook_events(:after_perform, 3, 4) }
|
55
|
+
assert_match /!!after_perform_foo!! \[3\, 4\]/, out
|
56
|
+
end
|
57
|
+
|
58
|
+
it "should support fail case" do
|
59
|
+
assert_raises(HookFailError) do
|
60
|
+
silenced { @res = HookedObjectAfterPerformFail.invoke_hook_events(:after_perform, 5, 6) }
|
61
|
+
end
|
62
|
+
end
|
63
|
+
end # after_perform
|
64
|
+
|
65
|
+
describe "with on_failure" do
|
66
|
+
it "should support successful invocation" do
|
67
|
+
out = silenced { HookedObjectSuccess.invoke_hook_events(:on_failure, RuntimeError, 10) }
|
68
|
+
assert_match /!!on_failure_foo!! RuntimeError \[10\]/, out
|
69
|
+
end
|
70
|
+
end # on_failure
|
71
|
+
end # invoke_hook_events
|
72
|
+
|
73
|
+
describe "for around_hook_events method" do
|
74
|
+
describe "with around_perform" do
|
75
|
+
it "should support successful invocation" do
|
76
|
+
out = silenced do
|
77
|
+
HookedObjectSuccess.around_hook_events(:around_perform, 7, 8) {
|
78
|
+
puts "!!FIRED!!"
|
79
|
+
}
|
80
|
+
end
|
81
|
+
assert_match /BEGIN.*?bar.*BEGIN.*cat.*FIRED.*END.*cat.*END.*bar/m, out
|
82
|
+
assert_match /!!BEGIN around_perform_bar!! \[7\, 8\]/, out
|
83
|
+
assert_match /!!BEGIN around_perform_cat!! \[7\, 8\]/, out
|
84
|
+
assert_match /!!FIRED!!/, out
|
85
|
+
assert_match /!!END around_perform_cat!! \[7\, 8\]/, out
|
86
|
+
assert_match /!!END around_perform_bar!! \[7\, 8\]/, out
|
87
|
+
end
|
88
|
+
end # successful
|
89
|
+
end # around_hook_events
|
90
|
+
end # Hooks
|
data/test/worker_test.rb
CHANGED
@@ -1,32 +1,6 @@
|
|
1
1
|
require File.expand_path('../test_helper', __FILE__)
|
2
|
-
|
3
|
-
|
4
|
-
$worker_success = false
|
5
|
-
|
6
|
-
class TestJob
|
7
|
-
include Backburner::Queue
|
8
|
-
queue_priority 1000
|
9
|
-
def self.perform(x, y); $worker_test_count += x + y; end
|
10
|
-
end
|
11
|
-
|
12
|
-
class TestFailJob
|
13
|
-
include Backburner::Queue
|
14
|
-
def self.perform(x, y); raise RuntimeError; end
|
15
|
-
end
|
16
|
-
|
17
|
-
class TestRetryJob
|
18
|
-
include Backburner::Queue
|
19
|
-
def self.perform(x, y)
|
20
|
-
$worker_test_count += 1
|
21
|
-
raise RuntimeError unless $worker_test_count > 2
|
22
|
-
$worker_success = true
|
23
|
-
end
|
24
|
-
end
|
25
|
-
|
26
|
-
class TestAsyncJob
|
27
|
-
include Backburner::Performable
|
28
|
-
def self.foo(x, y); $worker_test_count = x * y; end
|
29
|
-
end
|
2
|
+
require File.expand_path('../fixtures/test_jobs', __FILE__)
|
3
|
+
require File.expand_path('../fixtures/hooked', __FILE__)
|
30
4
|
|
31
5
|
describe "Backburner::Worker module" do
|
32
6
|
before do
|
@@ -258,6 +232,79 @@ describe "Backburner::Worker module" do
|
|
258
232
|
assert_equal true, $worker_success
|
259
233
|
end # retrying, succeeds
|
260
234
|
|
235
|
+
it "should support event hooks without retry" do
|
236
|
+
$hooked_fail_count = 0
|
237
|
+
clear_jobs!('foo.bar.events')
|
238
|
+
out = silenced(2) do
|
239
|
+
HookedObjectSuccess.async(:queue => 'foo.bar.events').foo(5)
|
240
|
+
worker = Backburner::Worker.new('foo.bar.events')
|
241
|
+
worker.prepare
|
242
|
+
worker.work_one_job
|
243
|
+
end
|
244
|
+
assert_match /before_enqueue.*after_enqueue.*Working 1 queues/m, out
|
245
|
+
assert_match /!!before_enqueue_bar!! \[nil, :foo, 5\]/, out
|
246
|
+
assert_match /!!after_enqueue_bar!! \[nil, :foo, 5\]/, out
|
247
|
+
assert_match /!!before_perform_foo!! \[nil, "foo", 5\]/, out
|
248
|
+
assert_match /!!BEGIN around_perform_bar!! \[nil, "foo", 5\]/, out
|
249
|
+
assert_match /!!BEGIN around_perform_cat!! \[nil, "foo", 5\]/, out
|
250
|
+
assert_match /!!on_failure_foo!!.*HookFailError/, out
|
251
|
+
assert_match /attempt 1 of 1, burying/, out
|
252
|
+
end # event hooks, no retry
|
253
|
+
|
254
|
+
it "should support event hooks with retry" do
|
255
|
+
$hooked_fail_count = 0
|
256
|
+
clear_jobs!('foo.bar.events.retry')
|
257
|
+
Backburner.configure { |config| config.max_job_retries = 1; config.retry_delay = 0 }
|
258
|
+
out = silenced(2) do
|
259
|
+
HookedObjectSuccess.async(:queue => 'foo.bar.events.retry').foo(5)
|
260
|
+
worker = Backburner::Worker.new('foo.bar.events.retry')
|
261
|
+
worker.prepare
|
262
|
+
2.times do
|
263
|
+
worker.work_one_job
|
264
|
+
end
|
265
|
+
end
|
266
|
+
assert_match /before_enqueue.*after_enqueue.*Working 1 queues/m, out
|
267
|
+
assert_match /!!before_enqueue_bar!! \[nil, :foo, 5\]/, out
|
268
|
+
assert_match /!!after_enqueue_bar!! \[nil, :foo, 5\]/, out
|
269
|
+
assert_match /!!before_perform_foo!! \[nil, "foo", 5\]/, out
|
270
|
+
assert_match /!!BEGIN around_perform_bar!! \[nil, "foo", 5\]/, out
|
271
|
+
assert_match /!!BEGIN around_perform_cat!! \[nil, "foo", 5\]/, out
|
272
|
+
assert_match /!!on_failure_foo!!.*HookFailError/, out
|
273
|
+
assert_match /!!on_failure_foo!!.*retrying.*around_perform_bar.*around_perform_cat/m, out
|
274
|
+
assert_match /attempt 1 of 2, retrying/, out
|
275
|
+
assert_match /!!before_perform_foo!! \[nil, "foo", 5\]/, out
|
276
|
+
assert_match /!!END around_perform_bar!! \[nil, "foo", 5\]/, out
|
277
|
+
assert_match /!!END around_perform_cat!! \[nil, "foo", 5\]/, out
|
278
|
+
assert_match /!!after_perform_foo!! \[nil, "foo", 5\]/, out
|
279
|
+
assert_match /Finished HookedObjectSuccess/, out
|
280
|
+
end # event hooks, with retry
|
281
|
+
|
282
|
+
it "should support event hooks with stopping enqueue" do
|
283
|
+
$hooked_fail_count = 0
|
284
|
+
clear_jobs!('foo.bar.events.retry2')
|
285
|
+
out = silenced(2) do
|
286
|
+
HookedObjectBeforeEnqueueFail.async(:queue => 'foo.bar.events.retry2').foo(5)
|
287
|
+
end
|
288
|
+
expanded_tube = [Backburner.configuration.tube_namespace, 'foo.bar.events.retry2'].join(".")
|
289
|
+
assert_nil Backburner::Worker.connection.tubes[expanded_tube].peek(:ready)
|
290
|
+
end # stopping enqueue
|
291
|
+
|
292
|
+
it "should support event hooks with stopping perform" do
|
293
|
+
$hooked_fail_count = 0
|
294
|
+
clear_jobs!('foo.bar.events.retry3')
|
295
|
+
expanded_tube = [Backburner.configuration.tube_namespace, 'foo.bar.events.retry3'].join(".")
|
296
|
+
out = silenced(2) do
|
297
|
+
HookedObjectBeforePerformFail.async(:queue => 'foo.bar.events.retry3').foo(10)
|
298
|
+
worker = Backburner::Worker.new('foo.bar.events.retry3')
|
299
|
+
worker.prepare
|
300
|
+
worker.work_one_job
|
301
|
+
end
|
302
|
+
assert_match /!!before_perform_foo!! \[nil, "foo", 10\]/, out
|
303
|
+
assert_match /before_perform_foo.*Finished/m, out
|
304
|
+
refute_match(/Fail ran!!/, out)
|
305
|
+
refute_match(/HookFailError/, out)
|
306
|
+
end # stopping perform
|
307
|
+
|
261
308
|
after do
|
262
309
|
Backburner.configure { |config| config.max_job_retries = 0; config.retry_delay = 5 }
|
263
310
|
end
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: backburner
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.2.0
|
5
5
|
prerelease:
|
6
6
|
platform: ruby
|
7
7
|
authors:
|
@@ -103,6 +103,7 @@ files:
|
|
103
103
|
- .travis.yml
|
104
104
|
- CHANGELOG.md
|
105
105
|
- Gemfile
|
106
|
+
- HOOKS.md
|
106
107
|
- LICENSE
|
107
108
|
- README.md
|
108
109
|
- Rakefile
|
@@ -112,6 +113,7 @@ files:
|
|
112
113
|
- examples/custom.rb
|
113
114
|
- examples/demo.rb
|
114
115
|
- examples/god.rb
|
116
|
+
- examples/hooked.rb
|
115
117
|
- examples/retried.rb
|
116
118
|
- examples/simple.rb
|
117
119
|
- lib/backburner.rb
|
@@ -119,6 +121,7 @@ files:
|
|
119
121
|
- lib/backburner/configuration.rb
|
120
122
|
- lib/backburner/connection.rb
|
121
123
|
- lib/backburner/helpers.rb
|
124
|
+
- lib/backburner/hooks.rb
|
122
125
|
- lib/backburner/job.rb
|
123
126
|
- lib/backburner/logger.rb
|
124
127
|
- lib/backburner/performable.rb
|
@@ -128,7 +131,10 @@ files:
|
|
128
131
|
- lib/backburner/worker.rb
|
129
132
|
- test/back_burner_test.rb
|
130
133
|
- test/connection_test.rb
|
134
|
+
- test/fixtures/hooked.rb
|
135
|
+
- test/fixtures/test_jobs.rb
|
131
136
|
- test/helpers_test.rb
|
137
|
+
- test/hooks_test.rb
|
132
138
|
- test/job_test.rb
|
133
139
|
- test/logger_test.rb
|
134
140
|
- test/performable_test.rb
|
@@ -162,7 +168,10 @@ summary: Reliable beanstalk background job processing made easy for Ruby and Sin
|
|
162
168
|
test_files:
|
163
169
|
- test/back_burner_test.rb
|
164
170
|
- test/connection_test.rb
|
171
|
+
- test/fixtures/hooked.rb
|
172
|
+
- test/fixtures/test_jobs.rb
|
165
173
|
- test/helpers_test.rb
|
174
|
+
- test/hooks_test.rb
|
166
175
|
- test/job_test.rb
|
167
176
|
- test/logger_test.rb
|
168
177
|
- test/performable_test.rb
|