backburner 0.1.2 → 0.2.0
Sign up to get free protection for your applications and to get access to all the features.
- data/CHANGELOG.md +5 -1
- data/HOOKS.md +56 -0
- data/README.md +13 -5
- data/TODO +0 -2
- data/examples/hooked.rb +87 -0
- data/lib/backburner.rb +1 -0
- data/lib/backburner/hooks.rb +51 -0
- data/lib/backburner/job.rb +12 -1
- data/lib/backburner/queue.rb +2 -1
- data/lib/backburner/version.rb +1 -1
- data/lib/backburner/worker.rb +4 -0
- data/test/fixtures/hooked.rb +114 -0
- data/test/fixtures/test_jobs.rb +27 -0
- data/test/hooks_test.rb +90 -0
- data/test/worker_test.rb +75 -28
- metadata +10 -1
data/CHANGELOG.md
CHANGED
data/HOOKS.md
ADDED
@@ -0,0 +1,56 @@
|
|
1
|
+
# Backburner Hooks
|
2
|
+
|
3
|
+
You can customize Backburner or write plugins using its hook API.
|
4
|
+
In many cases you can use a hook rather than mess around with Backburner's internals.
|
5
|
+
|
6
|
+
## Job Hooks
|
7
|
+
|
8
|
+
Hooks are transparently adapted from [Resque](https://github.com/defunkt/resque/blob/master/docs/HOOKS.md), so
|
9
|
+
if you are familiar with their hook API, now you can use nearly the same ones with beanstalkd and backburner!
|
10
|
+
|
11
|
+
There are a variety of hooks available that are triggered during the lifecycle of a job:
|
12
|
+
|
13
|
+
* `before_enqueue`: Called with the job args before a job is placed on the queue.
|
14
|
+
If the hook returns `false`, the job will not be placed on the queue.
|
15
|
+
|
16
|
+
* `after_enqueue`: Called with the job args after a job is placed on the queue.
|
17
|
+
Any exception raised propagates up to the code which queued the job.
|
18
|
+
|
19
|
+
* `before_perform`: Called with the job args before perform. If a hook returns false,
|
20
|
+
the job is aborted. Other exceptions are treated like regular job exceptions.
|
21
|
+
|
22
|
+
* `after_perform`: Called with the job args after it performs. Uncaught
|
23
|
+
exceptions will be treated like regular job exceptions.
|
24
|
+
|
25
|
+
* `around_perform`: Called with the job args. It is expected to yield in order
|
26
|
+
to perform the job (but is not required to do so). It may handle exceptions
|
27
|
+
thrown by perform, but uncaught exceptions will be treated like regular job exceptions.
|
28
|
+
|
29
|
+
* `on_failure`: Called with the exception and job args if any exception occurs
|
30
|
+
while performing the job (or hooks).
|
31
|
+
|
32
|
+
Hooks are just methods prefixed with the hook type. For example:
|
33
|
+
|
34
|
+
```ruby
|
35
|
+
class SomeJob
|
36
|
+
def self.before_perform_log_job(*args)
|
37
|
+
Logger.info "About to perform #{self} with #{args.inspect}"
|
38
|
+
end
|
39
|
+
|
40
|
+
def self.on_failure_bury(e, *args)
|
41
|
+
Logger.info "Performing #{self} caused an exception (#{e})"
|
42
|
+
self.bury
|
43
|
+
end
|
44
|
+
|
45
|
+
def self.perform(*args)
|
46
|
+
# ...
|
47
|
+
end
|
48
|
+
end
|
49
|
+
```
|
50
|
+
|
51
|
+
You can also setup modules to create compose-able and reusable hooks for your jobs.
|
52
|
+
|
53
|
+
## Worker Hooks
|
54
|
+
|
55
|
+
Coming soon. What do you need here? Just let me know!
|
56
|
+
|
data/README.md
CHANGED
@@ -83,20 +83,20 @@ Backburner is extremely simple to setup. Just configure basic settings for backb
|
|
83
83
|
|
84
84
|
```ruby
|
85
85
|
Backburner.configure do |config|
|
86
|
-
config.beanstalk_url
|
87
|
-
config.tube_namespace
|
88
|
-
config.on_error
|
86
|
+
config.beanstalk_url = ["beanstalk://127.0.0.1", "beanstalk://127.0.0.1:11301"]
|
87
|
+
config.tube_namespace = "some.app.production"
|
88
|
+
config.on_error = lambda { |e| puts e }
|
89
89
|
config.max_job_retries = 3 # default 0 retries
|
90
90
|
config.retry_delay = 2 # default 5 seconds
|
91
91
|
config.default_priority = 65536
|
92
92
|
config.respond_timeout = 120
|
93
|
-
config.logger
|
93
|
+
config.logger = Logger.new(STDOUT)
|
94
94
|
end
|
95
95
|
```
|
96
96
|
|
97
97
|
* The `beanstalk_url` supports a string such as 'beanstalk://127.0.0.1' or an array of addresses.
|
98
98
|
* The `tube_namespace` is the prefix used for all tubes related to this backburner queue.
|
99
|
-
* The `on_error` is a callback that gets invoked with the error whenever
|
99
|
+
* The `on_error` is a callback that gets invoked with the error whenever any job in the system fails.
|
100
100
|
* The `max_job_retries` determines how many times to retry a job before burying
|
101
101
|
* The `retry_delay` determines the base time to wait (in secs) between retries
|
102
102
|
* The `logger` is the logger object written to when backburner wants to report info or errors.
|
@@ -231,6 +231,14 @@ Backburner.default_queues << NewsletterJob.queue
|
|
231
231
|
|
232
232
|
The `default_queues` stores the specific list of queues that should be processed by default by a worker.
|
233
233
|
|
234
|
+
### Hooks
|
235
|
+
|
236
|
+
Backburner is highly extensible and can be tailored to your needs by using various hooks that
|
237
|
+
can be triggered across the job processing lifecycle.
|
238
|
+
Often using hooks is much easier then trying to monkey patch the externals.
|
239
|
+
|
240
|
+
Check out [HOOKS.md](HOOKS.md) for a detailed overview on using hooks.
|
241
|
+
|
234
242
|
### Failures
|
235
243
|
|
236
244
|
When a job fails in backburner (usually because an exception was raised), the job will be released
|
data/TODO
CHANGED
@@ -1,6 +1,4 @@
|
|
1
1
|
- Custom front-end in sinatra for viewing beanstalk jobs
|
2
2
|
- Refer to https://github.com/denniskuczynski/beanstalkd_view
|
3
3
|
- Fork jobs to control memory
|
4
|
-
- https://github.com/michaeldwan/stalker/commit/386267690a7c03e11d1a8b7b6f08b7c9c7cd2c0d
|
5
|
-
- Add hooks
|
6
4
|
- https://github.com/michaeldwan/stalker/commit/386267690a7c03e11d1a8b7b6f08b7c9c7cd2c0d
|
data/examples/hooked.rb
ADDED
@@ -0,0 +1,87 @@
|
|
1
|
+
$:.unshift "lib"
|
2
|
+
require 'backburner'
|
3
|
+
|
4
|
+
$fail = 0
|
5
|
+
class User
|
6
|
+
include Backburner::Performable
|
7
|
+
|
8
|
+
# Called with the job args before a job is placed on the queue.
|
9
|
+
# !! If the hook returns `false`, the job will not be placed on the queue.
|
10
|
+
def self.before_enqueue_foo(*args)
|
11
|
+
puts "[before_enqueue] Just about to enqueue #{self} with #{args.inspect}"
|
12
|
+
end
|
13
|
+
|
14
|
+
# Called with the job args after a job is placed on the queue.
|
15
|
+
# !! Any exception raised propagates up to the code which queued the job.
|
16
|
+
def self.after_enqueue_foo(*args)
|
17
|
+
puts "[after_enqueue] Finished enqueuing #{self} with #{args.inspect}"
|
18
|
+
end
|
19
|
+
|
20
|
+
# Called with the job args before perform. If it raises
|
21
|
+
# `Backburner::Job::DontPerform`, the job is aborted. Other exceptions
|
22
|
+
# are treated like regular job exceptions.
|
23
|
+
def self.before_perform_foo(*args)
|
24
|
+
puts "[before_perform] Just about to perform #{self} with #{args.inspect}"
|
25
|
+
end
|
26
|
+
|
27
|
+
# Called with the job args after it performs. Uncaught
|
28
|
+
# exceptions will be treated like regular job exceptions.
|
29
|
+
def self.after_perform_foo(*args)
|
30
|
+
puts "[after_perform] Just finished performing #{self} with #{args.inspect}"
|
31
|
+
end
|
32
|
+
|
33
|
+
# Called with the job args. It is expected to yield in order
|
34
|
+
# to perform the job (but is not required to do so). It may handle exceptions
|
35
|
+
# thrown by perform, but uncaught exceptions will be treated like regular job exceptions.
|
36
|
+
def self.around_perform_bar(*args)
|
37
|
+
puts "[around_perform_bar before] About to perform #{self} with #{args.inspect}"
|
38
|
+
yield
|
39
|
+
puts "[around_perform_bar after] Just after performing #{self} with #{args.inspect}"
|
40
|
+
end
|
41
|
+
|
42
|
+
# Called with the job args. It is expected to yield in order
|
43
|
+
# to perform the job (but is not required to do so). It may handle exceptions
|
44
|
+
# thrown by perform, but uncaught exceptions will be treated like regular job exceptions.
|
45
|
+
def self.around_perform_cat(*args)
|
46
|
+
puts "[around_perform_cat before] About to perform #{self} with #{args.inspect}"
|
47
|
+
yield
|
48
|
+
puts "[around_perform_cat after] Just after performing #{self} with #{args.inspect}"
|
49
|
+
end
|
50
|
+
|
51
|
+
# Called with the job args. It is expected to yield in order
|
52
|
+
# to perform the job (but is not required to do so). It may handle exceptions
|
53
|
+
# thrown by perform, but uncaught exceptions will be treated like regular job exceptions.
|
54
|
+
def self.around_perform_foo(*args)
|
55
|
+
puts "[around_perform_foo before] About to perform #{self} with #{args.inspect}"
|
56
|
+
yield
|
57
|
+
puts "[around_perform_foo after] Just after performing #{self} with #{args.inspect}"
|
58
|
+
end
|
59
|
+
|
60
|
+
# Called with the exception and job args if any exception occurs
|
61
|
+
# while performing the job (or hooks).
|
62
|
+
def self.on_failure_foo(ex, *args)
|
63
|
+
puts "[on_failure] Failure #{ex.inspect} occurred for job #{self} with #{args.inspect}"
|
64
|
+
end
|
65
|
+
|
66
|
+
def self.foo
|
67
|
+
$fail += 1
|
68
|
+
raise "Fail!" if $fail == 1
|
69
|
+
puts "This is the job running successfully!!"
|
70
|
+
end
|
71
|
+
end
|
72
|
+
|
73
|
+
# Configure Backburner
|
74
|
+
Backburner.configure do |config|
|
75
|
+
config.beanstalk_url = "beanstalk://127.0.0.1"
|
76
|
+
config.tube_namespace = "demo.production"
|
77
|
+
config.on_error = lambda { |e| puts "HEY!!! #{e.class}" }
|
78
|
+
config.max_job_retries = 1
|
79
|
+
config.retry_delay = 0
|
80
|
+
end
|
81
|
+
|
82
|
+
# Enqueue tasks
|
83
|
+
User.async.foo
|
84
|
+
|
85
|
+
# Run work
|
86
|
+
# Backburner.default_queues << "user"
|
87
|
+
Backburner.work
|
data/lib/backburner.rb
CHANGED
@@ -0,0 +1,51 @@
|
|
1
|
+
module Backburner
|
2
|
+
module Hooks
|
3
|
+
# Triggers all method hooks that match the given event type with specified arguments.
|
4
|
+
#
|
5
|
+
# @example
|
6
|
+
# invoke_hook_events(:before_enqueue, 'some', 'args')
|
7
|
+
# invoke_hook_events(:after_perform, 5)
|
8
|
+
#
|
9
|
+
def invoke_hook_events(event, *args)
|
10
|
+
res = find_hook_events(event).map { |e| send(e, *args) }
|
11
|
+
return false if res.any? { |result| result == false }
|
12
|
+
res
|
13
|
+
end
|
14
|
+
|
15
|
+
# Triggers all method hooks that match given around event type. Used for 'around' hooks
|
16
|
+
# that stack over the original task cumulatively onto one another.
|
17
|
+
#
|
18
|
+
# The final block will be the one that actually invokes the
|
19
|
+
# original task after calling all other around blocks.
|
20
|
+
#
|
21
|
+
# @example
|
22
|
+
# around_hook_events(:around_perform) { job.perform }
|
23
|
+
#
|
24
|
+
def around_hook_events(event, *args, &block)
|
25
|
+
raise "Please pass a block to hook events!" unless block_given?
|
26
|
+
around_hooks = find_hook_events(event).reverse
|
27
|
+
aggregate_filter = Proc.new { |&blk| blk.call }
|
28
|
+
around_hooks.each do |ah|
|
29
|
+
prior_around_filter = aggregate_filter
|
30
|
+
aggregate_filter = Proc.new do |&blk|
|
31
|
+
method(ah).call(*args) do
|
32
|
+
prior_around_filter.call(&blk)
|
33
|
+
end
|
34
|
+
end
|
35
|
+
end
|
36
|
+
aggregate_filter.call(&block)
|
37
|
+
end
|
38
|
+
|
39
|
+
protected
|
40
|
+
|
41
|
+
# Returns all methods that match given hook type
|
42
|
+
#
|
43
|
+
# @example
|
44
|
+
# find_hook_events(:before_enqueue)
|
45
|
+
# # => ['before_enqueue_foo', 'before_enqueue_bar']
|
46
|
+
#
|
47
|
+
def find_hook_events(event)
|
48
|
+
(self.methods - Object.methods).grep(/^#{event}/).sort
|
49
|
+
end
|
50
|
+
end # Hooks
|
51
|
+
end # Backburner
|
data/lib/backburner/job.rb
CHANGED
@@ -40,8 +40,19 @@ module Backburner
|
|
40
40
|
# @task.process
|
41
41
|
#
|
42
42
|
def process
|
43
|
-
|
43
|
+
# Invoke before hook and stop if false
|
44
|
+
res = job_class.invoke_hook_events(:before_perform, *args)
|
45
|
+
return false unless res
|
46
|
+
# Execute the job
|
47
|
+
job_class.around_hook_events(:around_perform, *args) do
|
48
|
+
timeout_job_after(task.ttr - 1) { job_class.perform(*args) }
|
49
|
+
end
|
44
50
|
task.delete
|
51
|
+
# Invoke after perform hook
|
52
|
+
job_class.invoke_hook_events(:after_perform, *args)
|
53
|
+
rescue => e
|
54
|
+
job_class.invoke_hook_events(:on_failure, e, *args)
|
55
|
+
raise e
|
45
56
|
end
|
46
57
|
|
47
58
|
protected
|
data/lib/backburner/queue.rb
CHANGED
@@ -2,6 +2,7 @@ module Backburner
|
|
2
2
|
module Queue
|
3
3
|
def self.included(base)
|
4
4
|
base.send(:extend, Backburner::Helpers)
|
5
|
+
base.send(:extend, Backburner::Hooks)
|
5
6
|
base.extend ClassMethods
|
6
7
|
Backburner::Worker.known_queue_classes << base
|
7
8
|
end
|
@@ -35,5 +36,5 @@ module Backburner
|
|
35
36
|
end
|
36
37
|
end
|
37
38
|
end # ClassMethods
|
38
|
-
end #
|
39
|
+
end # Queue
|
39
40
|
end # Backburner
|
data/lib/backburner/version.rb
CHANGED
data/lib/backburner/worker.rb
CHANGED
@@ -24,8 +24,12 @@ module Backburner
|
|
24
24
|
delay = [0, opts[:delay].to_i].max
|
25
25
|
ttr = opts[:ttr] || Backburner.configuration.respond_timeout
|
26
26
|
tube = connection.tubes[expand_tube_name(opts[:queue] || job_class)]
|
27
|
+
res = job_class.invoke_hook_events(:before_enqueue, *args)
|
28
|
+
return false unless res # stop if hook is false
|
27
29
|
data = { :class => job_class.name, :args => args }
|
28
30
|
tube.put data.to_json, :pri => pri, :delay => delay, :ttr => ttr
|
31
|
+
job_class.invoke_hook_events(:after_enqueue, *args)
|
32
|
+
return true
|
29
33
|
end
|
30
34
|
|
31
35
|
# Starts processing jobs in the specified tube_names
|
@@ -0,0 +1,114 @@
|
|
1
|
+
$hooked_fail_count = 0
|
2
|
+
class HookFailError < RuntimeError; end
|
3
|
+
|
4
|
+
class HookedObjectBeforeEnqueueFail
|
5
|
+
include Backburner::Performable
|
6
|
+
|
7
|
+
def self.before_enqueue_abe(*args)
|
8
|
+
puts "!!before_enqueue_foo!! #{args.inspect}"
|
9
|
+
end
|
10
|
+
|
11
|
+
def self.before_enqueue_bar(*args)
|
12
|
+
return false
|
13
|
+
end
|
14
|
+
end
|
15
|
+
|
16
|
+
|
17
|
+
class HookedObjectAfterEnqueueFail
|
18
|
+
extend Backburner::Hooks
|
19
|
+
|
20
|
+
def self.after_enqueue_abe(*args)
|
21
|
+
puts "!!after_enqueue_foo!! #{args.inspect}"
|
22
|
+
end
|
23
|
+
|
24
|
+
def self.after_enqueue_bar(*args)
|
25
|
+
raise HookFailError, "Fail HookedObjectAfterEnqueueFail"
|
26
|
+
end
|
27
|
+
end
|
28
|
+
|
29
|
+
class HookedObjectBeforePerformFail
|
30
|
+
include Backburner::Performable
|
31
|
+
|
32
|
+
def self.before_perform_abe(*args)
|
33
|
+
puts "!!before_perform_foo!! #{args.inspect}"
|
34
|
+
end
|
35
|
+
|
36
|
+
def self.before_perform_foo(*args)
|
37
|
+
return false
|
38
|
+
end
|
39
|
+
|
40
|
+
def self.foo(x)
|
41
|
+
puts "Fail ran!!"
|
42
|
+
raise HookFailError, "HookedObjectJobFailure on foo!"
|
43
|
+
end
|
44
|
+
end
|
45
|
+
|
46
|
+
class HookedObjectAfterPerformFail
|
47
|
+
extend Backburner::Hooks
|
48
|
+
|
49
|
+
def self.after_perform_abe(*args)
|
50
|
+
puts "!!after_perform_foo!! #{args.inspect}"
|
51
|
+
end
|
52
|
+
|
53
|
+
def self.after_perform_bar(*args)
|
54
|
+
raise HookFailError, "Fail HookedObjectAfterEnqueueFail"
|
55
|
+
end
|
56
|
+
end
|
57
|
+
|
58
|
+
class HookedObjectJobFailure
|
59
|
+
extend Backburner::Hooks
|
60
|
+
|
61
|
+
def self.foo(x)
|
62
|
+
raise HookFailError, "HookedObjectJobFailure on foo!"
|
63
|
+
end
|
64
|
+
end
|
65
|
+
|
66
|
+
class HookedObjectSuccess
|
67
|
+
include Backburner::Performable
|
68
|
+
|
69
|
+
def self.before_enqueue_foo(*args)
|
70
|
+
puts "!!before_enqueue_foo!! #{args.inspect}"
|
71
|
+
end
|
72
|
+
|
73
|
+
def self.before_enqueue_bar(*args)
|
74
|
+
puts "!!before_enqueue_bar!! #{args.inspect}"
|
75
|
+
end
|
76
|
+
|
77
|
+
def self.after_enqueue_foo(*args)
|
78
|
+
puts "!!after_enqueue_foo!! #{args.inspect}"
|
79
|
+
end
|
80
|
+
|
81
|
+
def self.after_enqueue_bar(*args)
|
82
|
+
puts "!!after_enqueue_bar!! #{args.inspect}"
|
83
|
+
end
|
84
|
+
|
85
|
+
def self.before_perform_foo(*args)
|
86
|
+
puts "!!before_perform_foo!! #{args.inspect}"
|
87
|
+
end
|
88
|
+
|
89
|
+
def self.after_perform_foo(*args)
|
90
|
+
puts "!!after_perform_foo!! #{args.inspect}"
|
91
|
+
end
|
92
|
+
|
93
|
+
def self.around_perform_bar(*args)
|
94
|
+
puts "!!BEGIN around_perform_bar!! #{args.inspect}"
|
95
|
+
yield
|
96
|
+
puts "!!END around_perform_bar!! #{args.inspect}"
|
97
|
+
end
|
98
|
+
|
99
|
+
def self.around_perform_cat(*args)
|
100
|
+
puts "!!BEGIN around_perform_cat!! #{args.inspect}"
|
101
|
+
yield
|
102
|
+
puts "!!END around_perform_cat!! #{args.inspect}"
|
103
|
+
end
|
104
|
+
|
105
|
+
def self.on_failure_foo(ex, *args)
|
106
|
+
puts "!!on_failure_foo!! #{ex.inspect} #{args.inspect}"
|
107
|
+
end
|
108
|
+
|
109
|
+
def self.foo(x)
|
110
|
+
$hooked_fail_count += 1
|
111
|
+
raise HookFailError, "Fail!" if $hooked_fail_count == 1
|
112
|
+
puts "This is the job running successfully!! #{x.inspect}"
|
113
|
+
end
|
114
|
+
end # HookedObjectSuccess
|
@@ -0,0 +1,27 @@
|
|
1
|
+
$worker_test_count = 0
|
2
|
+
$worker_success = false
|
3
|
+
|
4
|
+
class TestJob
|
5
|
+
include Backburner::Queue
|
6
|
+
queue_priority 1000
|
7
|
+
def self.perform(x, y); $worker_test_count += x + y; end
|
8
|
+
end
|
9
|
+
|
10
|
+
class TestFailJob
|
11
|
+
include Backburner::Queue
|
12
|
+
def self.perform(x, y); raise RuntimeError; end
|
13
|
+
end
|
14
|
+
|
15
|
+
class TestRetryJob
|
16
|
+
include Backburner::Queue
|
17
|
+
def self.perform(x, y)
|
18
|
+
$worker_test_count += 1
|
19
|
+
raise RuntimeError unless $worker_test_count > 2
|
20
|
+
$worker_success = true
|
21
|
+
end
|
22
|
+
end
|
23
|
+
|
24
|
+
class TestAsyncJob
|
25
|
+
include Backburner::Performable
|
26
|
+
def self.foo(x, y); $worker_test_count = x * y; end
|
27
|
+
end
|
data/test/hooks_test.rb
ADDED
@@ -0,0 +1,90 @@
|
|
1
|
+
require File.expand_path('../test_helper', __FILE__)
|
2
|
+
require File.expand_path('../fixtures/hooked', __FILE__)
|
3
|
+
|
4
|
+
describe "Backburner::Hooks module" do
|
5
|
+
before do
|
6
|
+
$hooked_fail_count = 0
|
7
|
+
end
|
8
|
+
|
9
|
+
describe "for invoke_hook_events method" do
|
10
|
+
describe "with before_enqueue" do
|
11
|
+
it "should support successful invocation" do
|
12
|
+
out = silenced { @res = HookedObjectSuccess.invoke_hook_events(:before_enqueue, 5, 6) }
|
13
|
+
assert_equal [nil, nil], @res
|
14
|
+
assert_match /!!before_enqueue_foo!! \[5\, 6\]/, out
|
15
|
+
assert_match /!!before_enqueue_bar!! \[5\, 6\]/, out
|
16
|
+
end
|
17
|
+
|
18
|
+
it "should support fail case" do
|
19
|
+
out = silenced { @res = HookedObjectBeforeEnqueueFail.invoke_hook_events(:before_enqueue, 5, 6) }
|
20
|
+
assert_equal false, @res
|
21
|
+
assert_match /!!before_enqueue_foo!! \[5\, 6\]/, out
|
22
|
+
end
|
23
|
+
end # before_enqueue
|
24
|
+
|
25
|
+
describe "with after_enqueue" do
|
26
|
+
it "should support successful invocation" do
|
27
|
+
out = silenced { HookedObjectSuccess.invoke_hook_events(:after_enqueue, 7, 8) }
|
28
|
+
assert_match /!!after_enqueue_foo!! \[7\, 8\]/, out
|
29
|
+
assert_match /!!after_enqueue_bar!! \[7\, 8\]/, out
|
30
|
+
end
|
31
|
+
|
32
|
+
it "should support fail case" do
|
33
|
+
assert_raises(HookFailError) do
|
34
|
+
silenced { @res = HookedObjectAfterEnqueueFail.invoke_hook_events(:after_enqueue, 5, 6) }
|
35
|
+
end
|
36
|
+
end
|
37
|
+
end # after_enqueue
|
38
|
+
|
39
|
+
describe "with before_perform" do
|
40
|
+
it "should support successful invocation" do
|
41
|
+
out = silenced { HookedObjectSuccess.invoke_hook_events(:before_perform, 1, 2) }
|
42
|
+
assert_match /!!before_perform_foo!! \[1\, 2\]/, out
|
43
|
+
end
|
44
|
+
|
45
|
+
it "should support fail case" do
|
46
|
+
out = silenced { @res = HookedObjectBeforePerformFail.invoke_hook_events(:before_perform, 5, 6) }
|
47
|
+
assert_equal false, @res
|
48
|
+
assert_match /!!before_perform_foo!! \[5\, 6\]/, out
|
49
|
+
end
|
50
|
+
end # before_perform
|
51
|
+
|
52
|
+
describe "with after_perform" do
|
53
|
+
it "should support successful invocation" do
|
54
|
+
out = silenced { HookedObjectSuccess.invoke_hook_events(:after_perform, 3, 4) }
|
55
|
+
assert_match /!!after_perform_foo!! \[3\, 4\]/, out
|
56
|
+
end
|
57
|
+
|
58
|
+
it "should support fail case" do
|
59
|
+
assert_raises(HookFailError) do
|
60
|
+
silenced { @res = HookedObjectAfterPerformFail.invoke_hook_events(:after_perform, 5, 6) }
|
61
|
+
end
|
62
|
+
end
|
63
|
+
end # after_perform
|
64
|
+
|
65
|
+
describe "with on_failure" do
|
66
|
+
it "should support successful invocation" do
|
67
|
+
out = silenced { HookedObjectSuccess.invoke_hook_events(:on_failure, RuntimeError, 10) }
|
68
|
+
assert_match /!!on_failure_foo!! RuntimeError \[10\]/, out
|
69
|
+
end
|
70
|
+
end # on_failure
|
71
|
+
end # invoke_hook_events
|
72
|
+
|
73
|
+
describe "for around_hook_events method" do
|
74
|
+
describe "with around_perform" do
|
75
|
+
it "should support successful invocation" do
|
76
|
+
out = silenced do
|
77
|
+
HookedObjectSuccess.around_hook_events(:around_perform, 7, 8) {
|
78
|
+
puts "!!FIRED!!"
|
79
|
+
}
|
80
|
+
end
|
81
|
+
assert_match /BEGIN.*?bar.*BEGIN.*cat.*FIRED.*END.*cat.*END.*bar/m, out
|
82
|
+
assert_match /!!BEGIN around_perform_bar!! \[7\, 8\]/, out
|
83
|
+
assert_match /!!BEGIN around_perform_cat!! \[7\, 8\]/, out
|
84
|
+
assert_match /!!FIRED!!/, out
|
85
|
+
assert_match /!!END around_perform_cat!! \[7\, 8\]/, out
|
86
|
+
assert_match /!!END around_perform_bar!! \[7\, 8\]/, out
|
87
|
+
end
|
88
|
+
end # successful
|
89
|
+
end # around_hook_events
|
90
|
+
end # Hooks
|
data/test/worker_test.rb
CHANGED
@@ -1,32 +1,6 @@
|
|
1
1
|
require File.expand_path('../test_helper', __FILE__)
|
2
|
-
|
3
|
-
|
4
|
-
$worker_success = false
|
5
|
-
|
6
|
-
class TestJob
|
7
|
-
include Backburner::Queue
|
8
|
-
queue_priority 1000
|
9
|
-
def self.perform(x, y); $worker_test_count += x + y; end
|
10
|
-
end
|
11
|
-
|
12
|
-
class TestFailJob
|
13
|
-
include Backburner::Queue
|
14
|
-
def self.perform(x, y); raise RuntimeError; end
|
15
|
-
end
|
16
|
-
|
17
|
-
class TestRetryJob
|
18
|
-
include Backburner::Queue
|
19
|
-
def self.perform(x, y)
|
20
|
-
$worker_test_count += 1
|
21
|
-
raise RuntimeError unless $worker_test_count > 2
|
22
|
-
$worker_success = true
|
23
|
-
end
|
24
|
-
end
|
25
|
-
|
26
|
-
class TestAsyncJob
|
27
|
-
include Backburner::Performable
|
28
|
-
def self.foo(x, y); $worker_test_count = x * y; end
|
29
|
-
end
|
2
|
+
require File.expand_path('../fixtures/test_jobs', __FILE__)
|
3
|
+
require File.expand_path('../fixtures/hooked', __FILE__)
|
30
4
|
|
31
5
|
describe "Backburner::Worker module" do
|
32
6
|
before do
|
@@ -258,6 +232,79 @@ describe "Backburner::Worker module" do
|
|
258
232
|
assert_equal true, $worker_success
|
259
233
|
end # retrying, succeeds
|
260
234
|
|
235
|
+
it "should support event hooks without retry" do
|
236
|
+
$hooked_fail_count = 0
|
237
|
+
clear_jobs!('foo.bar.events')
|
238
|
+
out = silenced(2) do
|
239
|
+
HookedObjectSuccess.async(:queue => 'foo.bar.events').foo(5)
|
240
|
+
worker = Backburner::Worker.new('foo.bar.events')
|
241
|
+
worker.prepare
|
242
|
+
worker.work_one_job
|
243
|
+
end
|
244
|
+
assert_match /before_enqueue.*after_enqueue.*Working 1 queues/m, out
|
245
|
+
assert_match /!!before_enqueue_bar!! \[nil, :foo, 5\]/, out
|
246
|
+
assert_match /!!after_enqueue_bar!! \[nil, :foo, 5\]/, out
|
247
|
+
assert_match /!!before_perform_foo!! \[nil, "foo", 5\]/, out
|
248
|
+
assert_match /!!BEGIN around_perform_bar!! \[nil, "foo", 5\]/, out
|
249
|
+
assert_match /!!BEGIN around_perform_cat!! \[nil, "foo", 5\]/, out
|
250
|
+
assert_match /!!on_failure_foo!!.*HookFailError/, out
|
251
|
+
assert_match /attempt 1 of 1, burying/, out
|
252
|
+
end # event hooks, no retry
|
253
|
+
|
254
|
+
it "should support event hooks with retry" do
|
255
|
+
$hooked_fail_count = 0
|
256
|
+
clear_jobs!('foo.bar.events.retry')
|
257
|
+
Backburner.configure { |config| config.max_job_retries = 1; config.retry_delay = 0 }
|
258
|
+
out = silenced(2) do
|
259
|
+
HookedObjectSuccess.async(:queue => 'foo.bar.events.retry').foo(5)
|
260
|
+
worker = Backburner::Worker.new('foo.bar.events.retry')
|
261
|
+
worker.prepare
|
262
|
+
2.times do
|
263
|
+
worker.work_one_job
|
264
|
+
end
|
265
|
+
end
|
266
|
+
assert_match /before_enqueue.*after_enqueue.*Working 1 queues/m, out
|
267
|
+
assert_match /!!before_enqueue_bar!! \[nil, :foo, 5\]/, out
|
268
|
+
assert_match /!!after_enqueue_bar!! \[nil, :foo, 5\]/, out
|
269
|
+
assert_match /!!before_perform_foo!! \[nil, "foo", 5\]/, out
|
270
|
+
assert_match /!!BEGIN around_perform_bar!! \[nil, "foo", 5\]/, out
|
271
|
+
assert_match /!!BEGIN around_perform_cat!! \[nil, "foo", 5\]/, out
|
272
|
+
assert_match /!!on_failure_foo!!.*HookFailError/, out
|
273
|
+
assert_match /!!on_failure_foo!!.*retrying.*around_perform_bar.*around_perform_cat/m, out
|
274
|
+
assert_match /attempt 1 of 2, retrying/, out
|
275
|
+
assert_match /!!before_perform_foo!! \[nil, "foo", 5\]/, out
|
276
|
+
assert_match /!!END around_perform_bar!! \[nil, "foo", 5\]/, out
|
277
|
+
assert_match /!!END around_perform_cat!! \[nil, "foo", 5\]/, out
|
278
|
+
assert_match /!!after_perform_foo!! \[nil, "foo", 5\]/, out
|
279
|
+
assert_match /Finished HookedObjectSuccess/, out
|
280
|
+
end # event hooks, with retry
|
281
|
+
|
282
|
+
it "should support event hooks with stopping enqueue" do
|
283
|
+
$hooked_fail_count = 0
|
284
|
+
clear_jobs!('foo.bar.events.retry2')
|
285
|
+
out = silenced(2) do
|
286
|
+
HookedObjectBeforeEnqueueFail.async(:queue => 'foo.bar.events.retry2').foo(5)
|
287
|
+
end
|
288
|
+
expanded_tube = [Backburner.configuration.tube_namespace, 'foo.bar.events.retry2'].join(".")
|
289
|
+
assert_nil Backburner::Worker.connection.tubes[expanded_tube].peek(:ready)
|
290
|
+
end # stopping enqueue
|
291
|
+
|
292
|
+
it "should support event hooks with stopping perform" do
|
293
|
+
$hooked_fail_count = 0
|
294
|
+
clear_jobs!('foo.bar.events.retry3')
|
295
|
+
expanded_tube = [Backburner.configuration.tube_namespace, 'foo.bar.events.retry3'].join(".")
|
296
|
+
out = silenced(2) do
|
297
|
+
HookedObjectBeforePerformFail.async(:queue => 'foo.bar.events.retry3').foo(10)
|
298
|
+
worker = Backburner::Worker.new('foo.bar.events.retry3')
|
299
|
+
worker.prepare
|
300
|
+
worker.work_one_job
|
301
|
+
end
|
302
|
+
assert_match /!!before_perform_foo!! \[nil, "foo", 10\]/, out
|
303
|
+
assert_match /before_perform_foo.*Finished/m, out
|
304
|
+
refute_match(/Fail ran!!/, out)
|
305
|
+
refute_match(/HookFailError/, out)
|
306
|
+
end # stopping perform
|
307
|
+
|
261
308
|
after do
|
262
309
|
Backburner.configure { |config| config.max_job_retries = 0; config.retry_delay = 5 }
|
263
310
|
end
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: backburner
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.2.0
|
5
5
|
prerelease:
|
6
6
|
platform: ruby
|
7
7
|
authors:
|
@@ -103,6 +103,7 @@ files:
|
|
103
103
|
- .travis.yml
|
104
104
|
- CHANGELOG.md
|
105
105
|
- Gemfile
|
106
|
+
- HOOKS.md
|
106
107
|
- LICENSE
|
107
108
|
- README.md
|
108
109
|
- Rakefile
|
@@ -112,6 +113,7 @@ files:
|
|
112
113
|
- examples/custom.rb
|
113
114
|
- examples/demo.rb
|
114
115
|
- examples/god.rb
|
116
|
+
- examples/hooked.rb
|
115
117
|
- examples/retried.rb
|
116
118
|
- examples/simple.rb
|
117
119
|
- lib/backburner.rb
|
@@ -119,6 +121,7 @@ files:
|
|
119
121
|
- lib/backburner/configuration.rb
|
120
122
|
- lib/backburner/connection.rb
|
121
123
|
- lib/backburner/helpers.rb
|
124
|
+
- lib/backburner/hooks.rb
|
122
125
|
- lib/backburner/job.rb
|
123
126
|
- lib/backburner/logger.rb
|
124
127
|
- lib/backburner/performable.rb
|
@@ -128,7 +131,10 @@ files:
|
|
128
131
|
- lib/backburner/worker.rb
|
129
132
|
- test/back_burner_test.rb
|
130
133
|
- test/connection_test.rb
|
134
|
+
- test/fixtures/hooked.rb
|
135
|
+
- test/fixtures/test_jobs.rb
|
131
136
|
- test/helpers_test.rb
|
137
|
+
- test/hooks_test.rb
|
132
138
|
- test/job_test.rb
|
133
139
|
- test/logger_test.rb
|
134
140
|
- test/performable_test.rb
|
@@ -162,7 +168,10 @@ summary: Reliable beanstalk background job processing made easy for Ruby and Sin
|
|
162
168
|
test_files:
|
163
169
|
- test/back_burner_test.rb
|
164
170
|
- test/connection_test.rb
|
171
|
+
- test/fixtures/hooked.rb
|
172
|
+
- test/fixtures/test_jobs.rb
|
165
173
|
- test/helpers_test.rb
|
174
|
+
- test/hooks_test.rb
|
166
175
|
- test/job_test.rb
|
167
176
|
- test/logger_test.rb
|
168
177
|
- test/performable_test.rb
|