lantins-resque-multi-job-forks 0.3.0 → 0.3.1
Sign up to get free protection for your applications and to get access to all the features.
- data/README.md +64 -0
- data/lib/resque-multi-job-forks.rb +8 -5
- metadata +5 -5
- data/README.rdoc +0 -30
data/README.md
ADDED
@@ -0,0 +1,64 @@
|
|
1
|
+
resque-multi-job-forks
|
2
|
+
======================
|
3
|
+
|
4
|
+
If you have very frequent and fast resque jobs, the overhead of forking and
|
5
|
+
running your after_fork hook, might get too big. Using this resque plugin, you
|
6
|
+
can have your workers perform more than one job, before terminating.
|
7
|
+
|
8
|
+
Install & Quick Start
|
9
|
+
---------------------
|
10
|
+
|
11
|
+
To install:
|
12
|
+
|
13
|
+
$ gem install lantins-resque-multi-job-forks
|
14
|
+
|
15
|
+
You'll need to require the code; perhaps inside your `resque:setup` hook of
|
16
|
+
your `Rakefile` ?:
|
17
|
+
|
18
|
+
namespace :resque do
|
19
|
+
task :setup do
|
20
|
+
require 'resque-multi-job-forks'
|
21
|
+
end
|
22
|
+
end
|
23
|
+
|
24
|
+
### Limit Per Fork
|
25
|
+
|
26
|
+
You have two limit options available, only one can be used at once.
|
27
|
+
|
28
|
+
* `JOBS_PER_FORK` - number of jobs to process per fork.
|
29
|
+
* `MINUTES_PER_FORK` - process jobs for *n* minutes per fork.
|
30
|
+
|
31
|
+
**Default:** process jobs for *1 minute* before terminating.
|
32
|
+
|
33
|
+
You may specify the limit by setting the appropriate each variable:
|
34
|
+
|
35
|
+
# process jobs for 5 minutes, then terminate the fork.
|
36
|
+
QUEUE=* MINUTES_PER_FORK=5 rake resque:work
|
37
|
+
|
38
|
+
# process 5000 jobs per fork, then terminate.
|
39
|
+
QUEUE=* JOBS_PER_FORK=5000 rake resque:work
|
40
|
+
|
41
|
+
### Resque Hooks
|
42
|
+
|
43
|
+
This plugin also defines a new hook, that gets called right before the fork
|
44
|
+
terminates:
|
45
|
+
|
46
|
+
Resque.before_child_exit do |worker|
|
47
|
+
worker.log("#{worker.jobs_processed} were processed in this fork")
|
48
|
+
end
|
49
|
+
|
50
|
+
Note on Patches/Pull Requests
|
51
|
+
-----------------------------
|
52
|
+
|
53
|
+
* Fork the project.
|
54
|
+
* Make your feature addition or bug fix.
|
55
|
+
* Add tests for it. This is important so I don't break it in a
|
56
|
+
future version unintentionally.
|
57
|
+
* Commit, do not mess with rakefile, version, or history.
|
58
|
+
(if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
|
59
|
+
* Send me a pull request. Bonus points for topic branches.
|
60
|
+
|
61
|
+
Copyright
|
62
|
+
---------
|
63
|
+
|
64
|
+
Copyright (c) 2010 Mick Staugaard. See LICENSE for details.
|
@@ -2,10 +2,13 @@ require 'resque'
|
|
2
2
|
require 'resque/worker'
|
3
3
|
|
4
4
|
module Resque
|
5
|
+
|
6
|
+
# Add support to allow a single fork to process multiple jobs.
|
7
|
+
# You may limit the fork to n minutes of execution time, or a total job limit.
|
5
8
|
class Worker
|
6
|
-
attr_accessor :seconds_per_fork
|
7
|
-
attr_accessor :jobs_per_fork
|
8
|
-
attr_reader :jobs_processed
|
9
|
+
attr_accessor :seconds_per_fork # seconds fork may process jobs for.
|
10
|
+
attr_accessor :jobs_per_fork # total jobs to process per fork.
|
11
|
+
attr_reader :jobs_processed # jobs processed by this fork.
|
9
12
|
|
10
13
|
unless method_defined?(:shutdown_without_multi_job_forks)
|
11
14
|
def perform_with_multi_job_forks(job = nil)
|
@@ -70,8 +73,8 @@ module Resque
|
|
70
73
|
end
|
71
74
|
end
|
72
75
|
|
73
|
-
# the `before_child_exit` hook will run in the child process
|
74
|
-
#
|
76
|
+
# the `before_child_exit` hook will run in the child process right before
|
77
|
+
# the child process terminates.
|
75
78
|
#
|
76
79
|
# Call with a block to set the hook.
|
77
80
|
# Call with no arguments to return the hook.
|
metadata
CHANGED
@@ -1,13 +1,13 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: lantins-resque-multi-job-forks
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
hash:
|
4
|
+
hash: 17
|
5
5
|
prerelease: false
|
6
6
|
segments:
|
7
7
|
- 0
|
8
8
|
- 3
|
9
|
-
-
|
10
|
-
version: 0.3.
|
9
|
+
- 1
|
10
|
+
version: 0.3.1
|
11
11
|
platform: ruby
|
12
12
|
authors:
|
13
13
|
- Mick Staugaard
|
@@ -75,12 +75,12 @@ extra_rdoc_files: []
|
|
75
75
|
files:
|
76
76
|
- LICENSE
|
77
77
|
- Rakefile
|
78
|
-
- README.
|
78
|
+
- README.md
|
79
79
|
- test/helper.rb
|
80
80
|
- test/test_resque-multi-job-forks.rb
|
81
81
|
- lib/resque-multi-job-forks.rb
|
82
82
|
has_rdoc: false
|
83
|
-
homepage: http://github.com/lantins/
|
83
|
+
homepage: http://github.com/lantins/resque-multi-job-forks/tree/personal_gem
|
84
84
|
licenses: []
|
85
85
|
|
86
86
|
post_install_message:
|
data/README.rdoc
DELETED
@@ -1,30 +0,0 @@
|
|
1
|
-
= resque-multi-job-forks
|
2
|
-
|
3
|
-
If you have very frequent and fast resque jobs, the overhead of forking and running your after_fork hook, might get too big. Using this resque plugin, you can have your workers perform more than one job, before terminating.
|
4
|
-
|
5
|
-
You simply specify the number of minutes you want each fork to run using the MINUTES_PER_FORK environment variable:
|
6
|
-
|
7
|
-
QUEUE=* MINUTES_PER_FORK=5 rake resque:work
|
8
|
-
|
9
|
-
This will have each fork process jobs for 5 minutes, before terminating.
|
10
|
-
|
11
|
-
This plugin also defines a new hook, that gets called right before the fork terminates:
|
12
|
-
|
13
|
-
Resque.before_child_exit do |worker|
|
14
|
-
worker.log("#{worker.jobs_processed} were processed in this fork")
|
15
|
-
end
|
16
|
-
|
17
|
-
|
18
|
-
== Note on Patches/Pull Requests
|
19
|
-
|
20
|
-
* Fork the project.
|
21
|
-
* Make your feature addition or bug fix.
|
22
|
-
* Add tests for it. This is important so I don't break it in a
|
23
|
-
future version unintentionally.
|
24
|
-
* Commit, do not mess with rakefile, version, or history.
|
25
|
-
(if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
|
26
|
-
* Send me a pull request. Bonus points for topic branches.
|
27
|
-
|
28
|
-
== Copyright
|
29
|
-
|
30
|
-
Copyright (c) 2010 Mick Staugaard. See LICENSE for details.
|