xing-gearman-ruby 1.0.0 → 1.1.0
Sign up to get free protection for your applications and to get access to all the features.
- data/HOWTO +146 -0
- data/TODO +8 -0
- data/VERSION.yml +2 -2
- data/examples/calculus_client.rb +39 -0
- data/examples/calculus_worker.rb +45 -0
- data/examples/client_background.rb +15 -0
- data/examples/client_exception.rb +18 -0
- data/examples/client_prefix.rb +17 -0
- data/examples/gearman_environment.sh +25 -0
- data/examples/worker_exception.rb +14 -0
- data/examples/worker_prefix.rb +25 -0
- data/gearman-ruby.gemspec +24 -6
- data/lib/gearman/client.rb +15 -2
- data/lib/gearman/task.rb +40 -7
- data/lib/gearman/taskset.rb +33 -5
- data/lib/gearman/testlib.rb +4 -5
- data/lib/gearman/util.rb +13 -5
- data/lib/gearman/worker.rb +14 -6
- data/test/client_test.rb +34 -0
- data/test/mock_client_test.rb +154 -0
- data/test/mock_worker_test.rb +44 -0
- data/test/util_test.rb +17 -0
- metadata +23 -4
data/HOWTO
ADDED
@@ -0,0 +1,146 @@
|
|
1
|
+
= GEARMAN
|
2
|
+
|
3
|
+
"Gearman provides a generic application framework to farm out work to other
|
4
|
+
machines or processes that are better suited to do the work. It allows you to
|
5
|
+
do work in parallel, to load balance processing, and to call functions between
|
6
|
+
languages. It can be used in a variety of applications, from high-availability
|
7
|
+
web sites to the transport of database replication events. In other words, it
|
8
|
+
is the nervous system for how distributed processing communicates."
|
9
|
+
|
10
|
+
- http://www.gearman.org/
|
11
|
+
|
12
|
+
|
13
|
+
== Setting up a basic environment
|
14
|
+
|
15
|
+
A very basic German environment will look like this:
|
16
|
+
|
17
|
+
----------
|
18
|
+
| Client |
|
19
|
+
----------
|
20
|
+
|
|
21
|
+
--------------
|
22
|
+
| Job Server |
|
23
|
+
--------------
|
24
|
+
|
|
25
|
+
----------------------------------------------
|
26
|
+
| | | |
|
27
|
+
---------- ---------- ---------- ----------
|
28
|
+
| Worker | | Worker | | Worker | | Worker |
|
29
|
+
---------- ---------- ---------- ----------
|
30
|
+
|
31
|
+
And the behavior will be the following:
|
32
|
+
|
33
|
+
* JobServer: Acts as a message passing point.
|
34
|
+
* Client: Sends tasks to the JobServer. Will be connected to only one JobServer
|
35
|
+
in case more than one exits for failover purposes.
|
36
|
+
* Worker: Anounce his 'abilities' to the JobServer and waits for tasks.
|
37
|
+
|
38
|
+
For the JobServer we recommend to use the offical Perl version, there's also a
|
39
|
+
more performant C implementation of the server with support for persistent
|
40
|
+
queues, bells and whistles but is not stable enough for production use at the
|
41
|
+
time of this document was wrote.
|
42
|
+
|
43
|
+
The Client and the Worker can be implemented in any language. This way you can
|
44
|
+
send tasks from a Ruby client server, to a Perl or C worker in order to get
|
45
|
+
better performance.
|
46
|
+
|
47
|
+
== Installing the required software
|
48
|
+
|
49
|
+
For the JobServer we recommend to use the offical Perl version, to install it:
|
50
|
+
|
51
|
+
* Mac OS X: sudo port install p5-gearman-server
|
52
|
+
* Debian/Ubuntu: sudo apt-get install gearman-server
|
53
|
+
|
54
|
+
To get the Ruby libraries by Xing:
|
55
|
+
|
56
|
+
git clone git://github.com/xing/gearman-ruby.git
|
57
|
+
|
58
|
+
== Gearman demo
|
59
|
+
|
60
|
+
Now you're ready for you first experience with Gearman. In the cloned repository
|
61
|
+
you'll find an 'examples' directory.
|
62
|
+
|
63
|
+
Run the 'gearman_environment.sh' to build an environment like the one showed in
|
64
|
+
the diagram above.
|
65
|
+
|
66
|
+
* Client: Will ask you for an arithmetic operation, like: 2+3
|
67
|
+
The code of the client is in: 'examples/calculus_client.rb'
|
68
|
+
|
69
|
+
* JobServer: The Perl server.
|
70
|
+
|
71
|
+
* Workers: You'll have 4 worker, one for each of the basic arithmetic
|
72
|
+
operations.
|
73
|
+
The code of the worker is in: 'examples/calculus_worker.rb'
|
74
|
+
|
75
|
+
There are other demos in the examples folder you can give a look at. Each demo usually
|
76
|
+
consist in the client and server scripts.
|
77
|
+
|
78
|
+
=== Creating clients and tasks
|
79
|
+
|
80
|
+
In order to get a job scheduled by a Gearman server using the gearman ruby library, there
|
81
|
+
are three main objects you must interact with: Gearman::Client, Gearman::Task and Gearman::TaskSet.
|
82
|
+
Let's review all of them briefly:
|
83
|
+
|
84
|
+
- Gearman::Client -> the portion of the library storing the data about the connection to
|
85
|
+
the Gearman server.
|
86
|
+
- Gearman::Task -> a job execution request that will be dispatched by the Gearman server to
|
87
|
+
worker and whose result data will be returned to the client.
|
88
|
+
- Gearman::TaskSet -> a collection of tasks to be executed. The Taskset object will track the
|
89
|
+
execution of the tasks with the info returned from the Gearman server
|
90
|
+
and notify the client with the results or errors when all the tasks
|
91
|
+
have completed their execution.
|
92
|
+
|
93
|
+
To send a new task to the Gearman server, the client must build a new Gearman::Task object, add it to
|
94
|
+
a Gearman::TaskSet that must hold a reference to a Gearman::Client and send the wait message to
|
95
|
+
the TaskSet.
|
96
|
+
The following code taken from examples/client.rb shows the process:
|
97
|
+
|
98
|
+
----------------------------------------------------
|
99
|
+
servers = ['localhost:4730', 'localhost:4731']
|
100
|
+
|
101
|
+
client = Gearman::Client.new(servers)
|
102
|
+
taskset = Gearman::TaskSet.new(client)
|
103
|
+
|
104
|
+
task = Gearman::Task.new('sleep', 20)
|
105
|
+
task.on_complete {|d| puts d }
|
106
|
+
|
107
|
+
taskset.add_task(task)
|
108
|
+
taskset.wait(100)
|
109
|
+
----------------------------------------------------
|
110
|
+
|
111
|
+
The name of the function to be executed is the first parameter to the constructor of the Task object.
|
112
|
+
Take into account that the string you pass as a parameter will be used 'as it' by the Gearman server
|
113
|
+
to locate a suitable worker for that function.
|
114
|
+
The second parameter is the argument to be sent to the worker that will execute, if the arguments for
|
115
|
+
the task are complex, a serialization format like YAML or XML must be agreeded with the workers.
|
116
|
+
The last and optional parameter is a hash of options. The following options are currently available:
|
117
|
+
|
118
|
+
- :priority -> (:high | :low) the priority of the job, a high priority job is executed before a low on
|
119
|
+
- :background -> (true | false) a background task will return no further information to the client.
|
120
|
+
|
121
|
+
The execution of a task in a Gearman remote worker can fail, the worker can throw an exception, etc.
|
122
|
+
All these events can be handled by the client of the ruby library registering callback blocks.
|
123
|
+
The following events are currently available:
|
124
|
+
|
125
|
+
- on_complete -> the task was executed succesfully
|
126
|
+
- on_fail -> the task fail for some unknown reason
|
127
|
+
- on_retry -> after failure, the task is gonna be retried, the number of retries is passed
|
128
|
+
- on_exception -> the remote worker send an exception notification, the exception text is passed
|
129
|
+
- on_stauts -> a status update is sent by the remote worker
|
130
|
+
|
131
|
+
In order to receive exception notifications in the client, this option must be sent in the server, the
|
132
|
+
method option_request can be used this task. The following example, extracted from the examples/client_exception.rb
|
133
|
+
demo script shows the process:
|
134
|
+
|
135
|
+
----------------------------------------------------
|
136
|
+
client = Gearman::Client.new(servers)
|
137
|
+
#try this out
|
138
|
+
client.option_request("exceptions")
|
139
|
+
----------------------------------------------------
|
140
|
+
|
141
|
+
This feature will only works if the server and workers have implemented support for the OPT_REQ and WORK_EXCEPTION
|
142
|
+
messages of the Gearman protocol.
|
143
|
+
|
144
|
+
|
145
|
+
Enjoy.
|
146
|
+
|
data/TODO
ADDED
@@ -0,0 +1,8 @@
|
|
1
|
+
- Failover strategies
|
2
|
+
- Client:
|
3
|
+
* If connected for the first time, try to connect to at least one server from the server array
|
4
|
+
* If already connected to a server, and it goes down, the client should go down as well
|
5
|
+
- Worker:
|
6
|
+
* If connected for the first time, try to connect to as many servers as it can.
|
7
|
+
Loop trough the bad servers, trying to reconnect to them as well.
|
8
|
+
* If a already connected to a server, and it goes down, wait and try to reconnect again.
|
data/VERSION.yml
CHANGED
@@ -0,0 +1,39 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
require 'rubygems'
|
3
|
+
require '../lib/gearman'
|
4
|
+
#Gearman::Util.debug = true
|
5
|
+
|
6
|
+
# Connect to the local server (at the default port 7003)
|
7
|
+
client = Gearman::Client.new('localhost')
|
8
|
+
taskset = Gearman::TaskSet.new(client)
|
9
|
+
|
10
|
+
# Get something to echo
|
11
|
+
puts '[client] Write a basic arithmetic operation:'
|
12
|
+
input = gets
|
13
|
+
|
14
|
+
operations = input.chomp.scan(/\d+[\+\-\*\/]\d+/).compact
|
15
|
+
puts "[client] The following operations were found: #{operations.inspect}"
|
16
|
+
|
17
|
+
# Setup a task for operation
|
18
|
+
operations.each do |op|
|
19
|
+
# Determining the operation
|
20
|
+
case op
|
21
|
+
when /\+/
|
22
|
+
type, data = 'addition', op.split('+')
|
23
|
+
when /\-/
|
24
|
+
type, data = 'subtraction', op.split('-')
|
25
|
+
when /\*/
|
26
|
+
type, data = 'multiplication', op.split('*')
|
27
|
+
when /\//
|
28
|
+
type, data = 'division', op.split('/')
|
29
|
+
end
|
30
|
+
|
31
|
+
task = Gearman::Task.new(type, Marshal.dump(data.map {|v| v.to_i}))
|
32
|
+
task.on_complete {|r| puts "[client] #{type} result is: #{r}" }
|
33
|
+
|
34
|
+
# Sending the task to the server
|
35
|
+
puts "[client] Sending values: #{data.inspect}, to the '#{type}' worker"
|
36
|
+
taskset.add_task(task)
|
37
|
+
taskset.wait(100)
|
38
|
+
end
|
39
|
+
|
@@ -0,0 +1,45 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
require 'rubygems'
|
3
|
+
require '../lib/gearman'
|
4
|
+
|
5
|
+
#Gearman::Util.debug = true
|
6
|
+
|
7
|
+
worker = Gearman::Worker.new('localhost')
|
8
|
+
worker.reconnect_sec = 2
|
9
|
+
|
10
|
+
# Additon ability
|
11
|
+
worker.add_ability('addition') do |data,job|
|
12
|
+
values = Marshal.load(data)
|
13
|
+
puts "[addition_worker] Calculating #{values.inspect}..."
|
14
|
+
# sleep 5
|
15
|
+
values.first + values.last
|
16
|
+
end
|
17
|
+
|
18
|
+
# Subtraction ability
|
19
|
+
worker.add_ability('subtraction') do |data,job|
|
20
|
+
values = Marshal.load(data)
|
21
|
+
puts "[subtraction_worker] Calculating #{values.inspect}..."
|
22
|
+
# sleep 5
|
23
|
+
values.first - values.last
|
24
|
+
end
|
25
|
+
|
26
|
+
# Multiplication worker
|
27
|
+
worker.add_ability('multiplication') do |data,job|
|
28
|
+
values = Marshal.load(data)
|
29
|
+
puts "[multiplication_worker] Calculating #{values.inspect}..."
|
30
|
+
# sleep 5
|
31
|
+
values.first * values.last
|
32
|
+
end
|
33
|
+
|
34
|
+
# Division worker
|
35
|
+
worker.add_ability('division') do |data,job|
|
36
|
+
values = Marshal.load(data)
|
37
|
+
puts "[division_worker] Calculating #{data.inspect}..."
|
38
|
+
# sleep 5
|
39
|
+
values.first / values.last
|
40
|
+
end
|
41
|
+
|
42
|
+
# Running the workers
|
43
|
+
loop do
|
44
|
+
worker.work
|
45
|
+
end
|
@@ -0,0 +1,15 @@
|
|
1
|
+
require 'rubygems'
|
2
|
+
#require 'gearman'
|
3
|
+
require '../lib/gearman'
|
4
|
+
Gearman::Util.debug = true
|
5
|
+
|
6
|
+
servers = ['localhost:4730', 'localhost:4731']
|
7
|
+
|
8
|
+
client = Gearman::Client.new(servers)
|
9
|
+
taskset = Gearman::TaskSet.new(client)
|
10
|
+
|
11
|
+
task = Gearman::Task.new('sleep', 20, { :background => true })
|
12
|
+
task.on_complete {|d| puts d }
|
13
|
+
|
14
|
+
taskset.add_task(task)
|
15
|
+
taskset.wait(100)
|
@@ -0,0 +1,18 @@
|
|
1
|
+
require 'rubygems'
|
2
|
+
require '../lib/gearman'
|
3
|
+
Gearman::Util.debug = true
|
4
|
+
|
5
|
+
servers = ['localhost:4730']
|
6
|
+
|
7
|
+
client = Gearman::Client.new(servers)
|
8
|
+
#try this out
|
9
|
+
client.option_request("exceptions")
|
10
|
+
|
11
|
+
taskset = Gearman::TaskSet.new(client)
|
12
|
+
|
13
|
+
task = Gearman::Task.new('fail_with_exception', 20)
|
14
|
+
task.on_complete {|d| puts d }
|
15
|
+
task.on_exception {|message| puts message; false}
|
16
|
+
|
17
|
+
taskset.add_task(task)
|
18
|
+
taskset.wait(100)
|
@@ -0,0 +1,17 @@
|
|
1
|
+
require 'rubygems'
|
2
|
+
#require 'gearman'
|
3
|
+
require '../lib/gearman'
|
4
|
+
Gearman::Util.debug = true
|
5
|
+
|
6
|
+
servers = ['localhost:4730', 'localhost:4731']
|
7
|
+
|
8
|
+
ability_name_with_prefix = Gearman::Util.ability_name_with_prefix("test","sleep")
|
9
|
+
|
10
|
+
client = Gearman::Client.new(servers)
|
11
|
+
taskset = Gearman::TaskSet.new(client)
|
12
|
+
|
13
|
+
task = Gearman::Task.new(ability_name_with_prefix, 20)
|
14
|
+
task.on_complete {|d| puts d }
|
15
|
+
|
16
|
+
taskset.add_task(task)
|
17
|
+
taskset.wait(100)
|
@@ -0,0 +1,25 @@
|
|
1
|
+
#!/bin/bash
|
2
|
+
|
3
|
+
# Start Gearmand
|
4
|
+
echo ' + Starting Gearmand'
|
5
|
+
gearmand --daemon --pidfile=/tmp/gearmand.pid
|
6
|
+
|
7
|
+
# Start the client and the worker(s)
|
8
|
+
echo ' + Starting calculus_worker.rb'
|
9
|
+
ruby calculus_worker.rb &
|
10
|
+
|
11
|
+
sleep 3
|
12
|
+
|
13
|
+
echo ' + Starting calculus_client.rb'
|
14
|
+
ruby calculus_client.rb
|
15
|
+
|
16
|
+
echo ' +++ Example finished +++ '
|
17
|
+
|
18
|
+
# Stop Gearmand
|
19
|
+
echo ' - Stopping Gearmand'
|
20
|
+
kill -9 `cat /tmp/gearmand.pid`
|
21
|
+
|
22
|
+
# Stop the workers
|
23
|
+
echo ' - Stopping calculus_worker.rb'
|
24
|
+
kill -9 `ps ax|grep calculus_worker|grep ruby|awk -F' ' '{print $1}'`
|
25
|
+
|
@@ -0,0 +1,14 @@
|
|
1
|
+
require 'rubygems'
|
2
|
+
require '../lib/gearman'
|
3
|
+
|
4
|
+
Gearman::Util.debug = true
|
5
|
+
|
6
|
+
servers = ['localhost:4730']
|
7
|
+
w = Gearman::Worker.new(servers)
|
8
|
+
|
9
|
+
# Add a handler for a "sleep" function that takes a single argument, the
|
10
|
+
# number of seconds to sleep before reporting success.
|
11
|
+
w.add_ability('fail_with_exception') do |data,job|
|
12
|
+
raise Exception.new("fooexception")
|
13
|
+
end
|
14
|
+
loop { w.work }
|
@@ -0,0 +1,25 @@
|
|
1
|
+
require 'rubygems'
|
2
|
+
#require 'gearman'
|
3
|
+
require '../lib/gearman'
|
4
|
+
|
5
|
+
Gearman::Util.debug = true
|
6
|
+
|
7
|
+
servers = ['localhost:4730', 'localhost:4731']
|
8
|
+
w = Gearman::Worker.new(servers)
|
9
|
+
|
10
|
+
ability_name_with_prefix = Gearman::Util.ability_name_with_prefix("test","sleep")
|
11
|
+
|
12
|
+
# Add a handler for a "sleep" function that takes a single argument, the
|
13
|
+
# number of seconds to sleep before reporting success.
|
14
|
+
w.add_ability(ability_name_with_prefix) do |data,job|
|
15
|
+
seconds = data
|
16
|
+
(1..seconds.to_i).each do |i|
|
17
|
+
sleep 1
|
18
|
+
print i
|
19
|
+
# Report our progress to the job server every second.
|
20
|
+
job.report_status(i, seconds)
|
21
|
+
end
|
22
|
+
# Report success.
|
23
|
+
true
|
24
|
+
end
|
25
|
+
loop { w.work }
|
data/gearman-ruby.gemspec
CHANGED
@@ -2,11 +2,11 @@
|
|
2
2
|
|
3
3
|
Gem::Specification.new do |s|
|
4
4
|
s.name = %q{gearman-ruby}
|
5
|
-
s.version = "1.
|
5
|
+
s.version = "1.1.0"
|
6
6
|
|
7
7
|
s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
|
8
8
|
s.authors = ["Daniel Erat", "Ladislav Martincik"]
|
9
|
-
s.date = %q{2009-
|
9
|
+
s.date = %q{2009-07-16}
|
10
10
|
s.description = %q{Library for the Gearman distributed job system}
|
11
11
|
s.email = %q{ladislav.martincik@xing.com}
|
12
12
|
s.extra_rdoc_files = [
|
@@ -15,9 +15,11 @@ Gem::Specification.new do |s|
|
|
15
15
|
]
|
16
16
|
s.files = [
|
17
17
|
".gitignore",
|
18
|
+
"HOWTO",
|
18
19
|
"LICENSE",
|
19
20
|
"README",
|
20
21
|
"Rakefile",
|
22
|
+
"TODO",
|
21
23
|
"VERSION.yml",
|
22
24
|
"examples/Net/Gearman/Client.php",
|
23
25
|
"examples/Net/Gearman/Connection.php",
|
@@ -30,13 +32,21 @@ Gem::Specification.new do |s|
|
|
30
32
|
"examples/Net/Gearman/Set.php",
|
31
33
|
"examples/Net/Gearman/Task.php",
|
32
34
|
"examples/Net/Gearman/Worker.php",
|
35
|
+
"examples/calculus_client.rb",
|
36
|
+
"examples/calculus_worker.rb",
|
33
37
|
"examples/client.php",
|
34
38
|
"examples/client.rb",
|
39
|
+
"examples/client_background.rb",
|
40
|
+
"examples/client_exception.rb",
|
41
|
+
"examples/client_prefix.rb",
|
42
|
+
"examples/gearman_environment.sh",
|
35
43
|
"examples/scale_image.rb",
|
36
44
|
"examples/scale_image_worker.rb",
|
37
45
|
"examples/server.rb",
|
38
46
|
"examples/worker.php",
|
39
47
|
"examples/worker.rb",
|
48
|
+
"examples/worker_exception.rb",
|
49
|
+
"examples/worker_prefix.rb",
|
40
50
|
"gearman-ruby.gemspec",
|
41
51
|
"lib/gearman.rb",
|
42
52
|
"lib/gearman/client.rb",
|
@@ -49,29 +59,37 @@ Gem::Specification.new do |s|
|
|
49
59
|
"test/client_test.rb",
|
50
60
|
"test/mock_client_test.rb",
|
51
61
|
"test/mock_worker_test.rb",
|
62
|
+
"test/util_test.rb",
|
52
63
|
"test/worker_test.rb"
|
53
64
|
]
|
54
|
-
s.has_rdoc = true
|
55
65
|
s.homepage = %q{http://github.com/xing/gearman-ruby}
|
56
66
|
s.rdoc_options = ["--charset=UTF-8"]
|
57
67
|
s.require_paths = ["lib"]
|
58
|
-
s.rubygems_version = %q{1.3.
|
68
|
+
s.rubygems_version = %q{1.3.4}
|
59
69
|
s.summary = %q{Library for the Gearman distributed job system}
|
60
70
|
s.test_files = [
|
61
71
|
"test/client_test.rb",
|
62
72
|
"test/mock_client_test.rb",
|
63
73
|
"test/mock_worker_test.rb",
|
74
|
+
"test/util_test.rb",
|
64
75
|
"test/worker_test.rb",
|
76
|
+
"examples/calculus_client.rb",
|
77
|
+
"examples/calculus_worker.rb",
|
65
78
|
"examples/client.rb",
|
79
|
+
"examples/client_background.rb",
|
80
|
+
"examples/client_exception.rb",
|
81
|
+
"examples/client_prefix.rb",
|
66
82
|
"examples/scale_image.rb",
|
67
83
|
"examples/scale_image_worker.rb",
|
68
84
|
"examples/server.rb",
|
69
|
-
"examples/worker.rb"
|
85
|
+
"examples/worker.rb",
|
86
|
+
"examples/worker_exception.rb",
|
87
|
+
"examples/worker_prefix.rb"
|
70
88
|
]
|
71
89
|
|
72
90
|
if s.respond_to? :specification_version then
|
73
91
|
current_version = Gem::Specification::CURRENT_SPECIFICATION_VERSION
|
74
|
-
s.specification_version =
|
92
|
+
s.specification_version = 3
|
75
93
|
|
76
94
|
if Gem::Version.new(Gem::RubyGemsVersion) >= Gem::Version.new('1.2.0') then
|
77
95
|
else
|
data/lib/gearman/client.rb
CHANGED
@@ -23,8 +23,21 @@ class Client
|
|
23
23
|
@server_counter = -1
|
24
24
|
@bad_servers = []
|
25
25
|
end
|
26
|
-
attr_reader :job_servers, :bad_servers
|
27
|
-
attr_accessor :test_hostport, :task_create_timeout_sec
|
26
|
+
attr_reader :job_servers, :bad_servers
|
27
|
+
attr_accessor :test_hostport, :task_create_timeout_sec
|
28
|
+
|
29
|
+
##
|
30
|
+
# Set the options
|
31
|
+
#
|
32
|
+
# @options options to pass to the servers "exeptions"
|
33
|
+
def option_request(opts)
|
34
|
+
Util.log "Send options request with #{opts}"
|
35
|
+
request = Util.pack_request("option_req", opts)
|
36
|
+
sock= self.get_socket(self.get_job_server)
|
37
|
+
Util.send_request(sock, request)
|
38
|
+
response = Util.read_response(sock, 20)
|
39
|
+
raise ProtocolError, response[1] if response[0]==:error
|
40
|
+
end
|
28
41
|
|
29
42
|
##
|
30
43
|
# Set the job servers to be used by this client.
|
data/lib/gearman/task.rb
CHANGED
@@ -7,6 +7,7 @@ module Gearman
|
|
7
7
|
# == Description
|
8
8
|
# A task submitted to a Gearman job server.
|
9
9
|
class Task
|
10
|
+
|
10
11
|
##
|
11
12
|
# Create a new Task object.
|
12
13
|
#
|
@@ -16,8 +17,8 @@ class Task
|
|
16
17
|
def initialize(func, arg='', opts={})
|
17
18
|
@func = func.to_s
|
18
19
|
@arg = arg or '' # TODO: use something more ref-like?
|
19
|
-
%w{uniq on_complete on_fail on_retry on_status retry_count
|
20
|
-
|
20
|
+
%w{uniq on_complete on_fail on_retry on_exception on_status retry_count
|
21
|
+
priority background}.map {|s| s.to_sym }.each do |k|
|
21
22
|
instance_variable_set "@#{k}", opts[k]
|
22
23
|
opts.delete k
|
23
24
|
end
|
@@ -29,7 +30,7 @@ class Task
|
|
29
30
|
@retries_done = 0
|
30
31
|
@hash = nil
|
31
32
|
end
|
32
|
-
attr_accessor :uniq, :retry_count, :
|
33
|
+
attr_accessor :uniq, :retry_count, :priority, :background
|
33
34
|
attr_reader :successful, :func, :arg
|
34
35
|
|
35
36
|
##
|
@@ -62,6 +63,14 @@ class Task
|
|
62
63
|
@on_retry = f
|
63
64
|
end
|
64
65
|
|
66
|
+
##
|
67
|
+
# Set a block of code to be executed when a remote exception is sent by a worker.
|
68
|
+
# The block will receive the message of the exception passed from the worker.
|
69
|
+
# The user can return true for retrying or false to mark it as finished
|
70
|
+
def on_exception(&f)
|
71
|
+
@on_exception = f
|
72
|
+
end
|
73
|
+
|
65
74
|
##
|
66
75
|
# Set a block of code to be executed when we receive a status update for
|
67
76
|
# this task. The block will receive two arguments, a numerator and
|
@@ -94,6 +103,20 @@ class Task
|
|
94
103
|
true
|
95
104
|
end
|
96
105
|
|
106
|
+
##
|
107
|
+
# Record an exception and check whether we should be retried.
|
108
|
+
#
|
109
|
+
# @return true if we should be resubmitted; false otherwise
|
110
|
+
def handle_exception(exception)
|
111
|
+
if @on_exception
|
112
|
+
should_retry = @on_exception.call(exception)
|
113
|
+
@retries_done += 1 if should_retry
|
114
|
+
should_retry
|
115
|
+
else
|
116
|
+
false
|
117
|
+
end
|
118
|
+
end
|
119
|
+
|
97
120
|
##
|
98
121
|
# Handle a status update for the task.
|
99
122
|
def handle_status(numerator, denominator)
|
@@ -116,11 +139,21 @@ class Task
|
|
116
139
|
##
|
117
140
|
# Construct a packet to submit this task to a job server.
|
118
141
|
#
|
119
|
-
# @param background ??
|
120
142
|
# @return String representation of packet
|
121
|
-
def get_submit_packet(
|
122
|
-
mode = 'submit_job'
|
123
|
-
|
143
|
+
def get_submit_packet()
|
144
|
+
mode = 'submit_job'
|
145
|
+
if(@priority)
|
146
|
+
if(@priority == :high)
|
147
|
+
mode += "_high"
|
148
|
+
elsif(@priority == :low)
|
149
|
+
mode += "_low"
|
150
|
+
end
|
151
|
+
end
|
152
|
+
|
153
|
+
if(@background)
|
154
|
+
mode += "_bg"
|
155
|
+
end
|
156
|
+
|
124
157
|
Util::pack_request(mode, [func, get_uniq_hash, arg].join("\0"))
|
125
158
|
end
|
126
159
|
end
|
data/lib/gearman/taskset.rb
CHANGED
@@ -92,13 +92,16 @@ class TaskSet
|
|
92
92
|
def handle_job_created(hostport, data)
|
93
93
|
Util.log "Got job_created with handle #{data} from #{hostport}"
|
94
94
|
if not @task_waiting_for_handle
|
95
|
-
raise ProtocolError, "Got unexpected job_created notification " +
|
96
|
-
"with handle #{data} from #{hostport}"
|
95
|
+
raise ProtocolError, "Got unexpected job_created notification " + "with handle #{data} from #{hostport}"
|
97
96
|
end
|
98
97
|
js_handle = Util.handle_to_str(hostport, data)
|
99
98
|
task = @task_waiting_for_handle
|
100
99
|
@task_waiting_for_handle = nil
|
101
|
-
(
|
100
|
+
if(task.background)
|
101
|
+
@finished_tasks << task
|
102
|
+
else
|
103
|
+
(@tasks_in_progress[js_handle] ||= []) << task
|
104
|
+
end
|
102
105
|
nil
|
103
106
|
end
|
104
107
|
private :handle_job_created
|
@@ -124,9 +127,32 @@ class TaskSet
|
|
124
127
|
end
|
125
128
|
nil
|
126
129
|
end
|
127
|
-
|
128
130
|
private :handle_work_complete
|
129
131
|
|
132
|
+
##
|
133
|
+
# Handle a 'work_exception' response from a job server.
|
134
|
+
#
|
135
|
+
# @param hostport "host:port" of job server
|
136
|
+
# @param data data returned in packet from server
|
137
|
+
def handle_work_exception(hostport, data)
|
138
|
+
handle, exception = data.split("\0", 3)
|
139
|
+
Util.log "Got work_exception with handle #{handle} from #{hostport}: '#{exception}'"
|
140
|
+
js_handle = Util.handle_to_str(hostport, handle)
|
141
|
+
tasks = @tasks_in_progress.delete(js_handle)
|
142
|
+
if not tasks
|
143
|
+
raise ProtocolError, "Got unexpected work_status with handle " +
|
144
|
+
"#{handle} from #{hostport} (no task by that name)"
|
145
|
+
end
|
146
|
+
tasks.each do |t|
|
147
|
+
if t.handle_exception(exception)
|
148
|
+
add_task_internal(t, false)
|
149
|
+
else
|
150
|
+
@finished_tasks << t
|
151
|
+
end
|
152
|
+
end
|
153
|
+
end
|
154
|
+
private :handle_work_exception
|
155
|
+
|
130
156
|
##
|
131
157
|
# Handle a 'work_fail' response from a job server.
|
132
158
|
#
|
@@ -189,6 +215,8 @@ class TaskSet
|
|
189
215
|
handle_work_fail(hostport, data)
|
190
216
|
when :work_status
|
191
217
|
handle_work_status(hostport, data)
|
218
|
+
when :work_exception
|
219
|
+
handle_work_exception(hostport, data)
|
192
220
|
else
|
193
221
|
Util.log "Got #{type.to_s} from #{hostport}"
|
194
222
|
end
|
@@ -229,7 +257,7 @@ class TaskSet
|
|
229
257
|
@sockets.values.each {|s| @client.return_socket(s) }
|
230
258
|
@sockets = {}
|
231
259
|
@finished_tasks.each do |t|
|
232
|
-
if
|
260
|
+
if ( (t.background.nil? || t.background == false) && !t.successful)
|
233
261
|
Util.log "Taskset failed"
|
234
262
|
return false
|
235
263
|
end
|
data/lib/gearman/testlib.rb
CHANGED
@@ -33,10 +33,10 @@ class FakeJobServer
|
|
33
33
|
@tester.assert_true(sock.closed?)
|
34
34
|
end
|
35
35
|
|
36
|
-
def expect_request(sock, exp_type, exp_data='')
|
37
|
-
head = sock.recv(
|
36
|
+
def expect_request(sock, exp_type, exp_data='', size=12)
|
37
|
+
head = sock.recv(size)
|
38
38
|
magic, type, len = head.unpack('a4NN')
|
39
|
-
@tester.
|
39
|
+
@tester.assert("\0REQ" == magic || "\000REQ" == magic)
|
40
40
|
@tester.assert_equal(Gearman::Util::NUMS[exp_type.to_sym], type)
|
41
41
|
data = len > 0 ? sock.recv(len) : ''
|
42
42
|
@tester.assert_equal(exp_data, data)
|
@@ -52,8 +52,7 @@ class FakeJobServer
|
|
52
52
|
end
|
53
53
|
|
54
54
|
def send_response(sock, type, data='', bogus_size=nil)
|
55
|
-
type_num = Gearman::Util::NUMS[type.to_sym]
|
56
|
-
raise RuntimeError, "Invalid type #{type}" if not type_num
|
55
|
+
type_num = Gearman::Util::NUMS[type.to_sym] || 0
|
57
56
|
response = "\0RES" + [type_num, (bogus_size or data.size)].pack('NN') + data
|
58
57
|
sock.write(response)
|
59
58
|
end
|
data/lib/gearman/util.rb
CHANGED
@@ -24,8 +24,11 @@ class Util
|
|
24
24
|
|
25
25
|
6 => :noop, # J->W: --
|
26
26
|
7 => :submit_job, # C->J: FUNC[0]UNIQ[0]ARGS
|
27
|
-
|
28
|
-
|
27
|
+
33 => :submit_job_low, # C->J: FUNC[0]UNIQ[0]ARGS
|
28
|
+
34 => :submit_job_low_bg, # C->J: FUNC[0]UNIQ[0]ARGS
|
29
|
+
21 => :submit_job_high, # C->J: FUNC[0]UNIQ[0]ARGS
|
30
|
+
32 => :submit_job_high_bg, # C->J: FUNC[0]UNIQ[0]ARGS
|
31
|
+
18 => :submit_job_bg, # C->J: FUNC[0]UNIQ[0]ARGS
|
29
32
|
|
30
33
|
8 => :job_created, # J->C: HANDLE
|
31
34
|
9 => :grab_job, # W->J: --
|
@@ -36,9 +39,14 @@ class Util
|
|
36
39
|
13 => :work_complete, # W->J/C: HANDLE[0]RES
|
37
40
|
14 => :work_fail, # W->J/C: HANDLE
|
38
41
|
|
42
|
+
25 => :work_exception, # W->J: HANDLE[0]ARG
|
43
|
+
26 => :option_req, # C->J: TEXT
|
44
|
+
27 => :option_res, # J->C: TEXT
|
45
|
+
|
39
46
|
15 => :get_status, # C->J: HANDLE
|
40
47
|
20 => :status_res, # C->J: HANDLE[0]KNOWN[0]RUNNING[0]NUM[0]DENOM
|
41
48
|
|
49
|
+
25 => :work_exception, # W->J/C: HANDLE[0]ARG
|
42
50
|
16 => :echo_req, # ?->J: TEXT
|
43
51
|
17 => :echo_res, # J->?: TEXT
|
44
52
|
|
@@ -198,15 +206,15 @@ class Util
|
|
198
206
|
raise ServerDownException.new(ex.message)
|
199
207
|
end
|
200
208
|
end
|
201
|
-
|
209
|
+
|
202
210
|
def Util.ability_name_with_prefix(prefix,name)
|
203
211
|
"#{prefix}\t#{name}"
|
204
212
|
end
|
205
|
-
|
213
|
+
|
206
214
|
class << self
|
207
215
|
alias :ability_name_for_perl :ability_name_with_prefix
|
208
216
|
end
|
209
|
-
|
217
|
+
|
210
218
|
end
|
211
219
|
|
212
220
|
end
|
data/lib/gearman/worker.rb
CHANGED
@@ -259,17 +259,25 @@ class Worker
|
|
259
259
|
return false
|
260
260
|
end
|
261
261
|
|
262
|
-
|
262
|
+
exception = nil
|
263
|
+
begin
|
264
|
+
ret = ability.run(data, Job.new(sock, handle))
|
265
|
+
rescue Exception => e
|
266
|
+
exception = e
|
267
|
+
end
|
268
|
+
|
263
269
|
|
264
|
-
cmd = nil
|
265
|
-
if ret
|
270
|
+
cmd = if ret && exception.nil?
|
266
271
|
ret = ret.to_s
|
267
272
|
Util.log "Sending work_complete for #{handle} with #{ret.size} byte(s) " +
|
268
273
|
"to #{hostport}"
|
269
|
-
|
270
|
-
|
274
|
+
Util.pack_request(:work_complete, "#{handle}\0#{ret}")
|
275
|
+
elsif exception.nil?
|
271
276
|
Util.log "Sending work_fail for #{handle} to #{hostport}"
|
272
|
-
|
277
|
+
Util.pack_request(:work_fail, handle)
|
278
|
+
elsif exception
|
279
|
+
Util.log "Sending work_exception for #{handle} to #{hostport}"
|
280
|
+
Util.pack_request(:work_exception, "#{handle}\0#{exception.message}")
|
273
281
|
end
|
274
282
|
|
275
283
|
Util.send_request(sock, cmd)
|
data/test/client_test.rb
CHANGED
@@ -108,4 +108,38 @@ class TestClient < Test::Unit::TestCase
|
|
108
108
|
end
|
109
109
|
assert(should_be_true)
|
110
110
|
end
|
111
|
+
|
112
|
+
def test_option_request_exceptions
|
113
|
+
this_server = FakeJobServer.new(self)
|
114
|
+
Thread.new do
|
115
|
+
server_socket = this_server.expect_connection
|
116
|
+
this_server.expect_request(server_socket, "option_req", "exceptions")
|
117
|
+
this_server.send_response(server_socket, :job_created, 'a')
|
118
|
+
end
|
119
|
+
client = Gearman::Client.new
|
120
|
+
hostport = "localhost:#{this_server.port}"
|
121
|
+
client.job_servers = [hostport]
|
122
|
+
client.option_request("exceptions")
|
123
|
+
end
|
124
|
+
|
125
|
+
def test_option_request_bad
|
126
|
+
this_server = FakeJobServer.new(self)
|
127
|
+
Thread.new do
|
128
|
+
server_socket = this_server.expect_connection
|
129
|
+
this_server.expect_request(server_socket, "option_req", "cccceptionsccc")
|
130
|
+
this_server.send_response(server_socket, :exception, 'a')
|
131
|
+
end
|
132
|
+
|
133
|
+
client = Gearman::Client.new
|
134
|
+
hostport = "localhost:#{this_server.port}"
|
135
|
+
client.job_servers = [hostport]
|
136
|
+
begin
|
137
|
+
client.option_request("cccceptionsccc")
|
138
|
+
assert(false)
|
139
|
+
rescue Gearman::ProtocolError
|
140
|
+
assert(true)
|
141
|
+
end
|
142
|
+
end
|
143
|
+
|
144
|
+
|
111
145
|
end
|
data/test/mock_client_test.rb
CHANGED
@@ -56,6 +56,127 @@ class TestClient < Test::Unit::TestCase
|
|
56
56
|
assert_equal(15, res2)
|
57
57
|
end
|
58
58
|
|
59
|
+
##
|
60
|
+
# Tests that the high priority option can be set in a job request
|
61
|
+
def test_client_submit_priority_high
|
62
|
+
server = FakeJobServer.new(self)
|
63
|
+
client, task1, task2, taskset, sock, res1, res2 = nil
|
64
|
+
|
65
|
+
s = TestScript.new
|
66
|
+
c = TestScript.new
|
67
|
+
|
68
|
+
server_thread = Thread.new { s.loop_forever }.run
|
69
|
+
client_thread = Thread.new { c.loop_forever }.run
|
70
|
+
|
71
|
+
c.exec { client = Gearman::Client.new("localhost:#{server.port}") }
|
72
|
+
|
73
|
+
c.exec { task1 = Gearman::Task.new('add', '5 2', { :priority => :high }) }
|
74
|
+
c.exec { task1.on_complete {|d| res1 = d.to_i } }
|
75
|
+
c.exec { taskset = Gearman::TaskSet.new(client) }
|
76
|
+
c.exec { taskset.add_task(task1) }
|
77
|
+
s.exec { sock = server.expect_connection }
|
78
|
+
s.wait
|
79
|
+
|
80
|
+
s.exec { server.expect_request(sock, :submit_job_high, "add\000\0005 2") }
|
81
|
+
end
|
82
|
+
|
83
|
+
##
|
84
|
+
# Tests that the low priority option can be set in a job request
|
85
|
+
def test_client_submit_priority_low
|
86
|
+
server = FakeJobServer.new(self)
|
87
|
+
client, task1, task2, taskset, sock, res1, res2 = nil
|
88
|
+
|
89
|
+
s = TestScript.new
|
90
|
+
c = TestScript.new
|
91
|
+
|
92
|
+
server_thread = Thread.new { s.loop_forever }.run
|
93
|
+
client_thread = Thread.new { c.loop_forever }.run
|
94
|
+
|
95
|
+
c.exec { client = Gearman::Client.new("localhost:#{server.port}") }
|
96
|
+
|
97
|
+
c.exec { task1 = Gearman::Task.new('add', '5 2', { :priority => :low }) }
|
98
|
+
c.exec { task1.on_complete {|d| res1 = d.to_i } }
|
99
|
+
c.exec { taskset = Gearman::TaskSet.new(client) }
|
100
|
+
c.exec { taskset.add_task(task1) }
|
101
|
+
s.exec { sock = server.expect_connection }
|
102
|
+
s.wait
|
103
|
+
|
104
|
+
s.exec { server.expect_request(sock, :submit_job_low, "add\000\0005 2") }
|
105
|
+
end
|
106
|
+
|
107
|
+
|
108
|
+
##
|
109
|
+
# Check that the client sends a correct background job request
|
110
|
+
def test_client_submit_background
|
111
|
+
server = FakeJobServer.new(self)
|
112
|
+
client, task1, task2, taskset, sock, res1, res2 = nil
|
113
|
+
|
114
|
+
s = TestScript.new
|
115
|
+
c = TestScript.new
|
116
|
+
|
117
|
+
server_thread = Thread.new { s.loop_forever }.run
|
118
|
+
client_thread = Thread.new { c.loop_forever }.run
|
119
|
+
|
120
|
+
c.exec { client = Gearman::Client.new("localhost:#{server.port}") }
|
121
|
+
|
122
|
+
c.exec { task1 = Gearman::Task.new('add', '5 2', { :background => :true }) }
|
123
|
+
c.exec { task1.on_complete {|d| res1 = d.to_i } }
|
124
|
+
c.exec { taskset = Gearman::TaskSet.new(client) }
|
125
|
+
c.exec { taskset.add_task(task1) }
|
126
|
+
s.exec { sock = server.expect_connection }
|
127
|
+
s.wait
|
128
|
+
|
129
|
+
s.exec { server.expect_request(sock, :submit_job_bg, "add\000\0005 2") }
|
130
|
+
end
|
131
|
+
|
132
|
+
##
|
133
|
+
# Check that the client sends a correct background job with high priority request
|
134
|
+
def test_client_submit_background
|
135
|
+
server = FakeJobServer.new(self)
|
136
|
+
client, task1, task2, taskset, sock, res1, res2 = nil
|
137
|
+
|
138
|
+
s = TestScript.new
|
139
|
+
c = TestScript.new
|
140
|
+
|
141
|
+
server_thread = Thread.new { s.loop_forever }.run
|
142
|
+
client_thread = Thread.new { c.loop_forever }.run
|
143
|
+
|
144
|
+
c.exec { client = Gearman::Client.new("localhost:#{server.port}") }
|
145
|
+
|
146
|
+
c.exec { task1 = Gearman::Task.new('add', '5 2', { :background => :true, :priority => :high }) }
|
147
|
+
c.exec { task1.on_complete {|d| res1 = d.to_i } }
|
148
|
+
c.exec { taskset = Gearman::TaskSet.new(client) }
|
149
|
+
c.exec { taskset.add_task(task1) }
|
150
|
+
s.exec { sock = server.expect_connection }
|
151
|
+
s.wait
|
152
|
+
|
153
|
+
s.exec { server.expect_request(sock, :submit_job_high_bg, "add\000\0005 2") }
|
154
|
+
end
|
155
|
+
|
156
|
+
##
|
157
|
+
# Check that the client sends a correct background job with low priority request
|
158
|
+
def test_client_submit_background
|
159
|
+
server = FakeJobServer.new(self)
|
160
|
+
client, task1, task2, taskset, sock, res1, res2 = nil
|
161
|
+
|
162
|
+
s = TestScript.new
|
163
|
+
c = TestScript.new
|
164
|
+
|
165
|
+
server_thread = Thread.new { s.loop_forever }.run
|
166
|
+
client_thread = Thread.new { c.loop_forever }.run
|
167
|
+
|
168
|
+
c.exec { client = Gearman::Client.new("localhost:#{server.port}") }
|
169
|
+
|
170
|
+
c.exec { task1 = Gearman::Task.new('add', '5 2', { :background => :true, :priority => :low }) }
|
171
|
+
c.exec { task1.on_complete {|d| res1 = d.to_i } }
|
172
|
+
c.exec { taskset = Gearman::TaskSet.new(client) }
|
173
|
+
c.exec { taskset.add_task(task1) }
|
174
|
+
s.exec { sock = server.expect_connection }
|
175
|
+
s.wait
|
176
|
+
|
177
|
+
s.exec { server.expect_request(sock, :submit_job_low_bg, "add\000\0005 2") }
|
178
|
+
end
|
179
|
+
|
59
180
|
##
|
60
181
|
# Test Client#do_task.
|
61
182
|
def test_do_task
|
@@ -185,6 +306,39 @@ class TestClient < Test::Unit::TestCase
|
|
185
306
|
assert_equal(false, setres)
|
186
307
|
end
|
187
308
|
|
309
|
+
def test_exception
|
310
|
+
server = FakeJobServer.new(self)
|
311
|
+
client, task, taskset, sock = nil
|
312
|
+
res,exception, setres = nil
|
313
|
+
|
314
|
+
s = TestScript.new
|
315
|
+
c = TestScript.new
|
316
|
+
|
317
|
+
server_thread = Thread.new { s.loop_forever }.run
|
318
|
+
client_thread = Thread.new { c.loop_forever }.run
|
319
|
+
|
320
|
+
c.exec { client = Gearman::Client.new("localhost:#{server.port}") }
|
321
|
+
|
322
|
+
c.exec { task = Gearman::Task.new('func2', 'b') }
|
323
|
+
c.exec { task.on_complete {|d| res = d } }
|
324
|
+
c.exec { task.on_exception {|message| exception=message; false } }
|
325
|
+
c.exec { taskset = Gearman::TaskSet.new(client) }
|
326
|
+
c.exec { taskset.add_task(task) }
|
327
|
+
s.exec { sock = server.expect_connection }
|
328
|
+
|
329
|
+
s.exec { server.expect_request(sock, :submit_job, "func2\000\000b") }
|
330
|
+
s.exec { server.send_response(sock, :job_created, 'b') }
|
331
|
+
|
332
|
+
s.exec { server.send_response(sock, :work_exception, "b\0exceptionmsg") }
|
333
|
+
|
334
|
+
c.exec { setres = taskset.wait }
|
335
|
+
c.wait
|
336
|
+
s.wait
|
337
|
+
|
338
|
+
assert_equal(nil, res)
|
339
|
+
assert_equal('exceptionmsg',exception)
|
340
|
+
end
|
341
|
+
|
188
342
|
##
|
189
343
|
# Test that user-supplied uniq values are handled correctly.
|
190
344
|
def test_uniq
|
data/test/mock_worker_test.rb
CHANGED
@@ -210,4 +210,48 @@ class TestWorker < Test::Unit::TestCase
|
|
210
210
|
s.wait
|
211
211
|
w.wait
|
212
212
|
end
|
213
|
+
|
214
|
+
def test_exception
|
215
|
+
@server = FakeJobServer.new(self)
|
216
|
+
worker = nil
|
217
|
+
sock = nil
|
218
|
+
|
219
|
+
s = TestScript.new
|
220
|
+
w = TestScript.new
|
221
|
+
|
222
|
+
server_thread = Thread.new { s.loop_forever }.run
|
223
|
+
worker_thread = Thread.new { w.loop_forever }.run
|
224
|
+
|
225
|
+
# Create a worker and wait for it to connect to us.
|
226
|
+
w.exec {
|
227
|
+
worker = Gearman::Worker.new(
|
228
|
+
"localhost:#{@server.port}", { :client_id => 'test' })
|
229
|
+
}
|
230
|
+
s.exec { sock = @server.expect_connection }
|
231
|
+
s.wait
|
232
|
+
|
233
|
+
# After it connects, it should send its ID, and it should tell us its
|
234
|
+
# abilities when we report them.
|
235
|
+
s.exec { @server.expect_request(sock, :set_client_id, 'test') }
|
236
|
+
w.exec { worker.add_ability('echo') {|d,j| raise Exception.new("fooexception") } }
|
237
|
+
s.exec { @server.expect_request(sock, :can_do, 'echo') }
|
238
|
+
|
239
|
+
# It should try to grab a job when we tell it to work.
|
240
|
+
w.exec { worker.work }
|
241
|
+
s.exec { @server.expect_request(sock, :grab_job) }
|
242
|
+
|
243
|
+
# If we tell it there aren't any jobs, it should go to sleep.
|
244
|
+
s.exec { @server.send_response(sock, :no_job) }
|
245
|
+
s.exec { @server.expect_request(sock, :pre_sleep) }
|
246
|
+
|
247
|
+
# When we send it a noop, it should wake up and ask for a job again.
|
248
|
+
s.exec { @server.send_response(sock, :noop) }
|
249
|
+
s.exec { @server.expect_request(sock, :grab_job) }
|
250
|
+
|
251
|
+
# When we give it a job, it should raise an excpetion and notify the server
|
252
|
+
s.exec { @server.send_response(sock, :job_assign, "a\0echo\0foo") }
|
253
|
+
s.exec { @server.expect_request(sock, :work_exception, "a\0fooexception") }
|
254
|
+
|
255
|
+
s.wait
|
256
|
+
end
|
213
257
|
end
|
data/test/util_test.rb
ADDED
@@ -0,0 +1,17 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
|
3
|
+
$:.unshift('../lib')
|
4
|
+
require 'gearman'
|
5
|
+
require 'gearman/testlib'
|
6
|
+
require 'test/unit'
|
7
|
+
|
8
|
+
class TestUtil < Test::Unit::TestCase
|
9
|
+
|
10
|
+
def test_ability_prefix_name_builder
|
11
|
+
assert_equal(Gearman::Util.ability_name_with_prefix("test","a"),"test\ta")
|
12
|
+
end
|
13
|
+
|
14
|
+
def test_ability_name_for_perl
|
15
|
+
assert_equal(Gearman::Util.ability_name_for_perl("test","a"),"test\ta")
|
16
|
+
end
|
17
|
+
end
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: xing-gearman-ruby
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 1.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Daniel Erat
|
@@ -10,7 +10,7 @@ autorequire:
|
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
12
|
|
13
|
-
date: 2009-
|
13
|
+
date: 2009-07-16 00:00:00 -07:00
|
14
14
|
default_executable:
|
15
15
|
dependencies: []
|
16
16
|
|
@@ -25,9 +25,11 @@ extra_rdoc_files:
|
|
25
25
|
- README
|
26
26
|
files:
|
27
27
|
- .gitignore
|
28
|
+
- HOWTO
|
28
29
|
- LICENSE
|
29
30
|
- README
|
30
31
|
- Rakefile
|
32
|
+
- TODO
|
31
33
|
- VERSION.yml
|
32
34
|
- examples/Net/Gearman/Client.php
|
33
35
|
- examples/Net/Gearman/Connection.php
|
@@ -40,13 +42,21 @@ files:
|
|
40
42
|
- examples/Net/Gearman/Set.php
|
41
43
|
- examples/Net/Gearman/Task.php
|
42
44
|
- examples/Net/Gearman/Worker.php
|
45
|
+
- examples/calculus_client.rb
|
46
|
+
- examples/calculus_worker.rb
|
43
47
|
- examples/client.php
|
44
48
|
- examples/client.rb
|
49
|
+
- examples/client_background.rb
|
50
|
+
- examples/client_exception.rb
|
51
|
+
- examples/client_prefix.rb
|
52
|
+
- examples/gearman_environment.sh
|
45
53
|
- examples/scale_image.rb
|
46
54
|
- examples/scale_image_worker.rb
|
47
55
|
- examples/server.rb
|
48
56
|
- examples/worker.php
|
49
57
|
- examples/worker.rb
|
58
|
+
- examples/worker_exception.rb
|
59
|
+
- examples/worker_prefix.rb
|
50
60
|
- gearman-ruby.gemspec
|
51
61
|
- lib/gearman.rb
|
52
62
|
- lib/gearman/client.rb
|
@@ -59,8 +69,9 @@ files:
|
|
59
69
|
- test/client_test.rb
|
60
70
|
- test/mock_client_test.rb
|
61
71
|
- test/mock_worker_test.rb
|
72
|
+
- test/util_test.rb
|
62
73
|
- test/worker_test.rb
|
63
|
-
has_rdoc:
|
74
|
+
has_rdoc: false
|
64
75
|
homepage: http://github.com/xing/gearman-ruby
|
65
76
|
post_install_message:
|
66
77
|
rdoc_options:
|
@@ -84,15 +95,23 @@ requirements: []
|
|
84
95
|
rubyforge_project:
|
85
96
|
rubygems_version: 1.2.0
|
86
97
|
signing_key:
|
87
|
-
specification_version:
|
98
|
+
specification_version: 3
|
88
99
|
summary: Library for the Gearman distributed job system
|
89
100
|
test_files:
|
90
101
|
- test/client_test.rb
|
91
102
|
- test/mock_client_test.rb
|
92
103
|
- test/mock_worker_test.rb
|
104
|
+
- test/util_test.rb
|
93
105
|
- test/worker_test.rb
|
106
|
+
- examples/calculus_client.rb
|
107
|
+
- examples/calculus_worker.rb
|
94
108
|
- examples/client.rb
|
109
|
+
- examples/client_background.rb
|
110
|
+
- examples/client_exception.rb
|
111
|
+
- examples/client_prefix.rb
|
95
112
|
- examples/scale_image.rb
|
96
113
|
- examples/scale_image_worker.rb
|
97
114
|
- examples/server.rb
|
98
115
|
- examples/worker.rb
|
116
|
+
- examples/worker_exception.rb
|
117
|
+
- examples/worker_prefix.rb
|