divvy 1.0 → 1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/README.md +3 -3
- data/divvy.gemspec +1 -1
- data/example.rb +2 -2
- data/lib/divvy/parallelizable.rb +5 -5
- data/lib/divvy/worker.rb +1 -1
- data/test/master_test.rb +3 -3
- metadata +1 -1
data/README.md
CHANGED
@@ -18,7 +18,7 @@ and their associated storage components.
|
|
18
18
|
|
19
19
|
This is a simple and contrived example of a divvy job script. You must define a
|
20
20
|
class that includes the `Divvy::Parallelizable` module and implement the
|
21
|
-
`#dispatch` and `#
|
21
|
+
`#dispatch` and `#process` methods. There are also hooks available for tapping
|
22
22
|
into the worker process lifecycle.
|
23
23
|
|
24
24
|
``` ruby
|
@@ -38,7 +38,7 @@ class DanceParty
|
|
38
38
|
# This is the main loop responsible for generating work items for worker
|
39
39
|
# processes. It runs in the master process only. Each item yielded from this
|
40
40
|
# method is marshalled over a pipe and distributed to the next available
|
41
|
-
# worker process where it arrives at the #
|
41
|
+
# worker process where it arrives at the #process method (see below).
|
42
42
|
#
|
43
43
|
# In this example we're just going to generate a series of numbers to pass
|
44
44
|
# to the workers. The workers just write the number out with their pid and the
|
@@ -59,7 +59,7 @@ class DanceParty
|
|
59
59
|
# In this example we're given a Fixnum ticket number and asked to produce a
|
60
60
|
# code. Pretend this is a network intense operation where you're mostly
|
61
61
|
# sleeping waiting for a reply.
|
62
|
-
def
|
62
|
+
def process(ticket_number)
|
63
63
|
ticket_sha1 = Digest::SHA1.hexdigest(ticket_number.to_s)
|
64
64
|
printf "%5d %6d %s\n" % [$$, ticket_number, ticket_sha1]
|
65
65
|
sleep 0.150 # fake some latency
|
data/divvy.gemspec
CHANGED
data/example.rb
CHANGED
@@ -14,7 +14,7 @@ class DanceParty
|
|
14
14
|
# This is the main loop responsible for generating work items for worker
|
15
15
|
# processes. It runs in the master process only. Each item yielded from this
|
16
16
|
# method is marshalled over a pipe and distributed to the next available
|
17
|
-
# worker process where it arrives at the #
|
17
|
+
# worker process where it arrives at the #process method (see below).
|
18
18
|
#
|
19
19
|
# In this example we're just going to generate a series of numbers to pass
|
20
20
|
# to the workers. The workers just write the number out with their pid and the
|
@@ -35,7 +35,7 @@ class DanceParty
|
|
35
35
|
# In this example we're given a Fixnum ticket number and asked to produce a
|
36
36
|
# code. Pretend this is a network intense operation where you're mostly
|
37
37
|
# sleeping waiting for a reply.
|
38
|
-
def
|
38
|
+
def process(ticket_number)
|
39
39
|
ticket_sha1 = Digest::SHA1.hexdigest(ticket_number.to_s)
|
40
40
|
printf "%5d %6d %s\n" % [$$, ticket_number, ticket_sha1]
|
41
41
|
sleep 0.150 # fake some latency
|
data/lib/divvy/parallelizable.rb
CHANGED
@@ -1,15 +1,15 @@
|
|
1
1
|
module Divvy
|
2
2
|
# Module defining the main task interface. Parallelizable classes must respond
|
3
|
-
# to #dispatch and #
|
3
|
+
# to #dispatch and #process and may override hook methods to tap into the
|
4
4
|
# worker process lifecycle.
|
5
5
|
module Parallelizable
|
6
6
|
# The main loop responsible for generating task items to process in workers.
|
7
7
|
# Runs in the master process only. Each item this method yields is distributed
|
8
|
-
# to one of a pool of worker processes where #
|
8
|
+
# to one of a pool of worker processes where #process (see below) is invoked
|
9
9
|
# to process the task item.
|
10
10
|
#
|
11
11
|
# The arguments yielded to the block are passed with same arity to
|
12
|
-
# the #
|
12
|
+
# the #process method. Only marshallable types may be included.
|
13
13
|
#
|
14
14
|
# The dispatch method takes no arguments. It's expected that the receiving
|
15
15
|
# object is setup with all necessary state to generate task items or can
|
@@ -24,8 +24,8 @@ module Divvy
|
|
24
24
|
# Process an individual task item. Each item produced by #dispatch is sent here
|
25
25
|
# in one of a pool of the worker processes. The arguments to this method must
|
26
26
|
# match the arity of the task item yielded from #dispatch.
|
27
|
-
def
|
28
|
-
raise NotImplementedError, "#{self.class} must implement
|
27
|
+
def process(*args)
|
28
|
+
raise NotImplementedError, "#{self.class} must implement process method"
|
29
29
|
end
|
30
30
|
|
31
31
|
# Hook called after a worker process is forked off from the master process.
|
data/lib/divvy/worker.rb
CHANGED
data/test/master_test.rb
CHANGED
@@ -10,7 +10,7 @@ class MasterTest < MiniTest::Unit::TestCase
|
|
10
10
|
10.times(&block)
|
11
11
|
end
|
12
12
|
|
13
|
-
def
|
13
|
+
def process(num)
|
14
14
|
end
|
15
15
|
end
|
16
16
|
|
@@ -82,7 +82,7 @@ class MasterTest < MiniTest::Unit::TestCase
|
|
82
82
|
yield 'just one thing'
|
83
83
|
end
|
84
84
|
|
85
|
-
def
|
85
|
+
def process(arg)
|
86
86
|
if arg != 'just one thing'
|
87
87
|
fail "expected arg to be 'just one thing'"
|
88
88
|
end
|
@@ -102,7 +102,7 @@ class MasterTest < MiniTest::Unit::TestCase
|
|
102
102
|
10.times(&block)
|
103
103
|
end
|
104
104
|
|
105
|
-
def
|
105
|
+
def process(num)
|
106
106
|
if num % 2 == 0
|
107
107
|
fail "simulated failure"
|
108
108
|
else
|