larva 0.3.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG +3 -0
- data/README.md +31 -17
- data/lib/larva/processor.rb +16 -0
- data/lib/larva/version.rb +1 -1
- data/test/processor_test.rb +26 -15
- metadata +1 -1
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 586b2fb5174324740d8583b8b6b8c033afa44f9f
|
4
|
+
data.tar.gz: 9b73d51991cc7664660a85d6cacd61500f29f283
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 20c5fb5ec779e501424ab581dc30001d8043dc61f512cca4bdee8ae1e38814efe49d14c12b579af2326923bbe4a8b6c8f3e9368a82a08b11158eb982757f47f9
|
7
|
+
data.tar.gz: bf35e9031228e9690e0a678c964c28c6b1b30d7cf433e9f7a6ae73132963ae0aa6fc6b5c27d69028969a0a14663a67534b8850ea9bfd8bd2d80fd51a9b4a20de
|
data/CHANGELOG
CHANGED
data/README.md
CHANGED
@@ -18,9 +18,9 @@ And then execute:
|
|
18
18
|
|
19
19
|
## Usage
|
20
20
|
|
21
|
-
Larva provides you with listeners, processors and a worker pool to build an application that listens and responds to Propono messages.
|
21
|
+
Larva provides you with listeners, processors and a worker pool to quickly build an application that listens and responds to Propono messages.
|
22
22
|
|
23
|
-
Here is a sample application
|
23
|
+
Here is a sample application that forms the basis of a rake task for most Meducation daemons.
|
24
24
|
|
25
25
|
```ruby
|
26
26
|
require 'larva'
|
@@ -28,18 +28,17 @@ require 'larva'
|
|
28
28
|
# Setup Config for Filum and Propono
|
29
29
|
|
30
30
|
class MyProcessor < Larva::Processor
|
31
|
-
def
|
32
|
-
|
33
|
-
# Do something...
|
34
|
-
else
|
35
|
-
false
|
36
|
-
end
|
31
|
+
def comment_created
|
32
|
+
# I get called when the message is received :)
|
37
33
|
end
|
38
34
|
end
|
39
35
|
|
40
36
|
processors = {my_topic: MyProcessor}
|
41
37
|
Larva::WorkerPool.start(processors, "queue-suffix")
|
42
38
|
|
39
|
+
# In another application...
|
40
|
+
Propono.publish(:my_topic, {entity: "comment", action: "created", id: 8}
|
41
|
+
|
43
42
|
```
|
44
43
|
|
45
44
|
### Listeners
|
@@ -54,28 +53,43 @@ This will listen for messages on :my_topic and pass them to `processor.process`.
|
|
54
53
|
|
55
54
|
### Processors
|
56
55
|
|
57
|
-
Processors are used by listeners to handle the messages that are received.
|
56
|
+
Processors are used by listeners to handle the messages that are received.
|
58
57
|
|
59
|
-
|
58
|
+
If your messages have an `entity` and `action` fields, then you can create methods named `#{entity}_#{action}, which get called when a message is received.
|
60
59
|
|
61
60
|
For example:
|
62
61
|
|
63
62
|
```ruby
|
63
|
+
class MyProcessor < Larva::Processor
|
64
|
+
def comment_created
|
65
|
+
# I get called for the first message
|
66
|
+
end
|
67
|
+
end
|
68
|
+
Larva::Listener.listen(:my_topic, MyProcessor, "")
|
69
|
+
Propono.publish(:my_topic, {entity: "comment", action: "created", id: 8}
|
70
|
+
```
|
71
|
+
|
72
|
+
If those methods do not exist, then a method called `process` is called. This method has acccess to `message`, `action`, `entity` and `id` fields. If this returns true, then the message is considered processed, else if it returns false, an error wil be logged.
|
73
|
+
|
74
|
+
For example:
|
75
|
+
|
76
|
+
``` ruby
|
64
77
|
class MyProcessor
|
65
|
-
def process
|
66
|
-
if
|
67
|
-
#
|
78
|
+
def process
|
79
|
+
if message[:foo] == bar
|
80
|
+
# Larva will consider this message processed successfully
|
81
|
+
true
|
68
82
|
else
|
83
|
+
# An error is logged for this message
|
69
84
|
false
|
70
85
|
end
|
71
86
|
end
|
72
87
|
end
|
73
88
|
Larva::Listener.listen(:my_topic, MyProcessor, "")
|
74
|
-
Propono.publish(:my_topic, {
|
89
|
+
Propono.publish(:my_topic, {foo: "bar"} # -> Will be logged as a success
|
90
|
+
Propono.publish(:my_topic, {foo: "meh"} # -> Will be logged as unprocessed.
|
75
91
|
```
|
76
92
|
|
77
|
-
With this code `MyProcessor#process` will get called for each message, with extra logging around the call. Within the class you have access to `message`, `action`, `entity` and `id` methods.
|
78
|
-
|
79
93
|
### Worker Pool
|
80
94
|
|
81
95
|
The worker pool creates a listener for each topic, and proxies messages to the associated processors. If any processors die, the application will die.
|
@@ -102,7 +116,7 @@ We'd love to have you involved. Please read our [contributing guide](https://git
|
|
102
116
|
|
103
117
|
### Contributors
|
104
118
|
|
105
|
-
This project is managed by the [Meducation team](http://company.meducation.net/about#team).
|
119
|
+
This project is managed by the [Meducation team](http://company.meducation.net/about#team).
|
106
120
|
|
107
121
|
These individuals have come up with the ideas and written the code that made this possible:
|
108
122
|
|
data/lib/larva/processor.rb
CHANGED
@@ -14,6 +14,22 @@ module Larva
|
|
14
14
|
|
15
15
|
def process_with_logging
|
16
16
|
Propono.config.logger.info "Processing message: #{message}"
|
17
|
+
meta_process || normal_process
|
18
|
+
end
|
19
|
+
|
20
|
+
private
|
21
|
+
|
22
|
+
def meta_process
|
23
|
+
meta_method = "#{entity}_#{action}"
|
24
|
+
if respond_to? meta_method
|
25
|
+
self.send(meta_method)
|
26
|
+
true
|
27
|
+
else
|
28
|
+
false
|
29
|
+
end
|
30
|
+
end
|
31
|
+
|
32
|
+
def normal_process
|
17
33
|
if self.process
|
18
34
|
Propono.config.logger.info "Message Processed: #{message}"
|
19
35
|
else
|
data/lib/larva/version.rb
CHANGED
data/test/processor_test.rb
CHANGED
@@ -2,6 +2,15 @@ require File.expand_path('../test_helper', __FILE__)
|
|
2
2
|
|
3
3
|
module Larva
|
4
4
|
class ProcessorTest < Minitest::Test
|
5
|
+
|
6
|
+
class MetaProcessor < Processor
|
7
|
+
def media_file_processed; end
|
8
|
+
end
|
9
|
+
|
10
|
+
class NormalProcessor < Processor
|
11
|
+
def process; end
|
12
|
+
end
|
13
|
+
|
5
14
|
def test_initialize_should_extract_action_and_entity
|
6
15
|
entity = "media_file"
|
7
16
|
action = "processed"
|
@@ -11,24 +20,24 @@ module Larva
|
|
11
20
|
assert_equal action, processor.action
|
12
21
|
end
|
13
22
|
|
14
|
-
class GoodProcessor < Processor
|
15
|
-
def process
|
16
|
-
true
|
17
|
-
end
|
18
|
-
end
|
19
|
-
|
20
|
-
class BadProcessor < Processor
|
21
|
-
def process
|
22
|
-
false
|
23
|
-
end
|
24
|
-
end
|
25
|
-
|
26
23
|
def test_process_logs_message
|
27
24
|
message = {entity: "media_file", action: "processed", media_file_id: "8"}
|
28
25
|
output = "Processing message: #{message}"
|
29
26
|
Propono.config.logger.stubs(:info)
|
30
27
|
Propono.config.logger.expects(:info).with(output)
|
31
|
-
|
28
|
+
NormalProcessor.process(message)
|
29
|
+
end
|
30
|
+
|
31
|
+
def test_process_calls_meta_method
|
32
|
+
message = {entity: "media_file", action: "processed", media_file_id: "8"}
|
33
|
+
MetaProcessor.any_instance.expects(:media_file_processed)
|
34
|
+
MetaProcessor.process(message)
|
35
|
+
end
|
36
|
+
|
37
|
+
def test_process_is_called_if_no_meta_method_exists
|
38
|
+
message = {entity: "media_file", action: "processed", media_file_id: "8"}
|
39
|
+
NormalProcessor.expects(:process)
|
40
|
+
NormalProcessor.process(message)
|
32
41
|
end
|
33
42
|
|
34
43
|
def test_process_logs_success
|
@@ -36,7 +45,8 @@ module Larva
|
|
36
45
|
output = "Message Processed: #{message}"
|
37
46
|
Propono.config.logger.stubs(:info)
|
38
47
|
Propono.config.logger.expects(:info).with(output)
|
39
|
-
|
48
|
+
NormalProcessor.any_instance.expects(:process).returns(true)
|
49
|
+
NormalProcessor.process(message)
|
40
50
|
end
|
41
51
|
|
42
52
|
def test_process_logs_message
|
@@ -47,7 +57,8 @@ module Larva
|
|
47
57
|
|
48
58
|
Propono.config.logger.stubs(:info)
|
49
59
|
Propono.config.logger.expects(:info).with(output)
|
50
|
-
|
60
|
+
NormalProcessor.any_instance.expects(:process).returns(false)
|
61
|
+
NormalProcessor.process(message)
|
51
62
|
end
|
52
63
|
end
|
53
64
|
end
|