eventhub-processor2 1.2.0 → 1.3.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +15 -0
- data/Gemfile.lock +1 -1
- data/README.md +24 -5
- data/example/README.md +6 -2
- data/lib/eventhub/base.rb +2 -0
- data/lib/eventhub/base_exception.rb +12 -0
- data/lib/eventhub/configuration.rb +13 -0
- data/lib/eventhub/helper.rb +5 -2
- data/lib/eventhub/version.rb +1 -1
- metadata +5 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: b9919c5a02188e630c89bb5faafd474776be3080
|
4
|
+
data.tar.gz: fe6131479d6c831ce7719fbf700fdd2de2ea6b74
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 37abfe5a7fe17cc8dd59be24ed4ed77d196443a25cc830dcb93ab00719ffe621c37963105c4dd42bafb48029614852ae0e3f80ab5832860dd2f6e2d89f794e15
|
7
|
+
data.tar.gz: '085633ac5ea1b55d06b2b77b6660add117b9ab87933802e285d3b4a34188b8df6297c8fb20fd79f2c8936630a59ed08e710535264a92c4429c9e33bb75619893'
|
data/CHANGELOG.md
ADDED
@@ -0,0 +1,15 @@
|
|
1
|
+
# Changelog of EventHub::Processor2
|
2
|
+
|
3
|
+
## 1.3.0 / 2018-02-13
|
4
|
+
|
5
|
+
* Have EventHub::BaseException for easier migration
|
6
|
+
* Have EventHub::Configuration.instance.data for easier migration
|
7
|
+
* Some minor documentation updates
|
8
|
+
|
9
|
+
## 1.2.0 / 2018-02-09
|
10
|
+
|
11
|
+
* With documentation added
|
12
|
+
|
13
|
+
## 1.0.0 / 2017-12-08
|
14
|
+
|
15
|
+
* Initial release
|
data/Gemfile.lock
CHANGED
data/README.md
CHANGED
@@ -13,6 +13,8 @@ Processor2 has currently the following sub-components implemented
|
|
13
13
|
* Watchdog - Checks regularly broker connection and defined listener queue(s)
|
14
14
|
* Listener - Listens to defined queues, parses recevied message into a EventHub::Message instance and calls handle_message method as defined in derived class.
|
15
15
|
|
16
|
+
Processor2 is using Bunny http://rubybunny.info a feature complete RabbitMQ Client to interact with message broker. Processor2 can deal with long running message processing.
|
17
|
+
|
16
18
|
## Installation
|
17
19
|
|
18
20
|
Add this line to your application's Gemfile:
|
@@ -58,11 +60,19 @@ module EventHub
|
|
58
60
|
# the processor2 gem and returned
|
59
61
|
# to the event_hub.inbound queue
|
60
62
|
|
63
|
+
# it is possible to publish a message during message processing but it's a
|
64
|
+
# good style to return one or multiple messages at end of handle_message
|
65
|
+
publish(message: 'your message as a string') # default exchange_name is 'event_hub.inbound'
|
66
|
+
publish(message: 'your message as string', exchange_name: 'your_specfic_exchange')
|
67
|
+
|
61
68
|
# at the end return one of
|
62
|
-
message # return message if sucessfull processing
|
69
|
+
message_to_return = message.copy # return message if sucessfull processing
|
70
|
+
# message.copy sets status.code automatically
|
71
|
+
# to EventHub::STATUS_SUCCESS which signals
|
72
|
+
# dispatcher successful processing
|
63
73
|
|
64
74
|
# or if you have multiple messages to return to event_hub.inbound queue
|
65
|
-
[
|
75
|
+
[ message_to_return, new_message1, new_message2]
|
66
76
|
|
67
77
|
# or if there is no message to return to event_hub.inbound queue
|
68
78
|
nil # [] works as well
|
@@ -125,6 +135,8 @@ If --config option is not provided processor tries to load config/{class_name}.j
|
|
125
135
|
}
|
126
136
|
```
|
127
137
|
|
138
|
+
More details about TLS configuration for underlying Bunny gem can be found here: http://rubybunny.info/articles/tls.html.
|
139
|
+
|
128
140
|
Feel free to define additional hash key/values (outside of server and processor key) as required by your application.
|
129
141
|
|
130
142
|
```json
|
@@ -145,11 +157,18 @@ Feel free to define additional hash key/values (outside of server and processor
|
|
145
157
|
}
|
146
158
|
```
|
147
159
|
|
160
|
+
Processor2 symbolizes keys and sub-keys from configuration files automatically.
|
148
161
|
```ruby
|
149
162
|
# access configuration values in your application as follows
|
150
|
-
Configuration.
|
151
|
-
Configuration.
|
152
|
-
Configuration.
|
163
|
+
EventHub::Configuration.database[:user] # => "guest"
|
164
|
+
EventHub::Configuration.database[:password] # => "secret"
|
165
|
+
EventHub::Configuration.database[:name][:subname] # => "value"
|
166
|
+
|
167
|
+
# If you need strings instead of symbols you can do
|
168
|
+
database = stringify_keys(EventHub::Configuration.database)
|
169
|
+
database['user'] # => "guest"
|
170
|
+
database['password'] # => "secret"
|
171
|
+
database['name']['subname'] # => "value"
|
153
172
|
```
|
154
173
|
|
155
174
|
## Development
|
data/example/README.md
CHANGED
@@ -3,13 +3,17 @@
|
|
3
3
|
### Description
|
4
4
|
|
5
5
|
Example folder contains a series of applications in order to test reliability and performance of processor2 gem.
|
6
|
+
* publisher.rb - Creates a unique file and a message
|
7
|
+
* router.rb - routes messges via queues
|
8
|
+
* receiver.rb - receives message and does final processing
|
9
|
+
* crasher.rb - restarts message broker or sends signals to other processes
|
6
10
|
|
7
11
|
### How does it work?
|
8
12
|
|
9
13
|
A message is passed throuhg the following components.
|
10
14
|
publisher.rb => [example.outbound] => router.rb => [example.inbound] => receiver.rb
|
11
15
|
|
12
|
-
1. publisher.rb generates a unique ID, creates a message with the ID as payload, passes the message to example.outbound queue.
|
16
|
+
1. publisher.rb generates a unique ID, creates a json message with the ID as payload, save the message in a file, and passes the message to example.outbound queue.
|
13
17
|
|
14
18
|
2. router.rb receives the message and passes it to exmaple.outbound queue
|
15
19
|
|
@@ -18,7 +22,7 @@ publisher.rb => [example.outbound] => router.rb => [example.inbound] => receiver
|
|
18
22
|
### Goal
|
19
23
|
What ever happens to these components (restarted, killed and restarted, stopped and started, message broker killed, stopped and started) if you do a graceful shutdown at the end there should be no message in the /data folder (except store.json).
|
20
24
|
|
21
|
-
Graceful shutdown with CTRL-C or TERM signal to
|
25
|
+
Graceful shutdown with CTRL-C or TERM signal to pid
|
22
26
|
* Stop producer.rb. Leave the other components running until all messages in example.* queues are gone.
|
23
27
|
* Stop remaining components
|
24
28
|
* Check ./example/data folder
|
data/lib/eventhub/base.rb
CHANGED
@@ -1,6 +1,7 @@
|
|
1
1
|
require 'uuidtools'
|
2
2
|
require 'json'
|
3
3
|
require 'base64'
|
4
|
+
require 'optparse'
|
4
5
|
|
5
6
|
require 'eventhub/components'
|
6
7
|
require 'logstash-logger'
|
@@ -16,6 +17,7 @@ end
|
|
16
17
|
|
17
18
|
require_relative 'version'
|
18
19
|
require_relative 'constant'
|
20
|
+
require_relative 'base_exception'
|
19
21
|
require_relative 'logger'
|
20
22
|
require_relative 'helper'
|
21
23
|
require_relative 'sleeper'
|
@@ -0,0 +1,12 @@
|
|
1
|
+
# EventHub module
|
2
|
+
module EventHub
|
3
|
+
# BaseException class
|
4
|
+
class BaseException < RuntimeError
|
5
|
+
attr_accessor :code, :message
|
6
|
+
def initialize(message=nil, code=EventHub::STATUS_DEADLETTER)
|
7
|
+
@message = message
|
8
|
+
@code = code
|
9
|
+
super(@message)
|
10
|
+
end
|
11
|
+
end
|
12
|
+
end
|
@@ -4,6 +4,7 @@ module EventHub
|
|
4
4
|
module Configuration
|
5
5
|
# it's a singleton (don't allow to instantiate this class)
|
6
6
|
extend self
|
7
|
+
extend Helper
|
7
8
|
|
8
9
|
attr_reader :name # name of processor
|
9
10
|
attr_reader :environment # environment the processor is running
|
@@ -116,5 +117,17 @@ module EventHub
|
|
116
117
|
}
|
117
118
|
}
|
118
119
|
end
|
120
|
+
|
121
|
+
def instance
|
122
|
+
warn "[DEPRECATION] `instance` is deprecated. Please use new"\
|
123
|
+
" configuration access method."
|
124
|
+
self
|
125
|
+
end
|
126
|
+
|
127
|
+
def data
|
128
|
+
warn "[DEPRECATION] `data` is deprecated. Please use new configuration"\
|
129
|
+
" access method."
|
130
|
+
stringify_keys(@config_data)
|
131
|
+
end
|
119
132
|
end
|
120
133
|
end
|
data/lib/eventhub/helper.rb
CHANGED
@@ -1,5 +1,3 @@
|
|
1
|
-
require 'optparse'
|
2
|
-
|
3
1
|
# EventHub module
|
4
2
|
module EventHub
|
5
3
|
# Helper module
|
@@ -56,5 +54,10 @@ module EventHub
|
|
56
54
|
now ||= Time.now
|
57
55
|
now.utc.strftime("%Y-%m-%dT%H:%M:%S.%6NZ")
|
58
56
|
end
|
57
|
+
|
58
|
+
# stringify hash keys
|
59
|
+
def stringify_keys(h)
|
60
|
+
JSON.parse(h.to_json)
|
61
|
+
end
|
59
62
|
end
|
60
63
|
end
|
data/lib/eventhub/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: eventhub-processor2
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 1.3.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Steiner, Thomas
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2018-02-
|
11
|
+
date: 2018-02-13 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: celluloid
|
@@ -132,6 +132,7 @@ files:
|
|
132
132
|
- ".gitignore"
|
133
133
|
- ".rspec"
|
134
134
|
- ".travis.yml"
|
135
|
+
- CHANGELOG.md
|
135
136
|
- Gemfile
|
136
137
|
- Gemfile.lock
|
137
138
|
- LICENSE.txt
|
@@ -156,6 +157,7 @@ files:
|
|
156
157
|
- lib/eventhub/actor_publisher.rb
|
157
158
|
- lib/eventhub/actor_watchdog.rb
|
158
159
|
- lib/eventhub/base.rb
|
160
|
+
- lib/eventhub/base_exception.rb
|
159
161
|
- lib/eventhub/configuration.rb
|
160
162
|
- lib/eventhub/constant.rb
|
161
163
|
- lib/eventhub/consumer.rb
|
@@ -187,7 +189,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
187
189
|
version: '0'
|
188
190
|
requirements: []
|
189
191
|
rubyforge_project:
|
190
|
-
rubygems_version: 2.6.
|
192
|
+
rubygems_version: 2.6.14
|
191
193
|
signing_key:
|
192
194
|
specification_version: 4
|
193
195
|
summary: Next generation gem to build ruby based eventhub processor
|