eventboss 1.3.1 → 1.3.2

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: fbfa930452474047053b798205334360c07c6a90e912a90cb3012e9b64c75113
4
- data.tar.gz: 9b4a1361b3bc0f0e72986394cd8055ec79983aee97dc128c7b3645d54179784d
3
+ metadata.gz: eaa1906368e8751476a4df0ae1f274a8008735de72191a34867881b50b19492b
4
+ data.tar.gz: 20bfe9695bc682e801b5b6760b92b9d575d0f323218b4286196aaff6f702d30f
5
5
  SHA512:
6
- metadata.gz: a483280c7c418e23e85d1e0727de6f8ec50b231e6c4dee3ca513526bb6ed653a3d6a8935c9dcaad8706ed1dd394bed37e33e26ad0f7e145a1e22624ff6d7b8d0
7
- data.tar.gz: 211225253db25ec12e0d8f60de4b6abd077c8cec38b9771b0558854d68eba018ea0f0f4c3cb83e580000b99acfde5f0acfa3b138f2f3f8fb239ece14ff590353
6
+ metadata.gz: a1e03452ce05f49cd5cbc09875467145d41c9fe8e6239941f874394a9895c617a2a2843daa479833a498d8ccbed6e6ac64b85ca3dab8dc73f42e2f980ba99468
7
+ data.tar.gz: 6c8b23c2ace8d611e3a79f98dcf15a11a97dbba537a46ee97df970730ad0107eef117547697cdebbd2e4ab97404dd99dff388f6ccbfbccfcd9b5fdad9b68d5ae
@@ -0,0 +1,5 @@
1
+ lang: ruby
2
+
3
+ script: bundle exec rake
4
+
5
+ rvm: 2.6.5
@@ -1,7 +1,7 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- eventboss (1.3.1)
4
+ eventboss (1.3.2)
5
5
  aws-sdk-sns (>= 1.1.0)
6
6
  aws-sdk-sqs (>= 1.3.0)
7
7
  dotenv (~> 2.1, >= 2.1.1)
@@ -10,16 +10,16 @@ GEM
10
10
  remote: https://rubygems.org/
11
11
  specs:
12
12
  aws-eventstream (1.0.3)
13
- aws-partitions (1.232.0)
14
- aws-sdk-core (3.73.0)
13
+ aws-partitions (1.269.0)
14
+ aws-sdk-core (3.89.1)
15
15
  aws-eventstream (~> 1.0, >= 1.0.2)
16
- aws-partitions (~> 1, >= 1.228.0)
16
+ aws-partitions (~> 1, >= 1.239.0)
17
17
  aws-sigv4 (~> 1.1)
18
18
  jmespath (~> 1.0)
19
- aws-sdk-sns (1.20.0)
19
+ aws-sdk-sns (1.21.0)
20
20
  aws-sdk-core (~> 3, >= 3.71.0)
21
21
  aws-sigv4 (~> 1.1)
22
- aws-sdk-sqs (1.23.0)
22
+ aws-sdk-sqs (1.23.1)
23
23
  aws-sdk-core (~> 3, >= 3.71.0)
24
24
  aws-sigv4 (~> 1.1)
25
25
  aws-sigv4 (1.1.0)
data/README.md CHANGED
@@ -1,6 +1,7 @@
1
1
  # Eventboss
2
2
 
3
3
  [![Gem Version](https://badge.fury.io/rb/eventboss.svg)](https://badge.fury.io/rb/eventboss)
4
+ [![Build Status](https://travis-ci.org/AirHelp/eventboss.svg?branch=master)](https://travis-ci.org/AirHelp/eventboss)
4
5
 
5
6
  AWS based Pub/Sub implementation in Ruby.
6
7
 
@@ -16,10 +17,10 @@ AWS based Pub/Sub implementation in Ruby.
16
17
  * [x] support multiple environments in the same AWS account
17
18
  * [x] pluggable error handlers (airbrake, newrelic)
18
19
  * [x] utility tasks (deadletter reload)
19
- * [x] localstack compatible
20
+ * [x] [localstack](https://github.com/localstack/localstack) compatible
20
21
  * [x] rails support (preloads rails environment)
22
+ * [x] development mode (creates missing SNS/SQS on the fly)
21
23
  * [ ] terraform pub/sub scripts
22
- * [ ] development mode (creates missing SNS/SQS on the fly)
23
24
  * [ ] alternative infrastructure (redis?, kafka?)
24
25
  * [ ] message compression
25
26
  * [ ] alternative serialization (protobuf)
@@ -93,15 +94,19 @@ end
93
94
  Using ENVs:
94
95
 
95
96
  ```
96
- EVENTBUS_ACCOUNT_ID=12345676
97
- EVENTBUS_APP_NAME=application_name
98
- EVENTBUS_ENV=env_name # production/staging/test
99
- EVENTBUS_REGION=aws_region # i.e. eu-west-1
100
- EVENTBUS_CONCURRENCY=10 # default is 25
97
+ EVENTBOSS_ACCOUNT_ID=12345676
98
+ EVENTBOSS_APP_NAME=application_name
99
+ EVENTBOSS_ENV=env_name # production/staging/test
100
+ EVENTBOSS_REGION=aws_region # i.e. eu-west-1
101
+ EVENTBOSS_CONCURRENCY=10 # default is 25
101
102
 
102
103
  AWS_SNS_ENDPOINT=http://localhost:4575 # when using with localstack
103
104
  AWS_SQS_ENDPOINT=http://localhost:4576 # when using with localstack
104
105
  ```
106
+ Use fixed account ID for localstack setup:
107
+ ```
108
+ EVENTBUS_ACCOUNT_ID=000000000000
109
+ ```
105
110
 
106
111
  Be aware that `eventbus:deadletter:reload` rake task won't load your configuration if you are not using ENVs
107
112
  in non Rails app, although to make it work you can extend your `Rakefile` with:
@@ -128,12 +133,12 @@ listeners:
128
133
  exclude:
129
134
  - OtherListener # When include option is not set it will run all listeners except listed here (OtherListener). When include is set it will run only included (but not excluded) listeners.
130
135
  ```
131
- Yml config is optional and by default is loaded from `'./config/eventboss.yml'`.
136
+ YAML config is optional and by default is loaded from `'./config/eventboss.yml'`.
132
137
  You can also pass config path as an argument:
133
- ```bash
138
+ ```sh
134
139
  eventboss -C my/custom/path/to/config.yml
135
140
  ```
136
- Yml config content is merged to configuration last, which means it overwrites ENVs and `.configure`.
141
+ YAML config content is merged to configuration last, which means it overwrites ENVs and `.configure`.
137
142
 
138
143
  ### Logging and error handling
139
144
  To have more verbose logging, set `log_level` in configuration (default is `info`).
@@ -146,9 +151,33 @@ Eventboss.configure do |config|
146
151
  end
147
152
  ```
148
153
 
154
+ ## Development mode
155
+
156
+ In the _development mode_ you don't need to create the infrastructure required by the application - Eventboss will take care of this.
157
+
158
+ It works on AWS and [localstack](https://github.com/localstack/localstack).
159
+
160
+ Following resources are created:
161
+ * SNS topics - created when application starts and when message is published
162
+ * SQS queues (with SendMessage policy) - created when application starts
163
+ * subscriptions for topics and queues - created when application starts
164
+
165
+ Just enable it via environment variable...
166
+ ```
167
+ EVENTBUS_DEVELOPMENT_MODE=true
168
+ ```
169
+ use fixed account ID for localstack setup...
170
+ ```
171
+ EVENTBUS_ACCOUNT_ID=000000000000 # or set it via YAML
172
+ ```
173
+ ...and you're good to go:
174
+ ```shell script
175
+ bundle exec eventboss
176
+ ```
177
+
149
178
  ## Topics & Queues naming convention
150
179
 
151
- The SNSes should be name in the following pattern:
180
+ The SNSes should be named in the following pattern:
152
181
  ```
153
182
  eventboss-{src_app_name}-{event_name}-{environment}
154
183
  ```
@@ -158,7 +187,7 @@ i.e.
158
187
  eventboss-srcapp-transaction_change-staging
159
188
  ```
160
189
 
161
- The corresponding SQSes should be name like:
190
+ The corresponding SQSes should be named like:
162
191
  ```
163
192
  {dest_app_name}-eventboss-{src_app_name}-{event_name}-{environment}
164
193
  {dest_app_name}-eventboss-{src_app_name}-{event_name}-{environment}-deadletter
@@ -177,3 +206,4 @@ Bug reports and pull requests are welcome on GitHub at https://github.com/AirHel
177
206
  ## License
178
207
 
179
208
  The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
209
+
@@ -18,8 +18,10 @@ require 'eventboss/worker'
18
18
  require 'eventboss/fetcher'
19
19
  require 'eventboss/publisher'
20
20
  require 'eventboss/sender'
21
+ require 'eventboss/topic'
21
22
  require 'eventboss/runner'
22
23
  require 'eventboss/extensions'
24
+ require 'eventboss/development_mode'
23
25
 
24
26
  # For Rails use railtie, for plain Ruby apps use custom scripts loader
25
27
  if defined?(Rails)
@@ -33,21 +35,34 @@ module Eventboss
33
35
 
34
36
  class << self
35
37
  def publisher(event_name, opts = {})
36
- Publisher.new(event_name, configuration.sns_client, configuration, opts)
38
+ sns_client = configuration.sns_client
39
+
40
+ if configuration.development_mode?
41
+ source_app = configuration.eventboss_app_name unless opts[:generic]
42
+ topic_name = Topic.build_name(event_name: event_name, source_app: source_app)
43
+ sns_client.create_topic(name: topic_name)
44
+ end
45
+
46
+ Publisher.new(event_name, sns_client, configuration, opts)
37
47
  end
38
48
 
39
- def sender(event_name, destination_app, options = {})
40
- queue_name = Queue.build_name(
41
- destination: destination_app,
42
- source: configuration.eventboss_app_name,
43
- event: event_name,
44
- env: env,
45
- generic: options[:generic]
49
+ def sender(event_name, destination, options = {})
50
+ source_app = configuration.eventboss_app_name unless options[:generic]
51
+ queue = Queue.build(
52
+ destination: destination,
53
+ source_app: source_app,
54
+ event_name: event_name,
55
+ env: env
46
56
  )
57
+ sqs_client = configuration.sqs_client
58
+
59
+ if configuration.development_mode?
60
+ sqs_client.create_queue(queue_name: queue.name)
61
+ end
47
62
 
48
63
  Sender.new(
49
- client: configuration.sqs_client,
50
- queue: Queue.new(queue_name)
64
+ client: sqs_client,
65
+ queue: queue
51
66
  )
52
67
  end
53
68
 
@@ -56,7 +71,7 @@ module Eventboss
56
71
  end
57
72
 
58
73
  def env
59
- @env ||= ENV['EVENTBUS_ENV'] || ENV['RAILS_ENV'] || ENV['RACK_ENV']
74
+ @env ||= ENV['EVENTBOSS_ENV'] || ENV['EVENTBUS_ENV'] || ENV['RAILS_ENV'] || ENV['RACK_ENV']
60
75
  end
61
76
 
62
77
  def configure
@@ -1,31 +1,33 @@
1
+ # frozen_string_literal: true
2
+
1
3
  module Eventboss
2
4
  class Configuration
3
5
  OPTS_ALLOWED_IN_CONFIG_FILE = %i[
4
6
  concurrency
5
7
  sns_sqs_name_infix
6
8
  listeners
7
- ]
9
+ ].freeze
8
10
 
9
11
  attr_writer :raise_on_missing_configuration,
10
- :error_handlers,
11
- :concurrency,
12
- :log_level,
13
- :logger,
14
- :sns_client,
15
- :sqs_client,
16
- :eventboss_region,
17
- :eventboss_app_name,
18
- :eventboss_account_id,
19
- :aws_access_key_id,
20
- :aws_secret_access_key,
21
- :aws_sns_endpoint,
22
- :aws_sqs_endpoint,
23
- :sns_sqs_name_infix,
24
- :listeners
12
+ :error_handlers,
13
+ :concurrency,
14
+ :log_level,
15
+ :logger,
16
+ :sns_client,
17
+ :sqs_client,
18
+ :eventboss_region,
19
+ :eventboss_app_name,
20
+ :eventboss_account_id,
21
+ :aws_access_key_id,
22
+ :aws_secret_access_key,
23
+ :aws_sns_endpoint,
24
+ :aws_sqs_endpoint,
25
+ :sns_sqs_name_infix,
26
+ :listeners
25
27
 
26
28
 
27
29
  def raise_on_missing_configuration
28
- defined_or_default('raise_on_missing_configuration') { ENV['EVENTBUS_RAISE_ON_MISSING_CONFIGURATION']&.downcase == 'true' }
30
+ defined_or_default('raise_on_missing_configuration') { (ENV['EVENTBOSS_RAISE_ON_MISSING_CONFIGURATION'] || ENV['EVENTBUS_RAISE_ON_MISSING_CONFIGURATION'])&.downcase == 'true' }
29
31
  end
30
32
 
31
33
  def error_handlers
@@ -37,7 +39,10 @@ module Eventboss
37
39
  end
38
40
 
39
41
  def concurrency
40
- defined_or_default('concurrency') { ENV['EVENTBUS_CONCURRENCY'] ? ENV['EVENTBUS_CONCURRENCY'].to_i : 25 }
42
+ defined_or_default('concurrency') do
43
+ concurrency = ENV['EVENTBOSS_CONCURRENCY'] || ENV['EVENTBUS_CONCURRENCY']
44
+ concurrency ? concurrency.to_i : 25
45
+ end
41
46
  end
42
47
 
43
48
  def log_level
@@ -72,15 +77,15 @@ module Eventboss
72
77
  end
73
78
 
74
79
  def eventboss_region
75
- defined_or_default('eventboss_region') { ENV['EVENTBUS_REGION'] }
80
+ defined_or_default('eventboss_region') { ENV['EVENTBOSS_REGION'] || ENV['EVENTBUS_REGION'] }
76
81
  end
77
82
 
78
83
  def eventboss_app_name
79
- defined_or_default('eventboss_app_name') { ENV['EVENTBUS_APP_NAME'] }
84
+ defined_or_default('eventboss_app_name') { ENV['EVENTBOSS_APP_NAME'] || ENV['EVENTBUS_APP_NAME'] }
80
85
  end
81
86
 
82
87
  def eventboss_account_id
83
- defined_or_default('eventboss_account_id') { ENV['EVENTBUS_ACCOUNT_ID'] }
88
+ defined_or_default('eventboss_account_id') { ENV['EVENTBOSS_ACCOUNT_ID'] || ENV['EVENTBUS_ACCOUNT_ID'] }
84
89
  end
85
90
 
86
91
  def aws_access_key_id
@@ -100,13 +105,19 @@ module Eventboss
100
105
  end
101
106
 
102
107
  def sns_sqs_name_infix
103
- defined_or_default('sns_sqs_name_infix') { ENV['EVENTBUS_SQS_SNS_NAME_INFIX'] || 'eventboss' }
108
+ defined_or_default('sns_sqs_name_infix') { ENV['EVENTBOSS_SQS_SNS_NAME_INFIX'] || ENV['EVENTBUS_SQS_SNS_NAME_INFIX'] || 'eventboss' }
104
109
  end
105
110
 
106
111
  def listeners
107
112
  defined_or_default('listeners') { {} }
108
113
  end
109
114
 
115
+ def development_mode?
116
+ defined_or_default('development_mode') do
117
+ (ENV['EVENTBOSS_DEVELOPMENT_MODE']&.downcase || ENV['EVENTBUS_DEVELOPMENT_MODE'])&.downcase == 'true'
118
+ end
119
+ end
120
+
110
121
  private
111
122
 
112
123
  def defined_or_default(variable_name)
@@ -0,0 +1,48 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Eventboss
4
+ module DevelopmentMode
5
+ extend Logging
6
+
7
+ class << self
8
+ def setup_infrastructure(queues)
9
+ sns_client = Eventboss.configuration.sns_client
10
+ sqs_client = Eventboss.configuration.sqs_client
11
+
12
+ queues.each do |queue, listener|
13
+ topic_name = Eventboss::Topic.build_name(**listener.options)
14
+ logger.info('development-mode') { "Creating topic #{topic_name}..." }
15
+ topic = sns_client.create_topic(name: topic_name)
16
+
17
+ logger.info('development-mode') { "Creating queue #{queue.name}..." }
18
+ sqs_client.create_queue(queue_name: queue.name)
19
+
20
+ logger.info('development-mode') { "Setting up queue #{queue.name} policy..." }
21
+ policy = queue_policy(queue.arn, topic.topic_arn)
22
+ sqs_client.set_queue_attributes(queue_url: queue.url, attributes: { Policy: policy.to_json })
23
+
24
+ logger.info('development-mode') { "Creating subscription for topic #{topic.topic_arn} and #{queue.arn}..." }
25
+ sns_client.create_subscription(topic_arn: topic.topic_arn, queue_arn: queue.arn)
26
+ end
27
+ end
28
+
29
+ def queue_policy(queue_arn, topic_arn)
30
+ {
31
+ "Version": "2012-10-17",
32
+ "Statement": [{
33
+ "Sid": "queue-policy-#{queue_arn}-#{topic_arn}",
34
+ "Effect": "Allow",
35
+ "Principal": "*",
36
+ "Action": ["SQS:SendMessage"],
37
+ "Resource": "#{queue_arn}",
38
+ "Condition": {
39
+ "ArnEquals": {
40
+ "aws:SourceArn": "#{topic_arn}"
41
+ }
42
+ }
43
+ }]
44
+ }
45
+ end
46
+ end
47
+ end
48
+ end
@@ -6,7 +6,7 @@ module Eventboss
6
6
  notice[:jid] = notice[:processor].jid if notice[:processor]
7
7
  notice[:processor] = notice[:processor].class.to_s if notice[:processor]
8
8
  Eventboss.logger.error(notice) do
9
- "Failure processing request #{exception.message}"
9
+ "Failure processing request: #{exception.message}"
10
10
  end
11
11
  end
12
12
  end
@@ -70,7 +70,7 @@ module Eventboss
70
70
  end
71
71
 
72
72
  def new_worker(id)
73
- Worker.new(self, id, @client, @bus)
73
+ Worker.new(self, id, @bus)
74
74
  end
75
75
 
76
76
  def new_poller(queue, listener)
@@ -17,15 +17,12 @@ module Eventboss
17
17
  end
18
18
 
19
19
  module ClassMethods
20
- def eventboss_options(opts)
21
- source_app = opts[:source_app] ? "#{opts[:source_app]}-" : ""
22
- event_name = opts[:event_name]
23
- destination_app = opts[:destination_app]
20
+ attr_reader :options
24
21
 
25
- ACTIVE_LISTENERS["#{source_app}#{event_name}"] = {
26
- listener: self,
27
- destination_app: destination_app
28
- }
22
+ def eventboss_options(options)
23
+ @options = options.compact
24
+
25
+ ACTIVE_LISTENERS[@options] = self
29
26
  end
30
27
  end
31
28
  end
@@ -44,7 +44,7 @@ module Eventboss
44
44
  def fetch_and_dispatch
45
45
  fetch_messages.each do |message|
46
46
  logger.debug(id) { "enqueueing message #{message.message_id}" }
47
- @bus << UnitOfWork.new(queue, listener, message)
47
+ @bus << UnitOfWork.new(@client, queue, listener, message)
48
48
  end
49
49
  end
50
50
 
@@ -1,32 +1,28 @@
1
+ # frozen_string_literal: true
2
+
1
3
  module Eventboss
2
4
  class Publisher
3
5
  def initialize(event_name, sns_client, configuration, opts = {})
4
6
  @event_name = event_name
5
7
  @sns_client = sns_client
6
8
  @configuration = configuration
7
- @generic = opts[:generic]
9
+ @source = configuration.eventboss_app_name unless opts[:generic]
8
10
  end
9
11
 
10
12
  def publish(payload)
11
- sns_client.publish({
13
+ topic_arn = Topic.build_arn(event_name: event_name, source_app: source)
14
+ sns_client.publish(
12
15
  topic_arn: topic_arn,
13
16
  message: json_payload(payload)
14
- })
17
+ )
15
18
  end
16
19
 
17
20
  private
18
21
 
19
- attr_reader :event_name, :sns_client, :configuration
22
+ attr_reader :event_name, :sns_client, :configuration, :source
20
23
 
21
24
  def json_payload(payload)
22
25
  payload.is_a?(String) ? payload : payload.to_json
23
26
  end
24
-
25
- def topic_arn
26
- src_selector = @generic ? "" : "-#{configuration.eventboss_app_name}"
27
-
28
- "arn:aws:sns:#{configuration.eventboss_region}:#{configuration.eventboss_account_id}:\
29
- #{Eventboss.configuration.sns_sqs_name_infix}#{src_selector}-#{event_name}-#{Eventboss.env}"
30
- end
31
27
  end
32
28
  end
@@ -1,21 +1,34 @@
1
+ # frozen_string_literal: true
2
+
1
3
  module Eventboss
2
4
  class Queue
3
5
  include Comparable
4
6
  attr_reader :name
5
7
 
6
- def self.build_name(source:, destination:, event:, env:, generic:)
7
- source =
8
- if generic
9
- ''
10
- else
11
- "#{source}-"
12
- end
8
+ class << self
9
+ def build_name(destination:, event_name:, env:, source_app: nil)
10
+ [
11
+ destination,
12
+ Eventboss.configuration.sns_sqs_name_infix,
13
+ source_app,
14
+ event_name,
15
+ env
16
+ ].compact.join('-')
17
+ end
13
18
 
14
- "#{destination}-#{Eventboss.configuration.sns_sqs_name_infix}-#{source}#{event}-#{env}"
19
+ def build(destination:, event_name:, env:, source_app: nil)
20
+ name = build_name(
21
+ destination: destination,
22
+ event_name: event_name,
23
+ env: env,
24
+ source_app: source_app
25
+ )
26
+ Queue.new(name)
27
+ end
15
28
  end
16
29
 
17
- def initialize(name, configuration = Eventboss.configuration)
18
- @client = configuration.sqs_client
30
+ def initialize(name)
31
+ @client = Eventboss.configuration.sqs_client
19
32
  @name = name
20
33
  end
21
34
 
@@ -23,6 +36,15 @@ module Eventboss
23
36
  @url ||= client.get_queue_url(queue_name: name).queue_url
24
37
  end
25
38
 
39
+ def arn
40
+ [
41
+ 'arn:aws:sqs',
42
+ Eventboss.configuration.eventboss_region,
43
+ Eventboss.configuration.eventboss_account_id,
44
+ name
45
+ ].join(':')
46
+ end
47
+
26
48
  def <=>(another_queue)
27
49
  name <=> another_queue&.name
28
50
  end
@@ -35,6 +57,10 @@ module Eventboss
35
57
  name.hash
36
58
  end
37
59
 
60
+ def to_s
61
+ "<Eventboss::Queue: #{name}>"
62
+ end
63
+
38
64
  private
39
65
 
40
66
  attr_reader :client
@@ -1,7 +1,7 @@
1
1
  module Eventboss
2
2
  class QueueListener
3
3
  class << self
4
- def select(include:, exclude:)
4
+ def select(include: nil, exclude: nil)
5
5
  listeners = list.values.map(&:name)
6
6
 
7
7
  listeners &= include if include
@@ -13,19 +13,15 @@ module Eventboss
13
13
  private
14
14
 
15
15
  def list
16
- Hash[Eventboss::Listener::ACTIVE_LISTENERS.map do |src_app_event, opts|
17
- [
18
- Eventboss::Queue.new(
19
- [
20
- opts[:destination_app] || Eventboss.configuration.eventboss_app_name,
21
- Eventboss.configuration.sns_sqs_name_infix,
22
- src_app_event,
23
- Eventboss.env
24
- ].join('-')
25
- ),
26
- opts[:listener]
27
- ]
28
- end]
16
+ Eventboss::Listener::ACTIVE_LISTENERS.each_with_object({}) do |(eventboss_options, listener), queue_listeners|
17
+ queue = Eventboss::Queue.build(
18
+ destination: eventboss_options[:destination_app] || Eventboss.configuration.eventboss_app_name,
19
+ source_app: eventboss_options[:source_app],
20
+ event_name: eventboss_options[:event_name],
21
+ env: Eventboss.env
22
+ )
23
+ queue_listeners[queue] = listener
24
+ end
29
25
  end
30
26
  end
31
27
  end
@@ -1,3 +1,5 @@
1
+ # frozen_string_literal: true
2
+
1
3
  module Eventboss
2
4
  class Runner
3
5
  extend Logging
@@ -17,8 +19,11 @@ module Eventboss
17
19
 
18
20
  self_read = setup_signals([:SIGTERM])
19
21
 
20
- logger.info('Active Listeners:')
21
- logger.info(queues.to_s)
22
+ logger.info('Active listeners:')
23
+ queues.each { |queue, listener| logger.info("#{queue}: #{listener}") }
24
+
25
+ Eventboss::DevelopmentMode.setup_infrastructure(queues) if config.development_mode?
26
+
22
27
  begin
23
28
  launcher.start
24
29
  handle_signals(self_read, launcher)
@@ -45,7 +50,7 @@ module Eventboss
45
50
  def handle_signals(self_read, launcher)
46
51
  while readable_io = IO.select([self_read])
47
52
  signal = readable_io.first[0].gets.strip
48
- logger.info('runner') { "Received #{ signal } signal, gracefully shutdowning..." }
53
+ logger.info('runner') { "Received #{signal} signal, gracefully shutting down..." }
49
54
 
50
55
  launcher.stop
51
56
  exit 0
@@ -1,5 +1,8 @@
1
+ # frozen_string_literal: true
2
+
1
3
  module Eventboss
2
- class NotConfigured < StandardError; end
4
+ class NotConfigured < StandardError;
5
+ end
3
6
 
4
7
  class SnsClient
5
8
  def initialize(configuration)
@@ -10,10 +13,31 @@ module Eventboss
10
13
  backend.publish(payload)
11
14
  end
12
15
 
16
+ def create_topic(name:)
17
+ backend.create_topic(name: name)
18
+ end
19
+
20
+ def create_subscription(topic_arn:, queue_arn:)
21
+ subscription = backend.subscribe(
22
+ topic_arn: topic_arn,
23
+ endpoint: queue_arn,
24
+ protocol: 'sqs'
25
+ )
26
+ set_raw_message_delivery(subscription)
27
+ end
28
+
13
29
  private
14
30
 
15
31
  attr_reader :configuration
16
32
 
33
+ def set_raw_message_delivery(subscription)
34
+ backend.set_subscription_attributes(
35
+ subscription_arn: subscription.subscription_arn,
36
+ attribute_name: 'RawMessageDelivery',
37
+ attribute_value: 'true'
38
+ )
39
+ end
40
+
17
41
  def backend
18
42
  if configured?
19
43
  options = {
@@ -26,11 +50,10 @@ module Eventboss
26
50
  if configuration.aws_sns_endpoint
27
51
  options[:endpoint] = configuration.aws_sns_endpoint
28
52
  end
29
- Aws::SNS::Client.new(
30
- options
31
- )
53
+
54
+ Aws::SNS::Client.new(options)
32
55
  elsif configuration.raise_on_missing_configuration
33
- raise NotConfigured, 'Eventboss is not configured'
56
+ raise NotConfigured, 'Eventboss is not configured.'
34
57
  else
35
58
  Mock.new
36
59
  end
@@ -0,0 +1,28 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Eventboss
4
+ class Topic
5
+ class << self
6
+ def build_arn(event_name:, source_app: nil)
7
+ [
8
+ 'arn:aws:sns',
9
+ Eventboss.configuration.eventboss_region,
10
+ Eventboss.configuration.eventboss_account_id,
11
+ build_name(
12
+ event_name: event_name,
13
+ source_app: source_app
14
+ )
15
+ ].join(':')
16
+ end
17
+
18
+ def build_name(event_name:, source_app: nil)
19
+ [
20
+ Eventboss.configuration.sns_sqs_name_infix,
21
+ source_app,
22
+ event_name,
23
+ Eventboss.env
24
+ ].compact.join('-')
25
+ end
26
+ end
27
+ end
28
+ end
@@ -6,14 +6,15 @@ module Eventboss
6
6
 
7
7
  attr_accessor :queue, :listener, :message
8
8
 
9
- def initialize(queue, listener, message)
9
+ def initialize(client, queue, listener, message)
10
+ @client = client
10
11
  @queue = queue
11
12
  @listener = listener
12
13
  @message = message
13
14
  @logger = logger
14
15
  end
15
16
 
16
- def run(client)
17
+ def run
17
18
  logger.debug(@message.message_id) { 'Started' }
18
19
  processor = @listener.new
19
20
  processor.receive(JSON.parse(@message.body))
@@ -21,21 +22,21 @@ module Eventboss
21
22
  rescue StandardError => exception
22
23
  handle_exception(exception, processor: processor, message_id: @message.message_id)
23
24
  else
24
- cleanup(client) unless processor.postponed_by
25
+ cleanup unless processor.postponed_by
25
26
  ensure
26
- change_message_visibility(client, processor.postponed_by) if processor.postponed_by
27
+ change_message_visibility(processor.postponed_by) if processor.postponed_by
27
28
  end
28
29
 
29
- def change_message_visibility(client, postponed_by)
30
- client.change_message_visibility(
30
+ def change_message_visibility(postponed_by)
31
+ @client.change_message_visibility(
31
32
  queue_url: @queue.url,
32
33
  receipt_handle: @message.receipt_handle,
33
34
  visibility_timeout: postponed_by
34
35
  )
35
36
  end
36
37
 
37
- def cleanup(client)
38
- client.delete_message(
38
+ def cleanup
39
+ @client.delete_message(
39
40
  queue_url: @queue.url, receipt_handle: @message.receipt_handle
40
41
  )
41
42
  logger.debug(@message.message_id) { 'Deleting' }
@@ -1,3 +1,3 @@
1
1
  module Eventboss
2
- VERSION = "1.3.1"
2
+ VERSION = "1.3.2"
3
3
  end
@@ -6,12 +6,12 @@ module Eventboss
6
6
 
7
7
  attr_reader :id
8
8
 
9
- def initialize(launcher, id, client, bus)
9
+ def initialize(launcher, id, bus, restart_on: [Exception])
10
10
  @id = "worker-#{id}"
11
11
  @launcher = launcher
12
- @client = client
13
12
  @bus = bus
14
13
  @thread = nil
14
+ @restart_on = restart_on
15
15
  end
16
16
 
17
17
  def start
@@ -20,12 +20,12 @@ module Eventboss
20
20
 
21
21
  def run
22
22
  while (work = @bus.pop)
23
- work.run(@client)
23
+ work.run
24
24
  end
25
25
  @launcher.worker_stopped(self)
26
26
  rescue Eventboss::Shutdown
27
27
  @launcher.worker_stopped(self)
28
- rescue Exception => exception
28
+ rescue *@restart_on => exception
29
29
  handle_exception(exception, worker_id: id)
30
30
  # Restart the worker in case of hard exception
31
31
  # Message won't be delete from SQS and will be visible later
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: eventboss
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.3.1
4
+ version: 1.3.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - AirHelp
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2019-11-05 00:00:00.000000000 Z
11
+ date: 2020-02-03 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: aws-sdk-sqs
@@ -108,9 +108,9 @@ executables:
108
108
  extensions: []
109
109
  extra_rdoc_files: []
110
110
  files:
111
- - ".github/workflows/ruby.yml"
112
111
  - ".gitignore"
113
112
  - ".rspec"
113
+ - ".travis.yml"
114
114
  - CHANGELOG.md
115
115
  - Gemfile
116
116
  - Gemfile.lock
@@ -123,6 +123,7 @@ files:
123
123
  - lib/eventboss.rb
124
124
  - lib/eventboss/cli.rb
125
125
  - lib/eventboss/configuration.rb
126
+ - lib/eventboss/development_mode.rb
126
127
  - lib/eventboss/error_handlers/airbrake.rb
127
128
  - lib/eventboss/error_handlers/db_connection_drop_handler.rb
128
129
  - lib/eventboss/error_handlers/logger.rb
@@ -142,6 +143,7 @@ files:
142
143
  - lib/eventboss/scripts.rb
143
144
  - lib/eventboss/sender.rb
144
145
  - lib/eventboss/sns_client.rb
146
+ - lib/eventboss/topic.rb
145
147
  - lib/eventboss/unit_of_work.rb
146
148
  - lib/eventboss/version.rb
147
149
  - lib/eventboss/worker.rb
@@ -1,20 +0,0 @@
1
- name: Ruby
2
-
3
- on: [push]
4
-
5
- jobs:
6
- build:
7
-
8
- runs-on: ubuntu-latest
9
-
10
- steps:
11
- - uses: actions/checkout@v1
12
- - name: Set up Ruby 2.6
13
- uses: actions/setup-ruby@v1
14
- with:
15
- ruby-version: 2.6.x
16
- - name: Build and test with Rake
17
- run: |
18
- gem install bundler
19
- bundle install --jobs 4 --retry 3
20
- bundle exec rake