eventboss 1.3.1 → 1.3.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: fbfa930452474047053b798205334360c07c6a90e912a90cb3012e9b64c75113
4
- data.tar.gz: 9b4a1361b3bc0f0e72986394cd8055ec79983aee97dc128c7b3645d54179784d
3
+ metadata.gz: eaa1906368e8751476a4df0ae1f274a8008735de72191a34867881b50b19492b
4
+ data.tar.gz: 20bfe9695bc682e801b5b6760b92b9d575d0f323218b4286196aaff6f702d30f
5
5
  SHA512:
6
- metadata.gz: a483280c7c418e23e85d1e0727de6f8ec50b231e6c4dee3ca513526bb6ed653a3d6a8935c9dcaad8706ed1dd394bed37e33e26ad0f7e145a1e22624ff6d7b8d0
7
- data.tar.gz: 211225253db25ec12e0d8f60de4b6abd077c8cec38b9771b0558854d68eba018ea0f0f4c3cb83e580000b99acfde5f0acfa3b138f2f3f8fb239ece14ff590353
6
+ metadata.gz: a1e03452ce05f49cd5cbc09875467145d41c9fe8e6239941f874394a9895c617a2a2843daa479833a498d8ccbed6e6ac64b85ca3dab8dc73f42e2f980ba99468
7
+ data.tar.gz: 6c8b23c2ace8d611e3a79f98dcf15a11a97dbba537a46ee97df970730ad0107eef117547697cdebbd2e4ab97404dd99dff388f6ccbfbccfcd9b5fdad9b68d5ae
@@ -0,0 +1,5 @@
1
+ lang: ruby
2
+
3
+ script: bundle exec rake
4
+
5
+ rvm: 2.6.5
@@ -1,7 +1,7 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- eventboss (1.3.1)
4
+ eventboss (1.3.2)
5
5
  aws-sdk-sns (>= 1.1.0)
6
6
  aws-sdk-sqs (>= 1.3.0)
7
7
  dotenv (~> 2.1, >= 2.1.1)
@@ -10,16 +10,16 @@ GEM
10
10
  remote: https://rubygems.org/
11
11
  specs:
12
12
  aws-eventstream (1.0.3)
13
- aws-partitions (1.232.0)
14
- aws-sdk-core (3.73.0)
13
+ aws-partitions (1.269.0)
14
+ aws-sdk-core (3.89.1)
15
15
  aws-eventstream (~> 1.0, >= 1.0.2)
16
- aws-partitions (~> 1, >= 1.228.0)
16
+ aws-partitions (~> 1, >= 1.239.0)
17
17
  aws-sigv4 (~> 1.1)
18
18
  jmespath (~> 1.0)
19
- aws-sdk-sns (1.20.0)
19
+ aws-sdk-sns (1.21.0)
20
20
  aws-sdk-core (~> 3, >= 3.71.0)
21
21
  aws-sigv4 (~> 1.1)
22
- aws-sdk-sqs (1.23.0)
22
+ aws-sdk-sqs (1.23.1)
23
23
  aws-sdk-core (~> 3, >= 3.71.0)
24
24
  aws-sigv4 (~> 1.1)
25
25
  aws-sigv4 (1.1.0)
data/README.md CHANGED
@@ -1,6 +1,7 @@
1
1
  # Eventboss
2
2
 
3
3
  [![Gem Version](https://badge.fury.io/rb/eventboss.svg)](https://badge.fury.io/rb/eventboss)
4
+ [![Build Status](https://travis-ci.org/AirHelp/eventboss.svg?branch=master)](https://travis-ci.org/AirHelp/eventboss)
4
5
 
5
6
  AWS based Pub/Sub implementation in Ruby.
6
7
 
@@ -16,10 +17,10 @@ AWS based Pub/Sub implementation in Ruby.
16
17
  * [x] support multiple environments in the same AWS account
17
18
  * [x] pluggable error handlers (airbrake, newrelic)
18
19
  * [x] utility tasks (deadletter reload)
19
- * [x] localstack compatible
20
+ * [x] [localstack](https://github.com/localstack/localstack) compatible
20
21
  * [x] rails support (preloads rails environment)
22
+ * [x] development mode (creates missing SNS/SQS on the fly)
21
23
  * [ ] terraform pub/sub scripts
22
- * [ ] development mode (creates missing SNS/SQS on the fly)
23
24
  * [ ] alternative infrastructure (redis?, kafka?)
24
25
  * [ ] message compression
25
26
  * [ ] alternative serialization (protobuf)
@@ -93,15 +94,19 @@ end
93
94
  Using ENVs:
94
95
 
95
96
  ```
96
- EVENTBUS_ACCOUNT_ID=12345676
97
- EVENTBUS_APP_NAME=application_name
98
- EVENTBUS_ENV=env_name # production/staging/test
99
- EVENTBUS_REGION=aws_region # i.e. eu-west-1
100
- EVENTBUS_CONCURRENCY=10 # default is 25
97
+ EVENTBOSS_ACCOUNT_ID=12345676
98
+ EVENTBOSS_APP_NAME=application_name
99
+ EVENTBOSS_ENV=env_name # production/staging/test
100
+ EVENTBOSS_REGION=aws_region # i.e. eu-west-1
101
+ EVENTBOSS_CONCURRENCY=10 # default is 25
101
102
 
102
103
  AWS_SNS_ENDPOINT=http://localhost:4575 # when using with localstack
103
104
  AWS_SQS_ENDPOINT=http://localhost:4576 # when using with localstack
104
105
  ```
106
+ Use fixed account ID for localstack setup:
107
+ ```
108
+ EVENTBUS_ACCOUNT_ID=000000000000
109
+ ```
105
110
 
106
111
  Be aware that `eventbus:deadletter:reload` rake task won't load your configuration if you are not using ENVs
107
112
  in non Rails app, although to make it work you can extend your `Rakefile` with:
@@ -128,12 +133,12 @@ listeners:
128
133
  exclude:
129
134
  - OtherListener # When include option is not set it will run all listeners except listed here (OtherListener). When include is set it will run only included (but not excluded) listeners.
130
135
  ```
131
- Yml config is optional and by default is loaded from `'./config/eventboss.yml'`.
136
+ YAML config is optional and by default is loaded from `'./config/eventboss.yml'`.
132
137
  You can also pass config path as an argument:
133
- ```bash
138
+ ```sh
134
139
  eventboss -C my/custom/path/to/config.yml
135
140
  ```
136
- Yml config content is merged to configuration last, which means it overwrites ENVs and `.configure`.
141
+ YAML config content is merged to configuration last, which means it overwrites ENVs and `.configure`.
137
142
 
138
143
  ### Logging and error handling
139
144
  To have more verbose logging, set `log_level` in configuration (default is `info`).
@@ -146,9 +151,33 @@ Eventboss.configure do |config|
146
151
  end
147
152
  ```
148
153
 
154
+ ## Development mode
155
+
156
+ In the _development mode_ you don't need to create the infrastructure required by the application - Eventboss will take care of this.
157
+
158
+ It works on AWS and [localstack](https://github.com/localstack/localstack).
159
+
160
+ Following resources are created:
161
+ * SNS topics - created when application starts and when message is published
162
+ * SQS queues (with SendMessage policy) - created when application starts
163
+ * subscriptions for topics and queues - created when application starts
164
+
165
+ Just enable it via environment variable...
166
+ ```
167
+ EVENTBUS_DEVELOPMENT_MODE=true
168
+ ```
169
+ use fixed account ID for localstack setup...
170
+ ```
171
+ EVENTBUS_ACCOUNT_ID=000000000000 # or set it via YAML
172
+ ```
173
+ ...and you're good to go:
174
+ ```shell script
175
+ bundle exec eventboss
176
+ ```
177
+
149
178
  ## Topics & Queues naming convention
150
179
 
151
- The SNSes should be name in the following pattern:
180
+ The SNSes should be named in the following pattern:
152
181
  ```
153
182
  eventboss-{src_app_name}-{event_name}-{environment}
154
183
  ```
@@ -158,7 +187,7 @@ i.e.
158
187
  eventboss-srcapp-transaction_change-staging
159
188
  ```
160
189
 
161
- The corresponding SQSes should be name like:
190
+ The corresponding SQSes should be named like:
162
191
  ```
163
192
  {dest_app_name}-eventboss-{src_app_name}-{event_name}-{environment}
164
193
  {dest_app_name}-eventboss-{src_app_name}-{event_name}-{environment}-deadletter
@@ -177,3 +206,4 @@ Bug reports and pull requests are welcome on GitHub at https://github.com/AirHel
177
206
  ## License
178
207
 
179
208
  The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
209
+
@@ -18,8 +18,10 @@ require 'eventboss/worker'
18
18
  require 'eventboss/fetcher'
19
19
  require 'eventboss/publisher'
20
20
  require 'eventboss/sender'
21
+ require 'eventboss/topic'
21
22
  require 'eventboss/runner'
22
23
  require 'eventboss/extensions'
24
+ require 'eventboss/development_mode'
23
25
 
24
26
  # For Rails use railtie, for plain Ruby apps use custom scripts loader
25
27
  if defined?(Rails)
@@ -33,21 +35,34 @@ module Eventboss
33
35
 
34
36
  class << self
35
37
  def publisher(event_name, opts = {})
36
- Publisher.new(event_name, configuration.sns_client, configuration, opts)
38
+ sns_client = configuration.sns_client
39
+
40
+ if configuration.development_mode?
41
+ source_app = configuration.eventboss_app_name unless opts[:generic]
42
+ topic_name = Topic.build_name(event_name: event_name, source_app: source_app)
43
+ sns_client.create_topic(name: topic_name)
44
+ end
45
+
46
+ Publisher.new(event_name, sns_client, configuration, opts)
37
47
  end
38
48
 
39
- def sender(event_name, destination_app, options = {})
40
- queue_name = Queue.build_name(
41
- destination: destination_app,
42
- source: configuration.eventboss_app_name,
43
- event: event_name,
44
- env: env,
45
- generic: options[:generic]
49
+ def sender(event_name, destination, options = {})
50
+ source_app = configuration.eventboss_app_name unless options[:generic]
51
+ queue = Queue.build(
52
+ destination: destination,
53
+ source_app: source_app,
54
+ event_name: event_name,
55
+ env: env
46
56
  )
57
+ sqs_client = configuration.sqs_client
58
+
59
+ if configuration.development_mode?
60
+ sqs_client.create_queue(queue_name: queue.name)
61
+ end
47
62
 
48
63
  Sender.new(
49
- client: configuration.sqs_client,
50
- queue: Queue.new(queue_name)
64
+ client: sqs_client,
65
+ queue: queue
51
66
  )
52
67
  end
53
68
 
@@ -56,7 +71,7 @@ module Eventboss
56
71
  end
57
72
 
58
73
  def env
59
- @env ||= ENV['EVENTBUS_ENV'] || ENV['RAILS_ENV'] || ENV['RACK_ENV']
74
+ @env ||= ENV['EVENTBOSS_ENV'] || ENV['EVENTBUS_ENV'] || ENV['RAILS_ENV'] || ENV['RACK_ENV']
60
75
  end
61
76
 
62
77
  def configure
@@ -1,31 +1,33 @@
1
+ # frozen_string_literal: true
2
+
1
3
  module Eventboss
2
4
  class Configuration
3
5
  OPTS_ALLOWED_IN_CONFIG_FILE = %i[
4
6
  concurrency
5
7
  sns_sqs_name_infix
6
8
  listeners
7
- ]
9
+ ].freeze
8
10
 
9
11
  attr_writer :raise_on_missing_configuration,
10
- :error_handlers,
11
- :concurrency,
12
- :log_level,
13
- :logger,
14
- :sns_client,
15
- :sqs_client,
16
- :eventboss_region,
17
- :eventboss_app_name,
18
- :eventboss_account_id,
19
- :aws_access_key_id,
20
- :aws_secret_access_key,
21
- :aws_sns_endpoint,
22
- :aws_sqs_endpoint,
23
- :sns_sqs_name_infix,
24
- :listeners
12
+ :error_handlers,
13
+ :concurrency,
14
+ :log_level,
15
+ :logger,
16
+ :sns_client,
17
+ :sqs_client,
18
+ :eventboss_region,
19
+ :eventboss_app_name,
20
+ :eventboss_account_id,
21
+ :aws_access_key_id,
22
+ :aws_secret_access_key,
23
+ :aws_sns_endpoint,
24
+ :aws_sqs_endpoint,
25
+ :sns_sqs_name_infix,
26
+ :listeners
25
27
 
26
28
 
27
29
  def raise_on_missing_configuration
28
- defined_or_default('raise_on_missing_configuration') { ENV['EVENTBUS_RAISE_ON_MISSING_CONFIGURATION']&.downcase == 'true' }
30
+ defined_or_default('raise_on_missing_configuration') { (ENV['EVENTBOSS_RAISE_ON_MISSING_CONFIGURATION'] || ENV['EVENTBUS_RAISE_ON_MISSING_CONFIGURATION'])&.downcase == 'true' }
29
31
  end
30
32
 
31
33
  def error_handlers
@@ -37,7 +39,10 @@ module Eventboss
37
39
  end
38
40
 
39
41
  def concurrency
40
- defined_or_default('concurrency') { ENV['EVENTBUS_CONCURRENCY'] ? ENV['EVENTBUS_CONCURRENCY'].to_i : 25 }
42
+ defined_or_default('concurrency') do
43
+ concurrency = ENV['EVENTBOSS_CONCURRENCY'] || ENV['EVENTBUS_CONCURRENCY']
44
+ concurrency ? concurrency.to_i : 25
45
+ end
41
46
  end
42
47
 
43
48
  def log_level
@@ -72,15 +77,15 @@ module Eventboss
72
77
  end
73
78
 
74
79
  def eventboss_region
75
- defined_or_default('eventboss_region') { ENV['EVENTBUS_REGION'] }
80
+ defined_or_default('eventboss_region') { ENV['EVENTBOSS_REGION'] || ENV['EVENTBUS_REGION'] }
76
81
  end
77
82
 
78
83
  def eventboss_app_name
79
- defined_or_default('eventboss_app_name') { ENV['EVENTBUS_APP_NAME'] }
84
+ defined_or_default('eventboss_app_name') { ENV['EVENTBOSS_APP_NAME'] || ENV['EVENTBUS_APP_NAME'] }
80
85
  end
81
86
 
82
87
  def eventboss_account_id
83
- defined_or_default('eventboss_account_id') { ENV['EVENTBUS_ACCOUNT_ID'] }
88
+ defined_or_default('eventboss_account_id') { ENV['EVENTBOSS_ACCOUNT_ID'] || ENV['EVENTBUS_ACCOUNT_ID'] }
84
89
  end
85
90
 
86
91
  def aws_access_key_id
@@ -100,13 +105,19 @@ module Eventboss
100
105
  end
101
106
 
102
107
  def sns_sqs_name_infix
103
- defined_or_default('sns_sqs_name_infix') { ENV['EVENTBUS_SQS_SNS_NAME_INFIX'] || 'eventboss' }
108
+ defined_or_default('sns_sqs_name_infix') { ENV['EVENTBOSS_SQS_SNS_NAME_INFIX'] || ENV['EVENTBUS_SQS_SNS_NAME_INFIX'] || 'eventboss' }
104
109
  end
105
110
 
106
111
  def listeners
107
112
  defined_or_default('listeners') { {} }
108
113
  end
109
114
 
115
+ def development_mode?
116
+ defined_or_default('development_mode') do
117
+ (ENV['EVENTBOSS_DEVELOPMENT_MODE']&.downcase || ENV['EVENTBUS_DEVELOPMENT_MODE'])&.downcase == 'true'
118
+ end
119
+ end
120
+
110
121
  private
111
122
 
112
123
  def defined_or_default(variable_name)
@@ -0,0 +1,48 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Eventboss
4
+ module DevelopmentMode
5
+ extend Logging
6
+
7
+ class << self
8
+ def setup_infrastructure(queues)
9
+ sns_client = Eventboss.configuration.sns_client
10
+ sqs_client = Eventboss.configuration.sqs_client
11
+
12
+ queues.each do |queue, listener|
13
+ topic_name = Eventboss::Topic.build_name(**listener.options)
14
+ logger.info('development-mode') { "Creating topic #{topic_name}..." }
15
+ topic = sns_client.create_topic(name: topic_name)
16
+
17
+ logger.info('development-mode') { "Creating queue #{queue.name}..." }
18
+ sqs_client.create_queue(queue_name: queue.name)
19
+
20
+ logger.info('development-mode') { "Setting up queue #{queue.name} policy..." }
21
+ policy = queue_policy(queue.arn, topic.topic_arn)
22
+ sqs_client.set_queue_attributes(queue_url: queue.url, attributes: { Policy: policy.to_json })
23
+
24
+ logger.info('development-mode') { "Creating subscription for topic #{topic.topic_arn} and #{queue.arn}..." }
25
+ sns_client.create_subscription(topic_arn: topic.topic_arn, queue_arn: queue.arn)
26
+ end
27
+ end
28
+
29
+ def queue_policy(queue_arn, topic_arn)
30
+ {
31
+ "Version": "2012-10-17",
32
+ "Statement": [{
33
+ "Sid": "queue-policy-#{queue_arn}-#{topic_arn}",
34
+ "Effect": "Allow",
35
+ "Principal": "*",
36
+ "Action": ["SQS:SendMessage"],
37
+ "Resource": "#{queue_arn}",
38
+ "Condition": {
39
+ "ArnEquals": {
40
+ "aws:SourceArn": "#{topic_arn}"
41
+ }
42
+ }
43
+ }]
44
+ }
45
+ end
46
+ end
47
+ end
48
+ end
@@ -6,7 +6,7 @@ module Eventboss
6
6
  notice[:jid] = notice[:processor].jid if notice[:processor]
7
7
  notice[:processor] = notice[:processor].class.to_s if notice[:processor]
8
8
  Eventboss.logger.error(notice) do
9
- "Failure processing request #{exception.message}"
9
+ "Failure processing request: #{exception.message}"
10
10
  end
11
11
  end
12
12
  end
@@ -70,7 +70,7 @@ module Eventboss
70
70
  end
71
71
 
72
72
  def new_worker(id)
73
- Worker.new(self, id, @client, @bus)
73
+ Worker.new(self, id, @bus)
74
74
  end
75
75
 
76
76
  def new_poller(queue, listener)
@@ -17,15 +17,12 @@ module Eventboss
17
17
  end
18
18
 
19
19
  module ClassMethods
20
- def eventboss_options(opts)
21
- source_app = opts[:source_app] ? "#{opts[:source_app]}-" : ""
22
- event_name = opts[:event_name]
23
- destination_app = opts[:destination_app]
20
+ attr_reader :options
24
21
 
25
- ACTIVE_LISTENERS["#{source_app}#{event_name}"] = {
26
- listener: self,
27
- destination_app: destination_app
28
- }
22
+ def eventboss_options(options)
23
+ @options = options.compact
24
+
25
+ ACTIVE_LISTENERS[@options] = self
29
26
  end
30
27
  end
31
28
  end
@@ -44,7 +44,7 @@ module Eventboss
44
44
  def fetch_and_dispatch
45
45
  fetch_messages.each do |message|
46
46
  logger.debug(id) { "enqueueing message #{message.message_id}" }
47
- @bus << UnitOfWork.new(queue, listener, message)
47
+ @bus << UnitOfWork.new(@client, queue, listener, message)
48
48
  end
49
49
  end
50
50
 
@@ -1,32 +1,28 @@
1
+ # frozen_string_literal: true
2
+
1
3
  module Eventboss
2
4
  class Publisher
3
5
  def initialize(event_name, sns_client, configuration, opts = {})
4
6
  @event_name = event_name
5
7
  @sns_client = sns_client
6
8
  @configuration = configuration
7
- @generic = opts[:generic]
9
+ @source = configuration.eventboss_app_name unless opts[:generic]
8
10
  end
9
11
 
10
12
  def publish(payload)
11
- sns_client.publish({
13
+ topic_arn = Topic.build_arn(event_name: event_name, source_app: source)
14
+ sns_client.publish(
12
15
  topic_arn: topic_arn,
13
16
  message: json_payload(payload)
14
- })
17
+ )
15
18
  end
16
19
 
17
20
  private
18
21
 
19
- attr_reader :event_name, :sns_client, :configuration
22
+ attr_reader :event_name, :sns_client, :configuration, :source
20
23
 
21
24
  def json_payload(payload)
22
25
  payload.is_a?(String) ? payload : payload.to_json
23
26
  end
24
-
25
- def topic_arn
26
- src_selector = @generic ? "" : "-#{configuration.eventboss_app_name}"
27
-
28
- "arn:aws:sns:#{configuration.eventboss_region}:#{configuration.eventboss_account_id}:\
29
- #{Eventboss.configuration.sns_sqs_name_infix}#{src_selector}-#{event_name}-#{Eventboss.env}"
30
- end
31
27
  end
32
28
  end
@@ -1,21 +1,34 @@
1
+ # frozen_string_literal: true
2
+
1
3
  module Eventboss
2
4
  class Queue
3
5
  include Comparable
4
6
  attr_reader :name
5
7
 
6
- def self.build_name(source:, destination:, event:, env:, generic:)
7
- source =
8
- if generic
9
- ''
10
- else
11
- "#{source}-"
12
- end
8
+ class << self
9
+ def build_name(destination:, event_name:, env:, source_app: nil)
10
+ [
11
+ destination,
12
+ Eventboss.configuration.sns_sqs_name_infix,
13
+ source_app,
14
+ event_name,
15
+ env
16
+ ].compact.join('-')
17
+ end
13
18
 
14
- "#{destination}-#{Eventboss.configuration.sns_sqs_name_infix}-#{source}#{event}-#{env}"
19
+ def build(destination:, event_name:, env:, source_app: nil)
20
+ name = build_name(
21
+ destination: destination,
22
+ event_name: event_name,
23
+ env: env,
24
+ source_app: source_app
25
+ )
26
+ Queue.new(name)
27
+ end
15
28
  end
16
29
 
17
- def initialize(name, configuration = Eventboss.configuration)
18
- @client = configuration.sqs_client
30
+ def initialize(name)
31
+ @client = Eventboss.configuration.sqs_client
19
32
  @name = name
20
33
  end
21
34
 
@@ -23,6 +36,15 @@ module Eventboss
23
36
  @url ||= client.get_queue_url(queue_name: name).queue_url
24
37
  end
25
38
 
39
+ def arn
40
+ [
41
+ 'arn:aws:sqs',
42
+ Eventboss.configuration.eventboss_region,
43
+ Eventboss.configuration.eventboss_account_id,
44
+ name
45
+ ].join(':')
46
+ end
47
+
26
48
  def <=>(another_queue)
27
49
  name <=> another_queue&.name
28
50
  end
@@ -35,6 +57,10 @@ module Eventboss
35
57
  name.hash
36
58
  end
37
59
 
60
+ def to_s
61
+ "<Eventboss::Queue: #{name}>"
62
+ end
63
+
38
64
  private
39
65
 
40
66
  attr_reader :client
@@ -1,7 +1,7 @@
1
1
  module Eventboss
2
2
  class QueueListener
3
3
  class << self
4
- def select(include:, exclude:)
4
+ def select(include: nil, exclude: nil)
5
5
  listeners = list.values.map(&:name)
6
6
 
7
7
  listeners &= include if include
@@ -13,19 +13,15 @@ module Eventboss
13
13
  private
14
14
 
15
15
  def list
16
- Hash[Eventboss::Listener::ACTIVE_LISTENERS.map do |src_app_event, opts|
17
- [
18
- Eventboss::Queue.new(
19
- [
20
- opts[:destination_app] || Eventboss.configuration.eventboss_app_name,
21
- Eventboss.configuration.sns_sqs_name_infix,
22
- src_app_event,
23
- Eventboss.env
24
- ].join('-')
25
- ),
26
- opts[:listener]
27
- ]
28
- end]
16
+ Eventboss::Listener::ACTIVE_LISTENERS.each_with_object({}) do |(eventboss_options, listener), queue_listeners|
17
+ queue = Eventboss::Queue.build(
18
+ destination: eventboss_options[:destination_app] || Eventboss.configuration.eventboss_app_name,
19
+ source_app: eventboss_options[:source_app],
20
+ event_name: eventboss_options[:event_name],
21
+ env: Eventboss.env
22
+ )
23
+ queue_listeners[queue] = listener
24
+ end
29
25
  end
30
26
  end
31
27
  end
@@ -1,3 +1,5 @@
1
+ # frozen_string_literal: true
2
+
1
3
  module Eventboss
2
4
  class Runner
3
5
  extend Logging
@@ -17,8 +19,11 @@ module Eventboss
17
19
 
18
20
  self_read = setup_signals([:SIGTERM])
19
21
 
20
- logger.info('Active Listeners:')
21
- logger.info(queues.to_s)
22
+ logger.info('Active listeners:')
23
+ queues.each { |queue, listener| logger.info("#{queue}: #{listener}") }
24
+
25
+ Eventboss::DevelopmentMode.setup_infrastructure(queues) if config.development_mode?
26
+
22
27
  begin
23
28
  launcher.start
24
29
  handle_signals(self_read, launcher)
@@ -45,7 +50,7 @@ module Eventboss
45
50
  def handle_signals(self_read, launcher)
46
51
  while readable_io = IO.select([self_read])
47
52
  signal = readable_io.first[0].gets.strip
48
- logger.info('runner') { "Received #{ signal } signal, gracefully shutdowning..." }
53
+ logger.info('runner') { "Received #{signal} signal, gracefully shutting down..." }
49
54
 
50
55
  launcher.stop
51
56
  exit 0
@@ -1,5 +1,8 @@
1
+ # frozen_string_literal: true
2
+
1
3
  module Eventboss
2
- class NotConfigured < StandardError; end
4
+ class NotConfigured < StandardError;
5
+ end
3
6
 
4
7
  class SnsClient
5
8
  def initialize(configuration)
@@ -10,10 +13,31 @@ module Eventboss
10
13
  backend.publish(payload)
11
14
  end
12
15
 
16
+ def create_topic(name:)
17
+ backend.create_topic(name: name)
18
+ end
19
+
20
+ def create_subscription(topic_arn:, queue_arn:)
21
+ subscription = backend.subscribe(
22
+ topic_arn: topic_arn,
23
+ endpoint: queue_arn,
24
+ protocol: 'sqs'
25
+ )
26
+ set_raw_message_delivery(subscription)
27
+ end
28
+
13
29
  private
14
30
 
15
31
  attr_reader :configuration
16
32
 
33
+ def set_raw_message_delivery(subscription)
34
+ backend.set_subscription_attributes(
35
+ subscription_arn: subscription.subscription_arn,
36
+ attribute_name: 'RawMessageDelivery',
37
+ attribute_value: 'true'
38
+ )
39
+ end
40
+
17
41
  def backend
18
42
  if configured?
19
43
  options = {
@@ -26,11 +50,10 @@ module Eventboss
26
50
  if configuration.aws_sns_endpoint
27
51
  options[:endpoint] = configuration.aws_sns_endpoint
28
52
  end
29
- Aws::SNS::Client.new(
30
- options
31
- )
53
+
54
+ Aws::SNS::Client.new(options)
32
55
  elsif configuration.raise_on_missing_configuration
33
- raise NotConfigured, 'Eventboss is not configured'
56
+ raise NotConfigured, 'Eventboss is not configured.'
34
57
  else
35
58
  Mock.new
36
59
  end
@@ -0,0 +1,28 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Eventboss
4
+ class Topic
5
+ class << self
6
+ def build_arn(event_name:, source_app: nil)
7
+ [
8
+ 'arn:aws:sns',
9
+ Eventboss.configuration.eventboss_region,
10
+ Eventboss.configuration.eventboss_account_id,
11
+ build_name(
12
+ event_name: event_name,
13
+ source_app: source_app
14
+ )
15
+ ].join(':')
16
+ end
17
+
18
+ def build_name(event_name:, source_app: nil)
19
+ [
20
+ Eventboss.configuration.sns_sqs_name_infix,
21
+ source_app,
22
+ event_name,
23
+ Eventboss.env
24
+ ].compact.join('-')
25
+ end
26
+ end
27
+ end
28
+ end
@@ -6,14 +6,15 @@ module Eventboss
6
6
 
7
7
  attr_accessor :queue, :listener, :message
8
8
 
9
- def initialize(queue, listener, message)
9
+ def initialize(client, queue, listener, message)
10
+ @client = client
10
11
  @queue = queue
11
12
  @listener = listener
12
13
  @message = message
13
14
  @logger = logger
14
15
  end
15
16
 
16
- def run(client)
17
+ def run
17
18
  logger.debug(@message.message_id) { 'Started' }
18
19
  processor = @listener.new
19
20
  processor.receive(JSON.parse(@message.body))
@@ -21,21 +22,21 @@ module Eventboss
21
22
  rescue StandardError => exception
22
23
  handle_exception(exception, processor: processor, message_id: @message.message_id)
23
24
  else
24
- cleanup(client) unless processor.postponed_by
25
+ cleanup unless processor.postponed_by
25
26
  ensure
26
- change_message_visibility(client, processor.postponed_by) if processor.postponed_by
27
+ change_message_visibility(processor.postponed_by) if processor.postponed_by
27
28
  end
28
29
 
29
- def change_message_visibility(client, postponed_by)
30
- client.change_message_visibility(
30
+ def change_message_visibility(postponed_by)
31
+ @client.change_message_visibility(
31
32
  queue_url: @queue.url,
32
33
  receipt_handle: @message.receipt_handle,
33
34
  visibility_timeout: postponed_by
34
35
  )
35
36
  end
36
37
 
37
- def cleanup(client)
38
- client.delete_message(
38
+ def cleanup
39
+ @client.delete_message(
39
40
  queue_url: @queue.url, receipt_handle: @message.receipt_handle
40
41
  )
41
42
  logger.debug(@message.message_id) { 'Deleting' }
@@ -1,3 +1,3 @@
1
1
  module Eventboss
2
- VERSION = "1.3.1"
2
+ VERSION = "1.3.2"
3
3
  end
@@ -6,12 +6,12 @@ module Eventboss
6
6
 
7
7
  attr_reader :id
8
8
 
9
- def initialize(launcher, id, client, bus)
9
+ def initialize(launcher, id, bus, restart_on: [Exception])
10
10
  @id = "worker-#{id}"
11
11
  @launcher = launcher
12
- @client = client
13
12
  @bus = bus
14
13
  @thread = nil
14
+ @restart_on = restart_on
15
15
  end
16
16
 
17
17
  def start
@@ -20,12 +20,12 @@ module Eventboss
20
20
 
21
21
  def run
22
22
  while (work = @bus.pop)
23
- work.run(@client)
23
+ work.run
24
24
  end
25
25
  @launcher.worker_stopped(self)
26
26
  rescue Eventboss::Shutdown
27
27
  @launcher.worker_stopped(self)
28
- rescue Exception => exception
28
+ rescue *@restart_on => exception
29
29
  handle_exception(exception, worker_id: id)
30
30
  # Restart the worker in case of hard exception
31
31
  # Message won't be delete from SQS and will be visible later
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: eventboss
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.3.1
4
+ version: 1.3.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - AirHelp
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2019-11-05 00:00:00.000000000 Z
11
+ date: 2020-02-03 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: aws-sdk-sqs
@@ -108,9 +108,9 @@ executables:
108
108
  extensions: []
109
109
  extra_rdoc_files: []
110
110
  files:
111
- - ".github/workflows/ruby.yml"
112
111
  - ".gitignore"
113
112
  - ".rspec"
113
+ - ".travis.yml"
114
114
  - CHANGELOG.md
115
115
  - Gemfile
116
116
  - Gemfile.lock
@@ -123,6 +123,7 @@ files:
123
123
  - lib/eventboss.rb
124
124
  - lib/eventboss/cli.rb
125
125
  - lib/eventboss/configuration.rb
126
+ - lib/eventboss/development_mode.rb
126
127
  - lib/eventboss/error_handlers/airbrake.rb
127
128
  - lib/eventboss/error_handlers/db_connection_drop_handler.rb
128
129
  - lib/eventboss/error_handlers/logger.rb
@@ -142,6 +143,7 @@ files:
142
143
  - lib/eventboss/scripts.rb
143
144
  - lib/eventboss/sender.rb
144
145
  - lib/eventboss/sns_client.rb
146
+ - lib/eventboss/topic.rb
145
147
  - lib/eventboss/unit_of_work.rb
146
148
  - lib/eventboss/version.rb
147
149
  - lib/eventboss/worker.rb
@@ -1,20 +0,0 @@
1
- name: Ruby
2
-
3
- on: [push]
4
-
5
- jobs:
6
- build:
7
-
8
- runs-on: ubuntu-latest
9
-
10
- steps:
11
- - uses: actions/checkout@v1
12
- - name: Set up Ruby 2.6
13
- uses: actions/setup-ruby@v1
14
- with:
15
- ruby-version: 2.6.x
16
- - name: Build and test with Rake
17
- run: |
18
- gem install bundler
19
- bundle install --jobs 4 --retry 3
20
- bundle exec rake