mimi-messaging-sqs_sns 0.1.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: 799168326dcf4504b6ab6c62eb6da195015e615b
4
+ data.tar.gz: 67c0507d5937e2160d87706706ac8d73dcb922bc
5
+ SHA512:
6
+ metadata.gz: 4db933f3182eacfa87a3e37ca56fb517d38f59acfa023de8750849b3f6d497fdac8e10e952e016bcc177ebae08139b5de52116231401e1908492f1ceeb5c104b
7
+ data.tar.gz: 0ba15e0811746cfad55dfef7bd4e4136b3a217d1b5aa8f835cc595d236fb68c321fd68f78d0afd597bdd41f40d141e97fc6de160e58b06e6557645faffde7bba
data/.gitignore ADDED
@@ -0,0 +1,12 @@
1
+ /.bundle/
2
+ /.yardoc
3
+ /_yardoc/
4
+ /coverage/
5
+ /doc/
6
+ /pkg/
7
+ /spec/reports/
8
+ /tmp/
9
+ /Gemfile.lock
10
+
11
+ # rspec failure tracking
12
+ .rspec_status
data/.rspec ADDED
@@ -0,0 +1,3 @@
1
+ --format documentation
2
+ --color
3
+ --require spec_helper
data/.travis.yml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ sudo: false
3
+ language: ruby
4
+ cache: bundler
5
+ rvm:
6
+ - 2.4.6
7
+ before_install: gem install bundler -v 2.0.2
@@ -0,0 +1,74 @@
1
+ # Contributor Covenant Code of Conduct
2
+
3
+ ## Our Pledge
4
+
5
+ In the interest of fostering an open and welcoming environment, we as
6
+ contributors and maintainers pledge to making participation in our project and
7
+ our community a harassment-free experience for everyone, regardless of age, body
8
+ size, disability, ethnicity, gender identity and expression, level of experience,
9
+ nationality, personal appearance, race, religion, or sexual identity and
10
+ orientation.
11
+
12
+ ## Our Standards
13
+
14
+ Examples of behavior that contributes to creating a positive environment
15
+ include:
16
+
17
+ * Using welcoming and inclusive language
18
+ * Being respectful of differing viewpoints and experiences
19
+ * Gracefully accepting constructive criticism
20
+ * Focusing on what is best for the community
21
+ * Showing empathy towards other community members
22
+
23
+ Examples of unacceptable behavior by participants include:
24
+
25
+ * The use of sexualized language or imagery and unwelcome sexual attention or
26
+ advances
27
+ * Trolling, insulting/derogatory comments, and personal or political attacks
28
+ * Public or private harassment
29
+ * Publishing others' private information, such as a physical or electronic
30
+ address, without explicit permission
31
+ * Other conduct which could reasonably be considered inappropriate in a
32
+ professional setting
33
+
34
+ ## Our Responsibilities
35
+
36
+ Project maintainers are responsible for clarifying the standards of acceptable
37
+ behavior and are expected to take appropriate and fair corrective action in
38
+ response to any instances of unacceptable behavior.
39
+
40
+ Project maintainers have the right and responsibility to remove, edit, or
41
+ reject comments, commits, code, wiki edits, issues, and other contributions
42
+ that are not aligned to this Code of Conduct, or to ban temporarily or
43
+ permanently any contributor for other behaviors that they deem inappropriate,
44
+ threatening, offensive, or harmful.
45
+
46
+ ## Scope
47
+
48
+ This Code of Conduct applies both within project spaces and in public spaces
49
+ when an individual is representing the project or its community. Examples of
50
+ representing a project or community include using an official project e-mail
51
+ address, posting via an official social media account, or acting as an appointed
52
+ representative at an online or offline event. Representation of a project may be
53
+ further defined and clarified by project maintainers.
54
+
55
+ ## Enforcement
56
+
57
+ Instances of abusive, harassing, or otherwise unacceptable behavior may be
58
+ reported by contacting the project team at alex@kukushk.in. All
59
+ complaints will be reviewed and investigated and will result in a response that
60
+ is deemed necessary and appropriate to the circumstances. The project team is
61
+ obligated to maintain confidentiality with regard to the reporter of an incident.
62
+ Further details of specific enforcement policies may be posted separately.
63
+
64
+ Project maintainers who do not follow or enforce the Code of Conduct in good
65
+ faith may face temporary or permanent repercussions as determined by other
66
+ members of the project's leadership.
67
+
68
+ ## Attribution
69
+
70
+ This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
71
+ available at [http://contributor-covenant.org/version/1/4][version]
72
+
73
+ [homepage]: http://contributor-covenant.org
74
+ [version]: http://contributor-covenant.org/version/1/4/
data/Gemfile ADDED
@@ -0,0 +1,4 @@
1
+ source "https://rubygems.org"
2
+
3
+ # Specify your gem's dependencies in mimi-messaging-sqs-sns.gemspec
4
+ gemspec
data/LICENSE.txt ADDED
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2019 Alex Kukushkin
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in
13
+ all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21
+ THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,56 @@
1
+ # mimi-messaging-sqs_sns
2
+
3
+ AWS SQS/SNS adapter for [mimi-messaging](https://github.com/kukushkin/mimi-messaging).
4
+
5
+ ## Installation
6
+
7
+ Add this line to your application's Gemfile:
8
+
9
+ ```ruby
10
+ gem 'mimi-messaging-sqs_sns'
11
+ ```
12
+
13
+ And then execute:
14
+
15
+ $ bundle
16
+
17
+ Or install it yourself as:
18
+
19
+ $ gem install mimi-messaging-sqs_sns
20
+
21
+ ## Usage
22
+
23
+ ```ruby
24
+ require "mimi/messaging"
25
+ require "mimi/messaging/sqs_sns"
26
+
27
+ Mimi::Messaging.configure(
28
+ mq_adapter: "sqs_sns",
29
+
30
+ # if nil, AWS SDK will guess values from environment
31
+ mq_aws_region: nil,
32
+ mq_aws_access_key_id: nil,
33
+ mq_aws_secret_access_key: nil,
34
+ mq_aws_sqs_endpoint: nil,
35
+
36
+ mq_aws_sqs_read_timeout: 10, # seconds
37
+ mq_namespace: nil,
38
+ mq_default_query_timeout: 15, # seconds,
39
+ mq_reply_queue_prefix: "reply."
40
+ )
41
+
42
+ Mimi::Messaging.start
43
+ ```
44
+
45
+
46
+ ## Contributing
47
+
48
+ Bug reports and pull requests are welcome on GitHub at https://github.com/kukushkin/mimi-messaging-sqs_sns. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
49
+
50
+ ## License
51
+
52
+ The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
53
+
54
+ ## Code of Conduct
55
+
56
+ Everyone interacting in the mimi-messaging-sqs_sns project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/kukushkin/mimi-messaging-sqs_sns/blob/master/CODE_OF_CONDUCT.md).
data/Rakefile ADDED
@@ -0,0 +1,6 @@
1
+ require "bundler/gem_tasks"
2
+ require "rspec/core/rake_task"
3
+
4
+ RSpec::Core::RakeTask.new(:spec)
5
+
6
+ task :default => :spec
data/bin/console ADDED
@@ -0,0 +1,10 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ require "bundler/setup"
4
+ require "mimi/messaging/sqs_sns"
5
+
6
+ # You can add fixtures and/or initialization code here to make experimenting
7
+ # with your gem easier. You can also use a different console, if you like.
8
+
9
+ require "pry"
10
+ Pry.start
data/bin/setup ADDED
@@ -0,0 +1,8 @@
1
+ #!/usr/bin/env bash
2
+ set -euo pipefail
3
+ IFS=$'\n\t'
4
+ set -vx
5
+
6
+ bundle install
7
+
8
+ # Do any other automated setup that you need to do here
data/examples/query.rb ADDED
@@ -0,0 +1,36 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "mimi/messaging/sqs_sns"
4
+
5
+ COUNT = 10
6
+ AWS_REGION = "eu-west-1"
7
+ AWS_SQS_ENDPOINT_URL = "http://localstack:4576"
8
+ AWS_ACCESS_KEY_ID = "foo"
9
+ AWS_SECRET_ACCESS_KEY = "bar"
10
+
11
+ logger = Logger.new(STDOUT)
12
+ logger.level = Logger::INFO
13
+ Mimi::Messaging.use(logger: logger, serializer: Mimi::Messaging::JsonSerializer)
14
+ Mimi::Messaging.configure(
15
+ mq_adapter: "sqs_sns",
16
+ mq_aws_access_key_id: AWS_ACCESS_KEY_ID,
17
+ mq_aws_secret_access_key: AWS_SECRET_ACCESS_KEY,
18
+ mq_aws_region: AWS_REGION,
19
+ mq_aws_sqs_endpoint: AWS_SQS_ENDPOINT_URL
20
+ )
21
+ adapter = Mimi::Messaging.adapter
22
+
23
+ adapter.start
24
+
25
+ t_start = Time.now
26
+ COUNT.times do |i|
27
+ t = Time.now
28
+ result = adapter.query("test/hello", i: i) # rand(100))
29
+ puts "result: #{result.to_h}, t: %.3fs" % (Time.now - t)
30
+ sleep 1
31
+ end
32
+
33
+ t_elapsed = Time.now - t_start
34
+ puts "t_elapsed: %.3fs" % t_elapsed
35
+ adapter.stop
36
+ puts "t.avg: %.3fs" % (t_elapsed / COUNT)
@@ -0,0 +1,48 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "mimi/messaging/sqs_sns"
4
+
5
+ AWS_REGION = "eu-west-1"
6
+ AWS_SQS_ENDPOINT_URL = "http://localstack:4576"
7
+ AWS_ACCESS_KEY_ID = "foo"
8
+ AWS_SECRET_ACCESS_KEY = "bar"
9
+
10
+ class Processor
11
+ def self.call_command(method_name, message, opts)
12
+ puts "COMMAND: #{method_name}, #{message}, headers: #{opts[:headers]}"
13
+ end
14
+
15
+ def self.call_query(method_name, message, opts)
16
+ # puts "QUERY: #{method_name}, #{message}, headers: #{opts[:headers]}"
17
+ puts "QUERY: #{method_name}, headers: #{opts[:headers]}"
18
+ {}
19
+ end
20
+ end # class Processor
21
+
22
+
23
+ logger = Logger.new(STDOUT)
24
+ logger.level = Logger::INFO
25
+ Mimi::Messaging.use(logger: logger, serializer: Mimi::Messaging::JsonSerializer)
26
+ Mimi::Messaging.configure(
27
+ mq_adapter: "sqs_sns",
28
+ mq_aws_access_key_id: AWS_ACCESS_KEY_ID,
29
+ mq_aws_secret_access_key: AWS_SECRET_ACCESS_KEY,
30
+ mq_aws_region: AWS_REGION,
31
+ mq_aws_sqs_endpoint: AWS_SQS_ENDPOINT_URL
32
+ )
33
+ adapter = Mimi::Messaging.adapter
34
+ queue_name = "test"
35
+ adapter.start
36
+ puts "Registering processor on '#{queue_name}'"
37
+ adapter.start_request_processor(queue_name, Processor)
38
+
39
+
40
+ begin
41
+ loop do
42
+ sleep 1
43
+ end
44
+ ensure
45
+ puts "Stopping adapter"
46
+ adapter.stop
47
+ end
48
+
@@ -0,0 +1,288 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "mimi/messaging"
4
+ require "aws-sdk-sqs"
5
+ require "timeout"
6
+ require "securerandom"
7
+
8
+ module Mimi
9
+ module Messaging
10
+ module SQS_SNS
11
+ #
12
+ # AWS SQS/SNS adapter class
13
+ #
14
+ # An adapter implementation must implement the following methods:
15
+ # * #start()
16
+ # * #stop()
17
+ # * #command(target, message, opts)
18
+ # * #query(target, message, opts)
19
+ # * #event(target, message, opts)
20
+ # * #start_request_processor(queue_name, processor, opts)
21
+ # * #start_event_processor(topic_name, processor, opts)
22
+ # * #start_event_processor_with_queue(topic_name, queue_name, processor, opts)
23
+ # * #stop_all_processors
24
+ #
25
+ class Adapter < Mimi::Messaging::Adapters::Base
26
+ attr_reader :options, :sqs_client
27
+
28
+ register_adapter_name "sqs_sns"
29
+
30
+ DEFAULT_OPTIONS = {
31
+ mq_namespace: nil,
32
+ mq_default_query_timeout: 15, # seconds,
33
+ mq_reply_queue_prefix: "reply.",
34
+
35
+ # if nil, AWS SDK will guess values from environment
36
+ mq_aws_region: nil,
37
+ mq_aws_access_key_id: nil,
38
+ mq_aws_secret_access_key: nil,
39
+ mq_aws_sqs_endpoint: nil,
40
+
41
+ mq_aws_sqs_read_timeout: 10, # seconds
42
+ }.freeze
43
+
44
+ # Initializes SQS/SNS adapter
45
+ #
46
+ # @param options [Hash]
47
+ # @option options [String] :mq_adapter
48
+ # @option options [String,nil] :mq_aws_region
49
+ # @option options [String,nil] :mq_aws_access_key_id
50
+ # @option options [String,nil] :mq_aws_secret_access_key
51
+ # @option options [String,nil] :mq_aws_sqs_endpoint
52
+ # @option options [String,nil] :mq_namespace
53
+ # @option options [String,nil] :mq_reply_queue_prefix
54
+ # @option options [Integer,nil] :mq_default_query_timeout
55
+ #
56
+ def initialize(options)
57
+ @options = DEFAULT_OPTIONS.merge(options).dup
58
+ end
59
+
60
+ def start
61
+ @sqs_client = Aws::SQS::Client.new(sqs_client_config)
62
+ end
63
+
64
+ def stop
65
+ @consumers&.each(&:stop)
66
+ @consumers = nil
67
+ @reply_listener&.stop
68
+ @reply_listener = nil
69
+ @sqs_client = nil
70
+ end
71
+
72
+ # Sends the command to the given target
73
+ #
74
+ # Example:
75
+ # Mimi::Messaging.command("users/create", name: "John Smith")
76
+ #
77
+ # @param target [String] "<queue>/<method>"
78
+ # @param message [Hash]
79
+ # @param opts [Hash] additional adapter-specific options
80
+ #
81
+ # @return nil
82
+ #
83
+ def command(target, message, _opts = {})
84
+ queue_name, method_name = target.split("/")
85
+ message_payload = serialize(message)
86
+ queue_url = find_queue(queue_name)
87
+ deliver_message(message_payload, queue_url, __method: method_name)
88
+ end
89
+
90
+ # Executes the query to the given target and returns response
91
+ #
92
+ # @param target [String] "<queue>/<method>"
93
+ # @param message [Hash]
94
+ # @param opts [Hash] additional options, e.g. :timeout
95
+ #
96
+ # @return [Hash]
97
+ # @raise [SomeError,TimeoutError]
98
+ #
99
+ def query(target, message, opts = {})
100
+ queue_name, method_name = target.split("/")
101
+ message_payload = serialize(message)
102
+ queue_url = find_queue(queue_name)
103
+ request_id = SecureRandom.hex(8)
104
+ reply_queue = reply_listener.register_request_id(request_id)
105
+
106
+ deliver_message(
107
+ message_payload,
108
+ queue_url,
109
+ __method: method_name,
110
+ __reply_queue_url: reply_listener.reply_queue_url,
111
+ __request_id: request_id
112
+ )
113
+ timeout = opts[:timeout] || options[:mq_default_query_timeout]
114
+ response = nil
115
+ Timeout::timeout(timeout) do
116
+ response = reply_queue.pop
117
+ end
118
+ deserialize(response.body)
119
+ end
120
+
121
+ def event(target, message, _opts = {})
122
+ raise "Not implemented"
123
+ end
124
+
125
+ # Starts a request (command/query) processor.
126
+ #
127
+ # Processor must respond to #call_command() AND #call_query()
128
+ # which accepts 3 arguments: (method, message, opts).
129
+ #
130
+ # If the processor raises an error, the message will be NACK-ed and accepted again
131
+ # at a later time.
132
+ #
133
+ # @param queue_name [String] "<queue>"
134
+ # @param processor [#call_command(),#call_query()]
135
+ # @param opts [Hash] additional adapter-specific options
136
+ #
137
+ def start_request_processor(queue_name, processor, opts = {})
138
+ super
139
+ @consumers ||= []
140
+ opts = opts.dup
141
+ queue_url = find_or_create_queue(queue_name)
142
+ @consumers << Consumer.new(self, queue_url) do |m|
143
+ message = deserialize(m.body)
144
+ headers = deserialize_headers(m)
145
+ method_name = headers[:__method]
146
+ reply_to = headers[:__reply_queue_url]
147
+ if reply_to
148
+ response = processor.call_query(method_name, message, headers: headers)
149
+ deliver_message(serialize(response), reply_to, __request_id: headers[:__request_id])
150
+ else
151
+ processor.call_command(method_name, message, headers: headers)
152
+ end
153
+ end
154
+ end
155
+
156
+ def start_event_processor(topic_name, processor, opts = {})
157
+ raise "Not implemented"
158
+ end
159
+
160
+ def start_event_processor_with_queue(topic_name, queue_name, processor, opts = {})
161
+ raise "Not implemented"
162
+ end
163
+
164
+ # Creates a new queue
165
+ #
166
+ # @param queue_name [String] name of the topic to be created
167
+ # @return [String] a new queue URL
168
+ #
169
+ def create_queue(queue_name)
170
+ fqn = full_queue_name(queue_name)
171
+ Mimi::Messaging.log "Creating a queue: #{fqn}"
172
+ result = sqs_client.create_queue(queue_name: fqn)
173
+ result.queue_url
174
+ rescue StandardError => e
175
+ raise Mimi::Messaging::ConnectionError, "Failed to create queue '#{queue_name}': #{e}"
176
+ end
177
+
178
+ private
179
+
180
+ # Returns configuration parameters for AWS SQS client
181
+ #
182
+ # @return [Hash]
183
+ #
184
+ def sqs_client_config
185
+ params = {
186
+ region: options[:mq_aws_region],
187
+ endpoint: options[:mq_aws_sqs_endpoint],
188
+ access_key_id: options[:mq_aws_access_key_id],
189
+ secret_access_key: options[:mq_aws_secret_access_key]
190
+ }
191
+ params.compact
192
+ end
193
+
194
+ # Delivers a message to a queue with given URL.
195
+ #
196
+ # @param message [String]
197
+ # @param queue_url [String]
198
+ # @param headers [Hash<Symbol,String>]
199
+ #
200
+ def deliver_message(message, queue_url, headers = {})
201
+ raise ArgumentError, "Non-empty queue URL is expected" unless queue_url
202
+ Mimi::Messaging.log "Delivering message to: #{queue_url}"
203
+ sqs_client.send_message(
204
+ queue_url: queue_url,
205
+ message_body: message,
206
+ message_attributes: headers.map do |k, v|
207
+ [k.to_s, { data_type: "String", string_value: v.to_s }]
208
+ end.to_h
209
+ )
210
+ rescue StandardError => e
211
+ raise Mimi::Messaging::ConnectionError, "Failed to deliver message to '#{queue_url}': #{e}"
212
+ end
213
+
214
+ # Returns URL of a queue with a given name.
215
+ #
216
+ # If the queue with given name does not exist, returns nil
217
+ #
218
+ # @param queue_name [String]
219
+ # @return [String,nil] queue URL
220
+ #
221
+ def queue_registry(queue_name)
222
+ fqn = full_queue_name(queue_name)
223
+ @queue_registry ||= {}
224
+ @queue_registry[fqn] ||= begin
225
+ result = sqs_client.get_queue_url(queue_name: fqn)
226
+ result.queue_url
227
+ end
228
+ rescue Aws::SQS::Errors::NonExistentQueue
229
+ nil
230
+ end
231
+
232
+ # Converts a queue name to a fully qualified queue name
233
+ #
234
+ # @param queue_name [String]
235
+ # @return [String]
236
+ #
237
+ def full_queue_name(queue_name)
238
+ "#{options[:mq_namespace]}#{queue_name}"
239
+ end
240
+
241
+ # Finds a queue URL for a queue with a given name,
242
+ # or raises an error if the queue is not found.
243
+ #
244
+ # @param queue_name [String]
245
+ # @return [String] a queue URL
246
+ #
247
+ def find_queue(queue_name)
248
+ queue_registry(queue_name) || (
249
+ raise Mimi::Messaging::ConnectionError,
250
+ "Failed to find a queue with given name: '#{queue_name}'"
251
+ )
252
+ end
253
+
254
+ # Finds a queue URL for a queue with given name.
255
+ #
256
+ # If an existing queue with this name is not found,
257
+ # the method will try to create a new one.
258
+ #
259
+ # @param queue_name [String]
260
+ # @return [String] a queue URL
261
+ #
262
+ def find_or_create_queue(queue_name)
263
+ queue_registry(queue_name) || create_queue(queue_name)
264
+ end
265
+
266
+ # Returns the configured reply listener for this process
267
+ #
268
+ # @return [ReplyListener]
269
+ #
270
+ def reply_listener
271
+ @reply_listener ||= begin
272
+ reply_queue_name = options[:mq_reply_queue_prefix] + SecureRandom.hex(8)
273
+ Mimi::Messaging::SQS_SNS::ReplyListener.new(self, reply_queue_name)
274
+ end
275
+ end
276
+
277
+ # Deserializes headers from the message
278
+ #
279
+ # @param message
280
+ # @return [Hash<Symbol,String>] symbolized keys, string values
281
+ #
282
+ def deserialize_headers(message)
283
+ message.message_attributes.to_h.map { |k, v| [k.to_sym, v.string_value] }.to_h
284
+ end
285
+ end # class Adapter
286
+ end # module SQS_SNS
287
+ end # module Messaging
288
+ end # module Mimi
@@ -0,0 +1,53 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Mimi
4
+ module Messaging
5
+ module SQS_SNS
6
+ #
7
+ # Message processor for SQS queues
8
+ #
9
+ class Consumer
10
+ def initialize(adapter, queue_url, &block)
11
+ @block = block
12
+ @stop_requested = false
13
+ Mimi::Messaging.log "Starting consumer for: #{queue_url}"
14
+ @consumer_thread = Thread.new do
15
+ while not @stop_requested do
16
+ message = read_message(adapter, queue_url)
17
+ next unless message
18
+ Mimi::Messaging.log "Read message from: #{queue_url}"
19
+ block.call(message)
20
+ ack_message(adapter, queue_url, message)
21
+ end
22
+ Mimi::Messaging.log "Stopping consumer for: #{queue_url}"
23
+ end
24
+ end
25
+
26
+ def stop
27
+ @stop_requested = true
28
+ @consumer_thread.join
29
+ end
30
+
31
+ private
32
+
33
+ def read_message(adapter, queue_url)
34
+ result = adapter.sqs_client.receive_message(
35
+ queue_url: queue_url,
36
+ max_number_of_messages: 1,
37
+ wait_time_seconds: adapter.options[:mq_aws_sqs_read_timeout],
38
+ message_attribute_names: ["All"]
39
+ )
40
+ return nil if result.messages.count == 0
41
+ return result.messages.first if result.messages.count == 1
42
+ raise Mimi::Messaging::ConnectionError, "Unexpected number of messages read"
43
+ end
44
+
45
+ def ack_message(adapter, queue_url, msg)
46
+ adapter.sqs_client.delete_message(
47
+ queue_url: queue_url, receipt_handle: msg.receipt_handle
48
+ )
49
+ end
50
+ end # class Consumer
51
+ end # module SQS_SNS
52
+ end # module Messaging
53
+ end # module Mimi
@@ -0,0 +1,79 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Mimi
4
+ module Messaging
5
+ module SQS_SNS
6
+ #
7
+ # ReplyListener listens on a particular SQS queue for replies
8
+ # and passes them to registered Queues (see Ruby ::Queue class).
9
+ #
10
+ class ReplyListener
11
+ attr_reader :reply_queue_name, :reply_queue_url
12
+
13
+ def initialize(adapter, reply_queue_name)
14
+ @mutex = Mutex.new
15
+ @queues = {}
16
+ @reply_queue_name = reply_queue_name
17
+ @adapter = adapter
18
+ @reply_queue_url = adapter.create_queue(reply_queue_name)
19
+ @consumer = Consumer.new(adapter, reply_queue_url) do |message|
20
+ dispatch_message(message)
21
+ end
22
+ end
23
+
24
+ def stop
25
+ begin
26
+ @consumer.stop
27
+ rescue StandardError => e
28
+ raise Mimi::Messaging::Error, "Failed to stop consumer: #{e}"
29
+ end
30
+ # TODO: adapter.sqs_client.delete_queue(reply_queue_url)
31
+ end
32
+
33
+ # Register a new request_id to listen for.
34
+ #
35
+ # Whenever the message with given request_id will be received,
36
+ # it will be dispatched to a returned Queue.
37
+ #
38
+ # @param request_id [String]
39
+ # @return [Queue] a new Queue object registered for this request_id
40
+ #
41
+ def register_request_id(request_id)
42
+ queue = Queue.new
43
+ @mutex.synchronize do
44
+ queue = @queues[request_id] ||= queue
45
+ end
46
+ queue
47
+ end
48
+
49
+ private
50
+
51
+ # Deserializes headers from the message
52
+ #
53
+ # @param message
54
+ # @return [Hash<Symbol,String>] symbolized keys, string values
55
+ #
56
+ def deserialize_headers(message)
57
+ message.message_attributes.to_h.map { |k, v| [k.to_sym, v.string_value] }.to_h
58
+ end
59
+
60
+ # Dispatch message received on a reply queue
61
+ #
62
+ # @param message [] an AWS SQS message
63
+ #
64
+ def dispatch_message(message)
65
+ queue = nil
66
+ @mutex.synchronize do
67
+ headers = deserialize_headers(message)
68
+ request_id = headers[:__request_id]
69
+ queue = @queues.delete(request_id)
70
+ end
71
+ queue&.push(message)
72
+ rescue StandardError => e
73
+ Mimi::Messaging.log "reply listener failed to process reply: #{e}"
74
+ # TODO: propagate exception to main thread?
75
+ end
76
+ end # class ReplyListener
77
+ end # module SQS_SNS
78
+ end # module Messaging
79
+ end # module Mimi
@@ -0,0 +1,9 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Mimi
4
+ module Messaging
5
+ module SQS_SNS
6
+ VERSION = "0.1.4"
7
+ end
8
+ end
9
+ end
@@ -0,0 +1,15 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "mimi/messaging/sqs_sns/version"
4
+
5
+ module Mimi
6
+ module Messaging
7
+ module SQS_SNS
8
+ #
9
+ end # module SQS_SNS
10
+ end # module Messaging
11
+ end # module Mimi
12
+
13
+ require_relative "sqs_sns/adapter"
14
+ require_relative "sqs_sns/consumer"
15
+ require_relative "sqs_sns/reply_listener"
@@ -0,0 +1,44 @@
1
+ # frozen_string_literal: true
2
+
3
+ lib = File.expand_path("lib", __dir__)
4
+ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
5
+ require "mimi/messaging/sqs_sns/version"
6
+
7
+ Gem::Specification.new do |spec|
8
+ spec.name = "mimi-messaging-sqs_sns"
9
+ spec.version = Mimi::Messaging::SQS_SNS::VERSION
10
+ spec.authors = ["Alex Kukushkin"]
11
+ spec.email = ["alex@kukushk.in"]
12
+
13
+ spec.summary = "AWS SQS/SNS adapter for mimi-messaging"
14
+ spec.description = "AWS SQS/SNS adapter for mimi-messaging"
15
+ spec.homepage = "https://github.com/kukushkin/mimi-messaging-sqs_sns"
16
+ spec.license = "MIT"
17
+
18
+ if spec.respond_to?(:metadata)
19
+ spec.metadata["allowed_push_host"] = "https://rubygems.org/"
20
+ else
21
+ raise "RubyGems 2.0 or newer is required to protect against public gem pushes."
22
+ end
23
+
24
+ spec.metadata["homepage_uri"] = spec.homepage
25
+ spec.metadata["source_code_uri"] = "https://github.com/kukushkin/mimi-messaging-sqs_sns"
26
+
27
+ # Specify which files should be added to the gem when it is released.
28
+ # The `git ls-files -z` loads the files in the RubyGem that have been added into git.
29
+ spec.files = Dir.chdir(File.expand_path(__dir__)) do
30
+ `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
31
+ end
32
+ spec.bindir = "exe"
33
+ spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
34
+ spec.require_paths = ["lib"]
35
+
36
+ spec.add_dependency "mimi-messaging", "~> 1.0"
37
+ spec.add_dependency "aws-sdk-sqs", "~> 1.22"
38
+ spec.add_dependency "aws-sdk-sns", "~> 1.19"
39
+
40
+ spec.add_development_dependency "bundler", "~> 2.0"
41
+ spec.add_development_dependency "pry", "~> 0.12"
42
+ spec.add_development_dependency "rake", "~> 10.0"
43
+ spec.add_development_dependency "rspec", "~> 3.0"
44
+ end
metadata ADDED
@@ -0,0 +1,163 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: mimi-messaging-sqs_sns
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.4
5
+ platform: ruby
6
+ authors:
7
+ - Alex Kukushkin
8
+ autorequire:
9
+ bindir: exe
10
+ cert_chain: []
11
+ date: 2019-10-06 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: mimi-messaging
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - "~>"
18
+ - !ruby/object:Gem::Version
19
+ version: '1.0'
20
+ type: :runtime
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - "~>"
25
+ - !ruby/object:Gem::Version
26
+ version: '1.0'
27
+ - !ruby/object:Gem::Dependency
28
+ name: aws-sdk-sqs
29
+ requirement: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - "~>"
32
+ - !ruby/object:Gem::Version
33
+ version: '1.22'
34
+ type: :runtime
35
+ prerelease: false
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - "~>"
39
+ - !ruby/object:Gem::Version
40
+ version: '1.22'
41
+ - !ruby/object:Gem::Dependency
42
+ name: aws-sdk-sns
43
+ requirement: !ruby/object:Gem::Requirement
44
+ requirements:
45
+ - - "~>"
46
+ - !ruby/object:Gem::Version
47
+ version: '1.19'
48
+ type: :runtime
49
+ prerelease: false
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - "~>"
53
+ - !ruby/object:Gem::Version
54
+ version: '1.19'
55
+ - !ruby/object:Gem::Dependency
56
+ name: bundler
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - "~>"
60
+ - !ruby/object:Gem::Version
61
+ version: '2.0'
62
+ type: :development
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - "~>"
67
+ - !ruby/object:Gem::Version
68
+ version: '2.0'
69
+ - !ruby/object:Gem::Dependency
70
+ name: pry
71
+ requirement: !ruby/object:Gem::Requirement
72
+ requirements:
73
+ - - "~>"
74
+ - !ruby/object:Gem::Version
75
+ version: '0.12'
76
+ type: :development
77
+ prerelease: false
78
+ version_requirements: !ruby/object:Gem::Requirement
79
+ requirements:
80
+ - - "~>"
81
+ - !ruby/object:Gem::Version
82
+ version: '0.12'
83
+ - !ruby/object:Gem::Dependency
84
+ name: rake
85
+ requirement: !ruby/object:Gem::Requirement
86
+ requirements:
87
+ - - "~>"
88
+ - !ruby/object:Gem::Version
89
+ version: '10.0'
90
+ type: :development
91
+ prerelease: false
92
+ version_requirements: !ruby/object:Gem::Requirement
93
+ requirements:
94
+ - - "~>"
95
+ - !ruby/object:Gem::Version
96
+ version: '10.0'
97
+ - !ruby/object:Gem::Dependency
98
+ name: rspec
99
+ requirement: !ruby/object:Gem::Requirement
100
+ requirements:
101
+ - - "~>"
102
+ - !ruby/object:Gem::Version
103
+ version: '3.0'
104
+ type: :development
105
+ prerelease: false
106
+ version_requirements: !ruby/object:Gem::Requirement
107
+ requirements:
108
+ - - "~>"
109
+ - !ruby/object:Gem::Version
110
+ version: '3.0'
111
+ description: AWS SQS/SNS adapter for mimi-messaging
112
+ email:
113
+ - alex@kukushk.in
114
+ executables: []
115
+ extensions: []
116
+ extra_rdoc_files: []
117
+ files:
118
+ - ".gitignore"
119
+ - ".rspec"
120
+ - ".travis.yml"
121
+ - CODE_OF_CONDUCT.md
122
+ - Gemfile
123
+ - LICENSE.txt
124
+ - README.md
125
+ - Rakefile
126
+ - bin/console
127
+ - bin/setup
128
+ - examples/query.rb
129
+ - examples/responder.rb
130
+ - lib/mimi/messaging/sqs_sns.rb
131
+ - lib/mimi/messaging/sqs_sns/adapter.rb
132
+ - lib/mimi/messaging/sqs_sns/consumer.rb
133
+ - lib/mimi/messaging/sqs_sns/reply_listener.rb
134
+ - lib/mimi/messaging/sqs_sns/version.rb
135
+ - mimi-messaging-sqs_sns.gemspec
136
+ homepage: https://github.com/kukushkin/mimi-messaging-sqs_sns
137
+ licenses:
138
+ - MIT
139
+ metadata:
140
+ allowed_push_host: https://rubygems.org/
141
+ homepage_uri: https://github.com/kukushkin/mimi-messaging-sqs_sns
142
+ source_code_uri: https://github.com/kukushkin/mimi-messaging-sqs_sns
143
+ post_install_message:
144
+ rdoc_options: []
145
+ require_paths:
146
+ - lib
147
+ required_ruby_version: !ruby/object:Gem::Requirement
148
+ requirements:
149
+ - - ">="
150
+ - !ruby/object:Gem::Version
151
+ version: '0'
152
+ required_rubygems_version: !ruby/object:Gem::Requirement
153
+ requirements:
154
+ - - ">="
155
+ - !ruby/object:Gem::Version
156
+ version: '0'
157
+ requirements: []
158
+ rubyforge_project:
159
+ rubygems_version: 2.6.14.4
160
+ signing_key:
161
+ specification_version: 4
162
+ summary: AWS SQS/SNS adapter for mimi-messaging
163
+ test_files: []