circuitry 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.gitignore +12 -0
- data/.rspec +2 -0
- data/Gemfile +4 -0
- data/LICENSE.txt +21 -0
- data/README.md +183 -0
- data/Rakefile +6 -0
- data/bin/console +14 -0
- data/bin/setup +7 -0
- data/circle.yml +3 -0
- data/circuitry.gemspec +31 -0
- data/lib/circuitry.rb +24 -0
- data/lib/circuitry/concerns/async.rb +18 -0
- data/lib/circuitry/configuration.rb +22 -0
- data/lib/circuitry/message.rb +32 -0
- data/lib/circuitry/publisher.rb +60 -0
- data/lib/circuitry/services/sns.rb +11 -0
- data/lib/circuitry/services/sqs.rb +11 -0
- data/lib/circuitry/subscriber.rb +116 -0
- data/lib/circuitry/topic.rb +21 -0
- data/lib/circuitry/topic_creator.rb +28 -0
- data/lib/circuitry/version.rb +3 -0
- metadata +177 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: 2399e4bc4f1e1d82e117415684b20c2920e34020
|
4
|
+
data.tar.gz: 9b497dde60c50e60b945ef80be7dbe07382755e8
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: b026a20b13d88fcfa09b7ab069578c236fca5dff3aa94f80b98e13aa4d2520b195a4c0cbed62f679211ee27cc54bb05be2f8657f6b738e63620422883b692be7
|
7
|
+
data.tar.gz: 6248ef0005b4c98c9aa3d50cdf977a3426b78d8f8eac8a420ed4066ae258b2128e614e1ef091e5b829ba7eea1610546ec7b27692f554e2e2572f1fe5777149c8
|
data/.gitignore
ADDED
data/.rspec
ADDED
data/Gemfile
ADDED
data/LICENSE.txt
ADDED
@@ -0,0 +1,21 @@
|
|
1
|
+
The MIT License (MIT)
|
2
|
+
|
3
|
+
Copyright (c) 2015 Kapost
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
7
|
+
in the Software without restriction, including without limitation the rights
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
10
|
+
furnished to do so, subject to the following conditions:
|
11
|
+
|
12
|
+
The above copyright notice and this permission notice shall be included in
|
13
|
+
all copies or substantial portions of the Software.
|
14
|
+
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
21
|
+
THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,183 @@
|
|
1
|
+
# Circuitry
|
2
|
+
|
3
|
+
Notification pub/sub and message queue processing using Amazon
|
4
|
+
[SNS](http://aws.amazon.com/sns/) & [SQS](http://aws.amazon.com/sqs/).
|
5
|
+
|
6
|
+
[](https://codeclimate.com/repos/55720235e30ba0148f003033/feed)
|
7
|
+
[](https://codeclimate.com/repos/55720235e30ba0148f003033/coverage)
|
8
|
+
|
9
|
+
## Installation
|
10
|
+
|
11
|
+
Add this line to your application's Gemfile:
|
12
|
+
|
13
|
+
```ruby
|
14
|
+
gem 'circuitry'
|
15
|
+
```
|
16
|
+
|
17
|
+
And then execute:
|
18
|
+
|
19
|
+
$ bundle
|
20
|
+
|
21
|
+
Or install it yourself as:
|
22
|
+
|
23
|
+
$ gem install circuitry
|
24
|
+
|
25
|
+
## Usage
|
26
|
+
|
27
|
+
Circuitry is configured via its configuration object.
|
28
|
+
|
29
|
+
```ruby
|
30
|
+
Circuitry.config do |c|
|
31
|
+
c.access_key = 'YOUR_AWS_ACCESS_KEY'
|
32
|
+
c.secret_key = 'YOUR_AWS_SECRET_KEY'
|
33
|
+
c.region = 'us-east-1'
|
34
|
+
c.logger = Rails.logger
|
35
|
+
c.error_handler = proc do |error|
|
36
|
+
HoneyBadger.notify(error)
|
37
|
+
HoneyBadger.flush
|
38
|
+
end
|
39
|
+
end
|
40
|
+
```
|
41
|
+
|
42
|
+
Available configuration options include:
|
43
|
+
|
44
|
+
* `access_key`: The AWS access key ID that has access to SNS publishing and/or
|
45
|
+
SQS subscribing. *(required)*
|
46
|
+
* `secret_key`: The AWS secret access key that has access to SNS publishing
|
47
|
+
and/or SQS subscribing. *(required)*
|
48
|
+
* `region`: The AWS region that your SNS and/or SQS account lives in.
|
49
|
+
*(optional, default: "us-east-1")*
|
50
|
+
* `logger`: The logger to use for informational output, warnings, and error
|
51
|
+
messages. *(optional, default: `Logger.new(STDOUT)`)*
|
52
|
+
* `error_handler`: An object that responds to `call` with two arguments: the
|
53
|
+
deserialized message contents and the topic name used when publishing to SNS.
|
54
|
+
*(optional, default: `nil`)*
|
55
|
+
|
56
|
+
### Publishing
|
57
|
+
|
58
|
+
Publishing is done via the `Circuitry.publish` method. It accepts a topic name
|
59
|
+
the represents the SNS topic along with any non-nil object, representing the data
|
60
|
+
to be serialized. Whatever object is called will have its `to_json` method
|
61
|
+
called for serialization.
|
62
|
+
|
63
|
+
```ruby
|
64
|
+
obj = { foo: 'foo', bar: 'bar' }
|
65
|
+
Circuitry.publish('any-topic-name', obj)
|
66
|
+
```
|
67
|
+
|
68
|
+
The `publish` method also accepts options that impact instantiation of the
|
69
|
+
`Publisher` object, which currently includes the following options.
|
70
|
+
|
71
|
+
* `:async` - Whether or not publishing should occur in the background. Please
|
72
|
+
refer to the [Asynchronous Support](#asynchronous-support) section for more
|
73
|
+
details regarding this option. (default: false)
|
74
|
+
|
75
|
+
```ruby
|
76
|
+
obj = { foo: 'foo', bar: 'bar' }
|
77
|
+
Circuitry.publish('my-topic-name', obj, async: true)
|
78
|
+
```
|
79
|
+
|
80
|
+
Alternatively, if your options hash will remain unchanged, you can build a single
|
81
|
+
`Publisher` object to use for all publishing.
|
82
|
+
|
83
|
+
```ruby
|
84
|
+
options = { ... }
|
85
|
+
publisher = Circuitry::Publisher.new(options)
|
86
|
+
publisher.publish('my-topic-name', obj)
|
87
|
+
```
|
88
|
+
|
89
|
+
### Subscribing
|
90
|
+
|
91
|
+
Subscribing is done via the `Circuitry.subscribe` method. It accepts an SQS queue
|
92
|
+
URL and takes a block for processing each message. This method performs
|
93
|
+
synchronously by default, and as such does not return.
|
94
|
+
|
95
|
+
```ruby
|
96
|
+
Circuitry.subscribe('https://sqs.REGION.amazonaws.com/ACCOUNT-ID/QUEUE-NAME') do |message, topic_name|
|
97
|
+
puts "Received #{topic_name} message: #{message.inspect}"
|
98
|
+
end
|
99
|
+
```
|
100
|
+
|
101
|
+
The `subscribe` method also accepts options that impact instantiation of the
|
102
|
+
`Subscriber` object, which currently includes the following options.
|
103
|
+
|
104
|
+
* `:async` - Whether or not subscribing should occur in the background. Please
|
105
|
+
refer to the [Asynchronous Support](#asynchronous-support) section for more
|
106
|
+
details regarding this option. (default: false)
|
107
|
+
* `:wait_time` - The number of seconds to wait for messages while connected to
|
108
|
+
SQS. Anything above 0 results in long-polling, while 0 results in
|
109
|
+
short-polling. (default: 10)
|
110
|
+
* `:batch_size` - The number of messages to retrieve in a single SQS request.
|
111
|
+
(default: 10)
|
112
|
+
|
113
|
+
```ruby
|
114
|
+
Circuitry.subscribe('https://...', async: true, wait_time: 60, batch_size: 20) do |message, topic_name|
|
115
|
+
# ...
|
116
|
+
end
|
117
|
+
```
|
118
|
+
|
119
|
+
Alternatively, if your options hash will remain unchanged, you can build a single
|
120
|
+
`Subscriber` object to use for all subscribing.
|
121
|
+
|
122
|
+
```ruby
|
123
|
+
options = { ... }
|
124
|
+
subscriber = Circuitry::Subscriber.new(options)
|
125
|
+
subscriber.subscribe('https://...') do |message, topic_name|
|
126
|
+
# ...
|
127
|
+
end
|
128
|
+
```
|
129
|
+
|
130
|
+
### Asynchronous Support
|
131
|
+
|
132
|
+
Publishing or subscribing asynchronously occurs by forking a child process. That
|
133
|
+
child is then detached so that your application does not need to worry about
|
134
|
+
waiting for the process to finish.
|
135
|
+
|
136
|
+
There are two important notes regarding forking in general as it relates to
|
137
|
+
asynchronous support:
|
138
|
+
|
139
|
+
1. Forking is not supported on all platforms (e.g.: Windows and NetBSD 4),
|
140
|
+
requiring that your implementation use synchronous requests in such
|
141
|
+
circumstances. You can determine if asynchronous requests will work by
|
142
|
+
calling `Circuitry.platform_supports_async?`.
|
143
|
+
|
144
|
+
2. Forking results in resources being copied from the parent process to the child
|
145
|
+
process. In order to prevent database connection errors and the like, you
|
146
|
+
should properly handle closing and reopening resources before and after
|
147
|
+
forking, respectively. For example, if you are using Rails with Unicorn, you
|
148
|
+
may need to add the following code to your `unicorn.rb` configuration:
|
149
|
+
|
150
|
+
before_fork do |server, worker|
|
151
|
+
if defined?(ActiveRecord::Base)
|
152
|
+
ActiveRecord::Base.connection.disconnect!
|
153
|
+
end
|
154
|
+
end
|
155
|
+
|
156
|
+
after_fork do |server, worker|
|
157
|
+
if defined?(ActiveRecord::Base)
|
158
|
+
ActiveRecord::Base.establish_connection(
|
159
|
+
Rails.application.config.database_configuration[Rails.env]
|
160
|
+
)
|
161
|
+
end
|
162
|
+
end
|
163
|
+
|
164
|
+
Refer to your adapter's documentation to determine how resources are handled
|
165
|
+
with regards to forking.
|
166
|
+
|
167
|
+
## Development
|
168
|
+
|
169
|
+
After checking out the repo, run `bin/setup` to install dependencies. Then, run
|
170
|
+
`bin/console` for an interactive prompt that will allow you to experiment.
|
171
|
+
|
172
|
+
To install this gem onto your local machine, run `bundle exec rake install`. To
|
173
|
+
release a new version, update the version number in `version.rb`, and then run
|
174
|
+
`bundle exec rake release` to create a git tag for the version, push git commits
|
175
|
+
and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
|
176
|
+
|
177
|
+
## Contributing
|
178
|
+
|
179
|
+
1. Fork it ( https://github.com/kapost/circuitry/fork )
|
180
|
+
2. Create your feature branch (`git checkout -b my-new-feature`)
|
181
|
+
3. Commit your changes (`git commit -am 'Add some feature'`)
|
182
|
+
4. Push to the branch (`git push origin my-new-feature`)
|
183
|
+
5. Create a new Pull Request
|
data/Rakefile
ADDED
data/bin/console
ADDED
@@ -0,0 +1,14 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
|
3
|
+
require "bundler/setup"
|
4
|
+
require "circuitry"
|
5
|
+
|
6
|
+
# You can add fixtures and/or initialization code here to make experimenting
|
7
|
+
# with your gem easier. You can also use a different console, if you like.
|
8
|
+
|
9
|
+
# (If you use this, don't forget to add pry to your Gemfile!)
|
10
|
+
# require "pry"
|
11
|
+
# Pry.start
|
12
|
+
|
13
|
+
require "irb"
|
14
|
+
IRB.start
|
data/bin/setup
ADDED
data/circle.yml
ADDED
data/circuitry.gemspec
ADDED
@@ -0,0 +1,31 @@
|
|
1
|
+
# coding: utf-8
|
2
|
+
lib = File.expand_path('../lib', __FILE__)
|
3
|
+
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
|
4
|
+
require 'circuitry/version'
|
5
|
+
|
6
|
+
Gem::Specification.new do |spec|
|
7
|
+
spec.name = 'circuitry'
|
8
|
+
spec.version = Circuitry::VERSION
|
9
|
+
spec.authors = ['Matt Huggins']
|
10
|
+
spec.email = ['matt.huggins@kapost.com']
|
11
|
+
|
12
|
+
spec.summary = %q{Kapost notification pub/sub and message queue processing.}
|
13
|
+
spec.description = %q{Amazon SNS publishing and SQS queue processing.}
|
14
|
+
spec.homepage = 'https://github.com/kapost/circuitry'
|
15
|
+
spec.license = 'MIT'
|
16
|
+
|
17
|
+
spec.files = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
|
18
|
+
spec.bindir = 'exe'
|
19
|
+
spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
|
20
|
+
spec.require_paths = ['lib']
|
21
|
+
|
22
|
+
spec.add_dependency 'fog-aws', '~> 0.4'
|
23
|
+
spec.add_dependency 'virtus', '~> 1.0'
|
24
|
+
|
25
|
+
spec.add_development_dependency 'pry-nav'
|
26
|
+
spec.add_development_dependency 'bundler', '~> 1.8'
|
27
|
+
spec.add_development_dependency 'rake', '~> 10.0'
|
28
|
+
spec.add_development_dependency 'rspec', '~> 3.2'
|
29
|
+
spec.add_development_dependency 'rspec-its', '~> 1.2'
|
30
|
+
spec.add_development_dependency 'codeclimate-test-reporter'
|
31
|
+
end
|
data/lib/circuitry.rb
ADDED
@@ -0,0 +1,24 @@
|
|
1
|
+
require 'circuitry/version'
|
2
|
+
require 'circuitry/configuration'
|
3
|
+
require 'circuitry/publisher'
|
4
|
+
require 'circuitry/subscriber'
|
5
|
+
|
6
|
+
module Circuitry
|
7
|
+
def self.config(&block)
|
8
|
+
@config ||= Configuration.new
|
9
|
+
block.call(@config) if block_given?
|
10
|
+
@config
|
11
|
+
end
|
12
|
+
|
13
|
+
def self.publish(topic_name, object, options = {})
|
14
|
+
Publisher.new(options).publish(topic_name, object)
|
15
|
+
end
|
16
|
+
|
17
|
+
def self.subscribe(queue, options = {}, &block)
|
18
|
+
Subscriber.new(queue, options).subscribe(&block)
|
19
|
+
end
|
20
|
+
|
21
|
+
def self.platform_supports_async?
|
22
|
+
Process.respond_to?(:fork)
|
23
|
+
end
|
24
|
+
end
|
@@ -0,0 +1,18 @@
|
|
1
|
+
module Circuitry
|
2
|
+
class NotSupportedError < StandardError; end
|
3
|
+
|
4
|
+
module Concerns
|
5
|
+
module Async
|
6
|
+
def process_asynchronously(&block)
|
7
|
+
raise NotSupportedError, 'Your platform does not support forking' unless platform_supports_async?
|
8
|
+
|
9
|
+
pid = fork(&block)
|
10
|
+
Process.detach(pid)
|
11
|
+
end
|
12
|
+
|
13
|
+
def platform_supports_async?
|
14
|
+
Circuitry.platform_supports_async?
|
15
|
+
end
|
16
|
+
end
|
17
|
+
end
|
18
|
+
end
|
@@ -0,0 +1,22 @@
|
|
1
|
+
require 'logger'
|
2
|
+
require 'virtus'
|
3
|
+
|
4
|
+
module Circuitry
|
5
|
+
class Configuration
|
6
|
+
include Virtus::Model
|
7
|
+
|
8
|
+
attribute :access_key, String
|
9
|
+
attribute :secret_key, String
|
10
|
+
attribute :region, String, default: 'us-east-1'
|
11
|
+
attribute :logger, Logger, default: Logger.new(STDERR)
|
12
|
+
attribute :error_handler
|
13
|
+
|
14
|
+
def aws_options
|
15
|
+
{
|
16
|
+
aws_access_key_id: access_key,
|
17
|
+
aws_secret_access_key: secret_key,
|
18
|
+
region: region,
|
19
|
+
}
|
20
|
+
end
|
21
|
+
end
|
22
|
+
end
|
@@ -0,0 +1,32 @@
|
|
1
|
+
require 'json'
|
2
|
+
require 'circuitry/topic'
|
3
|
+
|
4
|
+
module Circuitry
|
5
|
+
class Message
|
6
|
+
attr_reader :raw
|
7
|
+
|
8
|
+
def initialize(raw)
|
9
|
+
@raw = raw
|
10
|
+
end
|
11
|
+
|
12
|
+
def context
|
13
|
+
@context ||= JSON.parse(raw['Body'])
|
14
|
+
end
|
15
|
+
|
16
|
+
def body
|
17
|
+
@body ||= JSON.parse(context['Message'], quirks_mode: true)
|
18
|
+
end
|
19
|
+
|
20
|
+
def topic
|
21
|
+
@topic ||= Topic.new(context['TopicArn'])
|
22
|
+
end
|
23
|
+
|
24
|
+
def id
|
25
|
+
raw['MessageId']
|
26
|
+
end
|
27
|
+
|
28
|
+
def receipt_handle
|
29
|
+
raw['ReceiptHandle']
|
30
|
+
end
|
31
|
+
end
|
32
|
+
end
|
@@ -0,0 +1,60 @@
|
|
1
|
+
require 'json'
|
2
|
+
require 'circuitry/concerns/async'
|
3
|
+
require 'circuitry/services/sns'
|
4
|
+
require 'circuitry/topic_creator'
|
5
|
+
|
6
|
+
module Circuitry
|
7
|
+
class PublishError < StandardError; end
|
8
|
+
|
9
|
+
class Publisher
|
10
|
+
include Concerns::Async
|
11
|
+
include Services::SNS
|
12
|
+
|
13
|
+
DEFAULT_OPTIONS = {
|
14
|
+
async: false,
|
15
|
+
}.freeze
|
16
|
+
|
17
|
+
def initialize(options = {})
|
18
|
+
options = DEFAULT_OPTIONS.merge(options)
|
19
|
+
|
20
|
+
@async = !!options[:async]
|
21
|
+
end
|
22
|
+
|
23
|
+
def publish(topic_name, object)
|
24
|
+
raise ArgumentError.new('topic_name cannot be nil') if topic_name.nil?
|
25
|
+
raise ArgumentError.new('object cannot be nil') if object.nil?
|
26
|
+
|
27
|
+
unless can_publish?
|
28
|
+
logger.warn('Circuitry unable to publish: AWS configuration is not set.')
|
29
|
+
return
|
30
|
+
end
|
31
|
+
|
32
|
+
process = -> do
|
33
|
+
topic = TopicCreator.find_or_create(topic_name)
|
34
|
+
sns.publish(topic.arn, object.to_json)
|
35
|
+
end
|
36
|
+
|
37
|
+
if async?
|
38
|
+
process_asynchronously(&process)
|
39
|
+
else
|
40
|
+
process.call
|
41
|
+
end
|
42
|
+
end
|
43
|
+
|
44
|
+
def async?
|
45
|
+
@async
|
46
|
+
end
|
47
|
+
|
48
|
+
private
|
49
|
+
|
50
|
+
def logger
|
51
|
+
Circuitry.config.logger
|
52
|
+
end
|
53
|
+
|
54
|
+
def can_publish?
|
55
|
+
Circuitry.config.aws_options.values.all? do |value|
|
56
|
+
!value.nil? && !value.empty?
|
57
|
+
end
|
58
|
+
end
|
59
|
+
end
|
60
|
+
end
|
@@ -0,0 +1,116 @@
|
|
1
|
+
require 'circuitry/concerns/async'
|
2
|
+
require 'circuitry/services/sqs'
|
3
|
+
require 'circuitry/message'
|
4
|
+
|
5
|
+
module Circuitry
|
6
|
+
class SubscribeError < StandardError; end
|
7
|
+
|
8
|
+
class Subscriber
|
9
|
+
include Concerns::Async
|
10
|
+
include Services::SQS
|
11
|
+
|
12
|
+
attr_reader :queue, :wait_time, :batch_size, :max_retries, :failure_queue
|
13
|
+
|
14
|
+
DEFAULT_OPTIONS = {
|
15
|
+
async: false,
|
16
|
+
wait_time: 10,
|
17
|
+
batch_size: 10,
|
18
|
+
}.freeze
|
19
|
+
|
20
|
+
CONNECTION_ERRORS = [
|
21
|
+
Excon::Errors::Forbidden,
|
22
|
+
].freeze
|
23
|
+
|
24
|
+
def initialize(queue, options = {})
|
25
|
+
raise ArgumentError.new('queue cannot be nil') if queue.nil?
|
26
|
+
|
27
|
+
options = DEFAULT_OPTIONS.merge(options)
|
28
|
+
|
29
|
+
@queue = queue
|
30
|
+
@async = !!options[:async]
|
31
|
+
@wait_time = options[:wait_time]
|
32
|
+
@batch_size = options[:batch_size]
|
33
|
+
end
|
34
|
+
|
35
|
+
def subscribe(&block)
|
36
|
+
raise ArgumentError.new('block required') if block.nil?
|
37
|
+
|
38
|
+
unless can_subscribe?
|
39
|
+
logger.warn('Circuitry unable to subscribe: AWS configuration is not set.')
|
40
|
+
return
|
41
|
+
end
|
42
|
+
|
43
|
+
process = -> do
|
44
|
+
loop do
|
45
|
+
begin
|
46
|
+
receive_messages(&block)
|
47
|
+
rescue *CONNECTION_ERRORS => e
|
48
|
+
logger.error "Connection error to #{queue}: #{e}"
|
49
|
+
raise SubscribeError.new(e)
|
50
|
+
end
|
51
|
+
end
|
52
|
+
end
|
53
|
+
|
54
|
+
if async?
|
55
|
+
process_asynchronously(&process)
|
56
|
+
else
|
57
|
+
process.call
|
58
|
+
end
|
59
|
+
end
|
60
|
+
|
61
|
+
def async?
|
62
|
+
@async
|
63
|
+
end
|
64
|
+
|
65
|
+
private
|
66
|
+
|
67
|
+
def receive_messages(&block)
|
68
|
+
response = sqs.receive_message(queue, 'MaxNumberOfMessages' => batch_size, 'WaitTimeSeconds' => wait_time)
|
69
|
+
messages = response.body['Message']
|
70
|
+
return if messages.empty?
|
71
|
+
|
72
|
+
messages.each do |message|
|
73
|
+
process_message(message, &block)
|
74
|
+
end
|
75
|
+
end
|
76
|
+
|
77
|
+
def process_message(message, &block)
|
78
|
+
message = Message.new(message)
|
79
|
+
|
80
|
+
unless message.nil?
|
81
|
+
logger.info "Processing message #{message.id}"
|
82
|
+
handle_message(message, &block)
|
83
|
+
delete_message(message)
|
84
|
+
end
|
85
|
+
rescue => e
|
86
|
+
logger.error "Error processing message #{message.id}: #{e}"
|
87
|
+
error_handler.call(e) if error_handler
|
88
|
+
end
|
89
|
+
|
90
|
+
def handle_message(message, &block)
|
91
|
+
block.call(message.body, message.topic.name)
|
92
|
+
rescue => e
|
93
|
+
logger.error("Error handling message #{message.id}: #{e}")
|
94
|
+
raise e
|
95
|
+
end
|
96
|
+
|
97
|
+
def delete_message(message)
|
98
|
+
logger.info("Removing message #{message.id} from queue")
|
99
|
+
sqs.delete_message(queue, message.receipt_handle)
|
100
|
+
end
|
101
|
+
|
102
|
+
def logger
|
103
|
+
Circuitry.config.logger
|
104
|
+
end
|
105
|
+
|
106
|
+
def error_handler
|
107
|
+
Circuitry.config.error_handler
|
108
|
+
end
|
109
|
+
|
110
|
+
def can_subscribe?
|
111
|
+
Circuitry.config.aws_options.values.all? do |value|
|
112
|
+
!value.nil? && !value.empty?
|
113
|
+
end
|
114
|
+
end
|
115
|
+
end
|
116
|
+
end
|
@@ -0,0 +1,21 @@
|
|
1
|
+
module Circuitry
|
2
|
+
class Topic
|
3
|
+
attr_reader :arn
|
4
|
+
|
5
|
+
def initialize(arn)
|
6
|
+
@arn = arn
|
7
|
+
end
|
8
|
+
|
9
|
+
def name
|
10
|
+
arn.split(':').last
|
11
|
+
end
|
12
|
+
|
13
|
+
def ==(obj)
|
14
|
+
obj.hash == self.hash
|
15
|
+
end
|
16
|
+
|
17
|
+
def hash
|
18
|
+
[self.class, arn].hash
|
19
|
+
end
|
20
|
+
end
|
21
|
+
end
|
@@ -0,0 +1,28 @@
|
|
1
|
+
require 'circuitry/services/sns'
|
2
|
+
require 'circuitry/topic'
|
3
|
+
|
4
|
+
module Circuitry
|
5
|
+
class TopicCreatorError < StandardError; end
|
6
|
+
|
7
|
+
class TopicCreator
|
8
|
+
include Services::SNS
|
9
|
+
|
10
|
+
attr_reader :topic_name
|
11
|
+
|
12
|
+
def self.find_or_create(topic_name)
|
13
|
+
new(topic_name).topic
|
14
|
+
end
|
15
|
+
|
16
|
+
def initialize(topic_name)
|
17
|
+
@topic_name = topic_name
|
18
|
+
end
|
19
|
+
|
20
|
+
def topic
|
21
|
+
return @topic if defined?(@topic)
|
22
|
+
|
23
|
+
response = sns.create_topic(topic_name)
|
24
|
+
arn = response.body.fetch('TopicArn') { raise TopicCreatorError.new('No TopicArn returned from SNS') }
|
25
|
+
@topic = Topic.new(arn)
|
26
|
+
end
|
27
|
+
end
|
28
|
+
end
|
metadata
ADDED
@@ -0,0 +1,177 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: circuitry
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 1.0.0
|
5
|
+
platform: ruby
|
6
|
+
authors:
|
7
|
+
- Matt Huggins
|
8
|
+
autorequire:
|
9
|
+
bindir: exe
|
10
|
+
cert_chain: []
|
11
|
+
date: 2015-06-25 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: fog-aws
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - "~>"
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: '0.4'
|
20
|
+
type: :runtime
|
21
|
+
prerelease: false
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - "~>"
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: '0.4'
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
name: virtus
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - "~>"
|
32
|
+
- !ruby/object:Gem::Version
|
33
|
+
version: '1.0'
|
34
|
+
type: :runtime
|
35
|
+
prerelease: false
|
36
|
+
version_requirements: !ruby/object:Gem::Requirement
|
37
|
+
requirements:
|
38
|
+
- - "~>"
|
39
|
+
- !ruby/object:Gem::Version
|
40
|
+
version: '1.0'
|
41
|
+
- !ruby/object:Gem::Dependency
|
42
|
+
name: pry-nav
|
43
|
+
requirement: !ruby/object:Gem::Requirement
|
44
|
+
requirements:
|
45
|
+
- - ">="
|
46
|
+
- !ruby/object:Gem::Version
|
47
|
+
version: '0'
|
48
|
+
type: :development
|
49
|
+
prerelease: false
|
50
|
+
version_requirements: !ruby/object:Gem::Requirement
|
51
|
+
requirements:
|
52
|
+
- - ">="
|
53
|
+
- !ruby/object:Gem::Version
|
54
|
+
version: '0'
|
55
|
+
- !ruby/object:Gem::Dependency
|
56
|
+
name: bundler
|
57
|
+
requirement: !ruby/object:Gem::Requirement
|
58
|
+
requirements:
|
59
|
+
- - "~>"
|
60
|
+
- !ruby/object:Gem::Version
|
61
|
+
version: '1.8'
|
62
|
+
type: :development
|
63
|
+
prerelease: false
|
64
|
+
version_requirements: !ruby/object:Gem::Requirement
|
65
|
+
requirements:
|
66
|
+
- - "~>"
|
67
|
+
- !ruby/object:Gem::Version
|
68
|
+
version: '1.8'
|
69
|
+
- !ruby/object:Gem::Dependency
|
70
|
+
name: rake
|
71
|
+
requirement: !ruby/object:Gem::Requirement
|
72
|
+
requirements:
|
73
|
+
- - "~>"
|
74
|
+
- !ruby/object:Gem::Version
|
75
|
+
version: '10.0'
|
76
|
+
type: :development
|
77
|
+
prerelease: false
|
78
|
+
version_requirements: !ruby/object:Gem::Requirement
|
79
|
+
requirements:
|
80
|
+
- - "~>"
|
81
|
+
- !ruby/object:Gem::Version
|
82
|
+
version: '10.0'
|
83
|
+
- !ruby/object:Gem::Dependency
|
84
|
+
name: rspec
|
85
|
+
requirement: !ruby/object:Gem::Requirement
|
86
|
+
requirements:
|
87
|
+
- - "~>"
|
88
|
+
- !ruby/object:Gem::Version
|
89
|
+
version: '3.2'
|
90
|
+
type: :development
|
91
|
+
prerelease: false
|
92
|
+
version_requirements: !ruby/object:Gem::Requirement
|
93
|
+
requirements:
|
94
|
+
- - "~>"
|
95
|
+
- !ruby/object:Gem::Version
|
96
|
+
version: '3.2'
|
97
|
+
- !ruby/object:Gem::Dependency
|
98
|
+
name: rspec-its
|
99
|
+
requirement: !ruby/object:Gem::Requirement
|
100
|
+
requirements:
|
101
|
+
- - "~>"
|
102
|
+
- !ruby/object:Gem::Version
|
103
|
+
version: '1.2'
|
104
|
+
type: :development
|
105
|
+
prerelease: false
|
106
|
+
version_requirements: !ruby/object:Gem::Requirement
|
107
|
+
requirements:
|
108
|
+
- - "~>"
|
109
|
+
- !ruby/object:Gem::Version
|
110
|
+
version: '1.2'
|
111
|
+
- !ruby/object:Gem::Dependency
|
112
|
+
name: codeclimate-test-reporter
|
113
|
+
requirement: !ruby/object:Gem::Requirement
|
114
|
+
requirements:
|
115
|
+
- - ">="
|
116
|
+
- !ruby/object:Gem::Version
|
117
|
+
version: '0'
|
118
|
+
type: :development
|
119
|
+
prerelease: false
|
120
|
+
version_requirements: !ruby/object:Gem::Requirement
|
121
|
+
requirements:
|
122
|
+
- - ">="
|
123
|
+
- !ruby/object:Gem::Version
|
124
|
+
version: '0'
|
125
|
+
description: Amazon SNS publishing and SQS queue processing.
|
126
|
+
email:
|
127
|
+
- matt.huggins@kapost.com
|
128
|
+
executables: []
|
129
|
+
extensions: []
|
130
|
+
extra_rdoc_files: []
|
131
|
+
files:
|
132
|
+
- ".gitignore"
|
133
|
+
- ".rspec"
|
134
|
+
- Gemfile
|
135
|
+
- LICENSE.txt
|
136
|
+
- README.md
|
137
|
+
- Rakefile
|
138
|
+
- bin/console
|
139
|
+
- bin/setup
|
140
|
+
- circle.yml
|
141
|
+
- circuitry.gemspec
|
142
|
+
- lib/circuitry.rb
|
143
|
+
- lib/circuitry/concerns/async.rb
|
144
|
+
- lib/circuitry/configuration.rb
|
145
|
+
- lib/circuitry/message.rb
|
146
|
+
- lib/circuitry/publisher.rb
|
147
|
+
- lib/circuitry/services/sns.rb
|
148
|
+
- lib/circuitry/services/sqs.rb
|
149
|
+
- lib/circuitry/subscriber.rb
|
150
|
+
- lib/circuitry/topic.rb
|
151
|
+
- lib/circuitry/topic_creator.rb
|
152
|
+
- lib/circuitry/version.rb
|
153
|
+
homepage: https://github.com/kapost/circuitry
|
154
|
+
licenses:
|
155
|
+
- MIT
|
156
|
+
metadata: {}
|
157
|
+
post_install_message:
|
158
|
+
rdoc_options: []
|
159
|
+
require_paths:
|
160
|
+
- lib
|
161
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
162
|
+
requirements:
|
163
|
+
- - ">="
|
164
|
+
- !ruby/object:Gem::Version
|
165
|
+
version: '0'
|
166
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
167
|
+
requirements:
|
168
|
+
- - ">="
|
169
|
+
- !ruby/object:Gem::Version
|
170
|
+
version: '0'
|
171
|
+
requirements: []
|
172
|
+
rubyforge_project:
|
173
|
+
rubygems_version: 2.4.8
|
174
|
+
signing_key:
|
175
|
+
specification_version: 4
|
176
|
+
summary: Kapost notification pub/sub and message queue processing.
|
177
|
+
test_files: []
|