announce 0.1.1 → 0.2.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.travis.yml +0 -4
- data/Guardfile +1 -1
- data/README.md +307 -4
- data/Rakefile +1 -1
- data/announce.gemspec +0 -1
- data/lib/announce.rb +7 -18
- data/lib/announce/adapters/base_adapter.rb +8 -4
- data/lib/announce/adapters/shoryuken_adapter.rb +41 -57
- data/lib/announce/configuration.rb +31 -8
- data/lib/announce/core_ext.rb +12 -3
- data/lib/announce/publisher.rb +2 -0
- data/lib/announce/railtie.rb +8 -0
- data/lib/announce/subscriber.rb +0 -2
- data/lib/announce/version.rb +1 -1
- data/lib/tasks/announce.rake +8 -0
- metadata +4 -16
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 9789a278d07101dcd8a592549cd616a0d9792577
|
4
|
+
data.tar.gz: 9bdb61307e9b940ba340bd604a42acee381f0f26
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 0422db42eed7b943f3fde2133ad9e6b67438eab02b874697d4b39b5cb54b65bda5470e5b262241dfeec49332a2db3d71d137915f78d54cbf44da88a9229130ef
|
7
|
+
data.tar.gz: 0689cc11bb610d00bbd074bf41924d589a3be572b75f167f1e2a1817b363b8bdcf3298cbfd772847b3b0a8aa328418938ef861e83373b283eaf1a01c211b0c03
|
data/.travis.yml
CHANGED
data/Guardfile
CHANGED
@@ -26,7 +26,7 @@
|
|
26
26
|
guard :minitest do
|
27
27
|
# with Minitest::Unit
|
28
28
|
watch(%r{^test/(.*)\/?test_(.*)\.rb$})
|
29
|
-
watch(%r{^lib/
|
29
|
+
watch(%r{^lib/(.*/)?([^/]+)\.rb$}) { |m| "test/#{m[1]}test_#{m[2]}.rb" }
|
30
30
|
watch(%r{^test/test_helper\.rb$}) { 'test' }
|
31
31
|
|
32
32
|
# with Minitest::Spec
|
data/README.md
CHANGED
@@ -8,32 +8,335 @@
|
|
8
8
|
|
9
9
|
### Announce lets your services know about events, and helps you process them.
|
10
10
|
|
11
|
-
|
11
|
+
It supports the publish/subscribe pattern, applied to sending out messages for events structed as `action`s that happen to `subject`s.
|
12
|
+
|
13
|
+
Announce does not include its own job processor, as projects will likely already be running a process to handle asynchronous jobs. Instead, it is built to integrate with existing job processors, like `shoryuken`, so don't have workers written different ways or running in multiple processes.
|
14
|
+
|
15
|
+
You can also write ActiveJob classes to process announce messages, further allowing a consistent abstraction of how asynchronous processing is handled.
|
16
|
+
|
17
|
+
With announce, an application or service can send event messages out, but without knowing who will receive them or how they will be processed. Other services and applications can pick and choose which events to receive by subscribing to them and process them appropriately, all without the publishing app having any awareness.
|
18
|
+
|
19
|
+
Announce messages events are structured as a combination of `subject` and `action` to identify the type of message, and a message body containing the specific data about the event. All three required for sending a message:
|
20
|
+
```ruby
|
21
|
+
# You can send a message from the Announce module directly
|
22
|
+
Announce.publish(:story, :publish, id: story.id)
|
23
|
+
|
24
|
+
# There is also a module to make this slightly easier
|
25
|
+
class SomeController
|
26
|
+
include Announce::Publisher
|
27
|
+
|
28
|
+
def create
|
29
|
+
publish(:some, :create, id: @some.id)
|
30
|
+
end
|
31
|
+
end
|
32
|
+
```
|
33
|
+
|
34
|
+
When building a Ruby app or service to receive messages, subscribe a job class to announcements like this:
|
35
|
+
```
|
36
|
+
require 'announce'
|
37
|
+
|
38
|
+
class SomeCreateJob < ActiveJob::Base
|
39
|
+
include Announce::Subscriber
|
40
|
+
|
41
|
+
subscribe_to :some, [:action]
|
42
|
+
end
|
43
|
+
```
|
44
|
+
|
45
|
+
The `announce` gem uses adapters so different message brokers can be used, but currently only has `shoryuken` for production, an `inline` adapter where messages are only processed synchronously and within the existing app, and a `test` adapter useful for stubbing out messaging sending and receiving.
|
46
|
+
|
47
|
+
The `shoryuken` adapter uses a combination of SNS and SQS for message handling, and includes a `rake` task to create the required SNS topics, SQS queues, and subscriptions between the two.
|
12
48
|
|
13
|
-
It is built on top of other job processors, like shoryuken and ActiveJob to process the messages.
|
14
49
|
|
15
50
|
## Installation
|
16
51
|
|
17
52
|
Add this line to your application's Gemfile:
|
18
53
|
|
19
54
|
```ruby
|
55
|
+
gem 'shoryuken'
|
20
56
|
gem 'announce'
|
21
57
|
```
|
22
58
|
|
59
|
+
(The only interesting adapter at the moment is shoryuken, so you'll probably want to use that.)
|
60
|
+
|
23
61
|
And then execute:
|
24
62
|
|
25
63
|
$ bundle
|
26
64
|
|
27
65
|
Or install it yourself as:
|
28
66
|
|
29
|
-
$ gem install announce
|
67
|
+
$ gem install announce shoryuken
|
68
|
+
|
69
|
+
## Configuration
|
70
|
+
|
71
|
+
Announce is configured 3 ways:
|
72
|
+
|
73
|
+
### Existing defaults are used if there is an ActiveJob configuration
|
74
|
+
|
75
|
+
If you are using `announce` with Rails 4.2 and ActiveJob, add the following lines to your `config/application.rb` file:
|
76
|
+
```ruby
|
77
|
+
config.active_job.queue_adapter = :shoryuken
|
78
|
+
config.active_job.queue_name_prefix = Rails.env
|
79
|
+
config.active_job.queue_name_delimiter = '_'
|
80
|
+
```
|
81
|
+
|
82
|
+
You can set these values in announce itself, but if they are already in your application configuration, there's no reason to duplicate.
|
83
|
+
|
84
|
+
### `subscribe_to` calls in your job classes
|
85
|
+
|
86
|
+
These method calls are used to configure `shoryuken` in two ways.
|
87
|
+
|
88
|
+
It adds the queues related to these subscriptions to the shoryuken worker process, so that process will request new messages for these subscriptions.
|
89
|
+
|
90
|
+
It also registers that the class calling subscribe should receive the announcements for this subscription.
|
91
|
+
|
92
|
+
|
93
|
+
### An `announce.yml` config file
|
94
|
+
|
95
|
+
You can specify a different location, but by default `announce` will load a config file from `config/announce.yml`.
|
96
|
+
|
97
|
+
This file allows you to set all the configuration for announce.
|
98
|
+
The most important aspects are specifying what subjects and actions your app will send and receive, and the namespacing for your topics, queues, and subscriptions.
|
99
|
+
|
100
|
+
This file is what is used for configuring SNS/SQS, it is not used to specify what queues shoryuken listens to, that is done with class methods in the jobs.
|
101
|
+
|
102
|
+
Here is a documented example `announce.yml` file:
|
103
|
+
```yaml
|
104
|
+
################################################################################
|
105
|
+
# `queue_name_prefix`
|
106
|
+
# defaults to config.active_job.queue_name_prefix, RAILS_ENV or APP_ENV in ENV, or 'development'
|
107
|
+
################################################################################
|
108
|
+
queue_name_prefix: <%= Rails.env %>
|
109
|
+
|
110
|
+
################################################################################
|
111
|
+
# `queue_name_delimiter`
|
112
|
+
# defaults to config.active_job.queue_name_delimiter or '_'
|
113
|
+
################################################################################
|
114
|
+
queue_name_delimiter: _
|
115
|
+
|
116
|
+
################################################################################
|
117
|
+
# `adapter`
|
118
|
+
# which messaging adapter to use
|
119
|
+
# based on config.active_job.queue_adapter or 'inline' if unspecified
|
120
|
+
################################################################################
|
121
|
+
adapter: shoryuken
|
122
|
+
|
123
|
+
################################################################################
|
124
|
+
# `namespace`
|
125
|
+
# short name to relate all subjects for all subscribers and publishers
|
126
|
+
# limit to characters friendly to SNS/SQS, used in topic and queue names
|
127
|
+
# 'announce' is the default if unspecified
|
128
|
+
################################################################################
|
129
|
+
namespace: announce
|
130
|
+
|
131
|
+
################################################################################
|
132
|
+
# `app_name`
|
133
|
+
# short name of this application or service, added to each published message
|
134
|
+
# limit to characters friendly to SQS, used in making queue names
|
135
|
+
# 'app' is the default if unspecified
|
136
|
+
################################################################################
|
137
|
+
app_name: app
|
138
|
+
|
139
|
+
################################################################################
|
140
|
+
# `publish`
|
141
|
+
# subject keys with action list values for announcements this app will make
|
142
|
+
# each subject must have at least one action.
|
143
|
+
################################################################################
|
144
|
+
publish:
|
145
|
+
foo:
|
146
|
+
- create
|
147
|
+
|
148
|
+
################################################################################
|
149
|
+
# `subscribe`
|
150
|
+
# subject keys with action list values for announcements received by the app
|
151
|
+
# each subject must have at least one action.
|
152
|
+
################################################################################
|
153
|
+
subscribe:
|
154
|
+
bar:
|
155
|
+
- delete
|
156
|
+
|
157
|
+
################################################################################
|
158
|
+
# `queues`
|
159
|
+
# shoryuken adapter only, parameters SQS creating SQS queues
|
160
|
+
# below are the default values for each parameter
|
161
|
+
################################################################################
|
162
|
+
queues:
|
163
|
+
VisibilityTimeout: 3600
|
164
|
+
DelaySeconds: 0
|
165
|
+
MaximumMessageSize: 262144
|
166
|
+
VisibilityTimeout: 3600
|
167
|
+
ReceiveMessageWaitTimeSeconds: 0
|
168
|
+
MessageRetentionPeriod: 604800
|
169
|
+
```
|
30
170
|
|
31
171
|
## Usage
|
32
172
|
|
33
|
-
|
173
|
+
### Naming
|
174
|
+
|
175
|
+
For the shoryuken adapter (and in the abstract base adapter), topics are named in the following way:
|
176
|
+
```
|
177
|
+
"#{prefix}_#{namespace}_#{subject}_#{action}"
|
178
|
+
```
|
179
|
+
|
180
|
+
Queues names also include the specific subscriber's app_name:
|
181
|
+
```
|
182
|
+
"#{prefix}_#{namespace}_#{app_name}_#{subject}_#{action}"
|
183
|
+
```
|
184
|
+
|
185
|
+
For example, for a production environment, with the app `cms` subscribing to the `story` subject and `update` action, the Topic will be:
|
186
|
+
```
|
187
|
+
production_announce_story_update
|
188
|
+
```
|
189
|
+
The cms's subscribing Queue will be named:
|
190
|
+
```
|
191
|
+
production_announce_cms_story_update
|
192
|
+
```
|
193
|
+
|
194
|
+
### Configuring the Broker
|
195
|
+
|
196
|
+
To create the needed queues, topics, and subscriptions, first configure the `announce.yml` file, then run this rake task:
|
197
|
+
```
|
198
|
+
> bundle exec rake announce:configure_broker
|
199
|
+
```
|
200
|
+
|
201
|
+
If you are not using Rails, you can call the broker configuration from Ruby:
|
202
|
+
```ruby
|
203
|
+
Announce.configure(config_file: '/path/to/config/file/announce.yml')
|
204
|
+
Announce.configure_broker
|
205
|
+
```
|
206
|
+
|
207
|
+
### Sending
|
208
|
+
|
209
|
+
To send a message, the topics need to exists, so the broker must have been configured as per above.
|
210
|
+
|
211
|
+
An announcement is made using the `publish` method (also aliased as `announce`).
|
212
|
+
It take the 3 required params for `subject`, `action`, and the message `body`, and an optional `options` hash.
|
213
|
+
|
214
|
+
```ruby
|
215
|
+
# `publish` (or `announce`)
|
216
|
+
# subject - the type of entity that this event relates to
|
217
|
+
# action - the action that has occurred related to the entity
|
218
|
+
# body - the message body that will be sent out
|
219
|
+
# options (optional) - a hash of options to affect the publish, currently unused.
|
220
|
+
#
|
221
|
+
Announce.publish(:story, :publish, id: story.id, {})
|
222
|
+
Announce.announce(:story, :publish, id: story.id, {})
|
223
|
+
```
|
224
|
+
|
225
|
+
There is also a module you can include to get `publish` and `announce` methods:
|
226
|
+
```
|
227
|
+
class SomeController
|
228
|
+
include Announce::Publisher
|
229
|
+
|
230
|
+
def create
|
231
|
+
@some = Some.new(some_params)
|
232
|
+
@some.save!
|
233
|
+
|
234
|
+
# announce it!
|
235
|
+
publish(:some, :create, id: @some.id)
|
236
|
+
respond_with(@some)
|
237
|
+
end
|
238
|
+
end
|
239
|
+
```
|
240
|
+
|
241
|
+
The message `body` should be simple and serializable (json compatible), and something that any receiver could parse and understand, so don't include implementation details like specific Ruby classes.
|
242
|
+
|
243
|
+
Unlike other job libraries which specify the Ruby `Class` for processing as part of the message, these announcements are meant to be decoupled - the sender should make no assumptions about the receiving application(s).
|
244
|
+
|
245
|
+
|
246
|
+
### Processing
|
247
|
+
|
248
|
+
To designate that a Ruby class will process a message, you must include the `Announce:Subscriber` module and call `subscribe_to` class method.
|
249
|
+
|
250
|
+
```
|
251
|
+
require 'announce'
|
252
|
+
|
253
|
+
class SomeCreateJob < ActiveJob::Base
|
254
|
+
include Announce::Subscriber
|
255
|
+
|
256
|
+
subscribe_to :some, [:create]
|
257
|
+
|
258
|
+
def receive_some_create(body)
|
259
|
+
do_something(body[:id])
|
260
|
+
end
|
261
|
+
end
|
262
|
+
```
|
263
|
+
|
264
|
+
This works because the `Announce::Subscriber` adds the `subscribe_to` class method, but also a default `perform(*args)` instance method that delegates message handling to a job instance method named `"receive_#{subject}_#{action}"`.
|
265
|
+
|
266
|
+
This default `perform` method only passes the message `body` to `receive_subject_action` methods. The `subject`, but `action` and full `message` object are made available as instance properties of the subscriber.
|
267
|
+
```
|
268
|
+
require 'announce'
|
269
|
+
|
270
|
+
class SomeCreateJob < ActiveJob::Base
|
271
|
+
include Announce::Subscriber
|
272
|
+
|
273
|
+
subscribe_to :some, [:create]
|
274
|
+
|
275
|
+
def receive_some_create(body)
|
276
|
+
puts "subject: #{subject} is some"
|
277
|
+
puts "action: #{action} is create"
|
278
|
+
puts "message: #{message.inspect}"
|
279
|
+
end
|
280
|
+
end
|
281
|
+
```
|
282
|
+
|
283
|
+
The `message` includes the following standard attributes:
|
284
|
+
- `message_id`: a uuid,
|
285
|
+
- `app`: which app sent this message
|
286
|
+
- `sent_at`: UTC timestamp of when the message was published
|
287
|
+
|
288
|
+
along with the values from the publish call:
|
289
|
+
- `subject`
|
290
|
+
- `action`
|
291
|
+
- `body`
|
292
|
+
|
293
|
+
The `announce` gem relies on other libraries to actually receive messages and call workers to process them.
|
294
|
+
|
295
|
+
For `shoryuken`, `subscribe_to` registers the worker for the appropriate queue, and also adds the queue to the list that will be polled by the shoryuken worker process (i.e. you don't need to add the queue to the `config/shoryuken.yml` file, or the command line call). When starting up the shoryuken process, it should start retrieving messages on the appropriate queues without further shoryuken config.
|
296
|
+
|
297
|
+
```
|
298
|
+
# use -R to load the Rails application
|
299
|
+
# use -r <path to load> to load your workers if not in a Rails app
|
300
|
+
> bundle exec shoryuken -R
|
301
|
+
```
|
34
302
|
|
35
303
|
## Development
|
36
304
|
|
305
|
+
### Developing an Adapter Class
|
306
|
+
|
307
|
+
Adapter classes should be named with the following module structure: `Announce::Adapters::SomeBrokerAdapter`
|
308
|
+
For example, the above would work with the `adapter: some_broker` option in the ActiveJob or `announce.yml` config.
|
309
|
+
|
310
|
+
There are only 2 methods required of an adapter class, `publish` and `subscribe`.
|
311
|
+
(Optionally, the `configure_broker` method can be provided for the `rake announce:configure_broker` task.)
|
312
|
+
|
313
|
+
```ruby
|
314
|
+
module Announce
|
315
|
+
module Adapters
|
316
|
+
class SomeBroker
|
317
|
+
class << self
|
318
|
+
|
319
|
+
def publish(subject, action, body, options = {})
|
320
|
+
end
|
321
|
+
|
322
|
+
def subscribe(worker_class, subject, actions = [], options = {})
|
323
|
+
end
|
324
|
+
|
325
|
+
# optional
|
326
|
+
def configure_broker(options)
|
327
|
+
end
|
328
|
+
|
329
|
+
end
|
330
|
+
end
|
331
|
+
end
|
332
|
+
end
|
333
|
+
```
|
334
|
+
|
335
|
+
There is also an abstract `BaseAdapter` which provides basic implementations of the required methods and supporting classes.
|
336
|
+
This may or may not be helpful, but is currently used by the `ShoryukenAdapter`.
|
337
|
+
|
338
|
+
### General
|
339
|
+
|
37
340
|
After checking out the repo, run `bin/setup` to install dependencies. Then, run `bin/console` for an interactive prompt that will allow you to experiment.
|
38
341
|
|
39
342
|
To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release` to create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
|
data/Rakefile
CHANGED
data/announce.gemspec
CHANGED
@@ -27,7 +27,6 @@ Gem::Specification.new do |spec|
|
|
27
27
|
spec.add_development_dependency 'guard'
|
28
28
|
spec.add_development_dependency 'guard-minitest'
|
29
29
|
spec.add_development_dependency 'dotenv'
|
30
|
-
spec.add_development_dependency 'codeclimate-test-reporter'
|
31
30
|
spec.add_development_dependency 'simplecov'
|
32
31
|
spec.add_development_dependency 'coveralls'
|
33
32
|
end
|
data/lib/announce.rb
CHANGED
@@ -6,6 +6,7 @@ require 'announce/message'
|
|
6
6
|
require 'announce/publisher'
|
7
7
|
require 'announce/subscriber'
|
8
8
|
require 'announce/version'
|
9
|
+
require 'announce/railtie' if defined?(Rails)
|
9
10
|
|
10
11
|
module Announce
|
11
12
|
class << self
|
@@ -14,6 +15,8 @@ module Announce
|
|
14
15
|
adapter_class.publish(subject, action, message, options)
|
15
16
|
end
|
16
17
|
|
18
|
+
alias announce publish
|
19
|
+
|
17
20
|
def subscribe(worker_class, subject, actions = [], options = {})
|
18
21
|
adapter_class.subscribe(worker_class, subject, actions, options)
|
19
22
|
end
|
@@ -23,7 +26,7 @@ module Announce
|
|
23
26
|
end
|
24
27
|
|
25
28
|
def options
|
26
|
-
@options ||= default_options
|
29
|
+
@options ||= Announce::Configuration.default_options
|
27
30
|
end
|
28
31
|
|
29
32
|
def configure(opts = {})
|
@@ -31,24 +34,10 @@ module Announce
|
|
31
34
|
yield @options if block_given?
|
32
35
|
end
|
33
36
|
|
34
|
-
def default_options
|
35
|
-
{}.tap do |defaults|
|
36
|
-
if defined?(ActiveJob)
|
37
|
-
defaults[:name_prefix] = ::ActiveJob::Base.queue_name_prefix
|
38
|
-
defaults[:name_delimiter] = ::ActiveJob::Base.queue_name_delimiter
|
39
|
-
defaults[:adapter] = ::ActiveJob::Base.queue_adapter
|
40
|
-
else
|
41
|
-
defaults[:name_prefix] = ENV['RAILS_ENV'] || ENV['APP_ENV'] || 'development'
|
42
|
-
defaults[:name_delimiter] = '_'
|
43
|
-
defaults[:adapter] = :inline
|
44
|
-
end
|
45
|
-
end
|
46
|
-
end
|
47
|
-
|
48
37
|
def adapter_class
|
49
|
-
|
50
|
-
require "announce/adapters/#{
|
51
|
-
"::Announce::Adapters::#{
|
38
|
+
announce_adapter = Announce.options[:adapter]
|
39
|
+
require "announce/adapters/#{announce_adapter.to_s.downcase}_adapter"
|
40
|
+
"::Announce::Adapters::#{announce_adapter.to_s.camelize}Adapter".constantize
|
52
41
|
end
|
53
42
|
|
54
43
|
def logger
|
@@ -63,7 +63,7 @@ module Announce
|
|
63
63
|
end
|
64
64
|
|
65
65
|
def self.name_for(subject, action)
|
66
|
-
[prefix, subject, action].join(delimiter)
|
66
|
+
[prefix, namespace, subject, action].join(delimiter)
|
67
67
|
end
|
68
68
|
|
69
69
|
def initialize(subject, action, options = {})
|
@@ -77,16 +77,20 @@ module Announce
|
|
77
77
|
end
|
78
78
|
|
79
79
|
def self.prefix
|
80
|
-
::Announce.options[:
|
80
|
+
::Announce.options[:queue_name_prefix]
|
81
81
|
end
|
82
82
|
|
83
83
|
def self.delimiter
|
84
|
-
::Announce.options[:
|
84
|
+
::Announce.options[:queue_name_delimiter]
|
85
85
|
end
|
86
86
|
|
87
87
|
def self.app
|
88
88
|
::Announce.options[:app_name]
|
89
89
|
end
|
90
|
+
|
91
|
+
def self.namespace
|
92
|
+
::Announce.options[:namespace]
|
93
|
+
end
|
90
94
|
end
|
91
95
|
|
92
96
|
class Topic < Destination
|
@@ -94,7 +98,7 @@ module Announce
|
|
94
98
|
|
95
99
|
class Queue < Destination
|
96
100
|
def self.name_for(subject, action)
|
97
|
-
[prefix, app, subject, action].join(delimiter)
|
101
|
+
[prefix, namespace, app, subject, action].join(delimiter)
|
98
102
|
end
|
99
103
|
end
|
100
104
|
end
|
@@ -5,41 +5,75 @@ module Announce
|
|
5
5
|
module Adapters
|
6
6
|
class ShoryukenAdapter < BaseAdapter
|
7
7
|
|
8
|
+
class AnnounceWorker #:nodoc:
|
9
|
+
include Shoryuken::Worker
|
10
|
+
|
11
|
+
shoryuken_options body_parser: :json, auto_delete: true
|
12
|
+
|
13
|
+
# overriden in register_class
|
14
|
+
def job_class
|
15
|
+
end
|
16
|
+
|
17
|
+
def perform(sqs_msg, hash)
|
18
|
+
job = job_class.new(hash)
|
19
|
+
Base.execute(job.serialize)
|
20
|
+
end
|
21
|
+
end
|
22
|
+
|
8
23
|
class Subscriber < BaseAdapter::Subscriber
|
24
|
+
|
9
25
|
def subscribe(worker_class, subject, actions, options)
|
10
26
|
Array(actions).each do |action|
|
11
27
|
queue_name = Queue.name_for(subject, action)
|
12
|
-
Shoryuken.register_worker(queue_name, worker_class
|
28
|
+
Shoryuken.register_worker(queue_name, register_class(worker_class))
|
29
|
+
Shoryuken.queues << queue_name
|
13
30
|
end
|
14
31
|
end
|
32
|
+
|
33
|
+
def register_class(worker_class)
|
34
|
+
if active_job?
|
35
|
+
Class.new(AnnounceWorker).tap do |jc|
|
36
|
+
jc.class_eval("def job_class; #{worker_class.name}; end")
|
37
|
+
end
|
38
|
+
else
|
39
|
+
worker_class
|
40
|
+
end
|
41
|
+
end
|
42
|
+
|
43
|
+
def active_job?
|
44
|
+
defined?(::ActiveJob) &&
|
45
|
+
defined?(ActiveJob::QueueAdapters::ShoryukenAdapter) &&
|
46
|
+
ActiveJob::Base.queue_adapter == ActiveJob::QueueAdapters::ShoryukenAdapter
|
47
|
+
end
|
15
48
|
end
|
16
49
|
|
17
50
|
class BrokerManager < BaseAdapter::BrokerManager
|
18
51
|
|
19
52
|
# actually configure the broker queues, topics, and subscriptions
|
20
53
|
def configure
|
21
|
-
configure_publishing
|
22
|
-
configure_subscribing
|
54
|
+
configure_publishing && configure_subscribing
|
23
55
|
end
|
24
56
|
|
25
57
|
def configure_publishing
|
26
58
|
(options[:publish] || {}).each do |subject, actions|
|
27
59
|
Array(actions).each do |action|
|
28
|
-
|
60
|
+
ShoryukenAdapter::Topic.new(subject, action, options).create
|
29
61
|
end
|
30
62
|
end
|
63
|
+
true
|
31
64
|
end
|
32
65
|
|
33
66
|
def configure_subscribing
|
34
67
|
(options[:subscribe] || {}).each do |subject, actions|
|
35
68
|
Array(actions).each do |action|
|
36
|
-
topic =
|
37
|
-
queue =
|
69
|
+
topic = ShoryukenAdapter::Topic.new(subject, action, options)
|
70
|
+
queue = ShoryukenAdapter::Queue.new(subject, action, options)
|
38
71
|
topic.create
|
39
72
|
queue.create
|
40
73
|
topic.subscribe(queue)
|
41
74
|
end
|
42
75
|
end
|
76
|
+
true
|
43
77
|
end
|
44
78
|
end
|
45
79
|
|
@@ -110,7 +144,7 @@ module Announce
|
|
110
144
|
)
|
111
145
|
|
112
146
|
attrs = sqs.get_queue_attributes(
|
113
|
-
queue_url: dlq
|
147
|
+
queue_url: dlq[:queue_url],
|
114
148
|
attribute_names: ['QueueArn']
|
115
149
|
)
|
116
150
|
|
@@ -157,58 +191,8 @@ module Announce
|
|
157
191
|
Shoryuken::Client.sqs
|
158
192
|
end
|
159
193
|
end
|
160
|
-
|
161
|
-
class AnnounceWorker #:nodoc:
|
162
|
-
include Shoryuken::Worker
|
163
|
-
|
164
|
-
attr_accessor :job_class
|
165
|
-
|
166
|
-
shoryuken_options body_parser: :json, auto_delete: true
|
167
|
-
|
168
|
-
def perform(sqs_msg, hash)
|
169
|
-
job = job_class.new(hash)
|
170
|
-
Base.execute(job.serialize)
|
171
|
-
end
|
172
|
-
end
|
173
|
-
|
174
|
-
class AnnounceWorkerRegistry < Shoryuken::DefaultWorkerRegistry
|
175
|
-
|
176
|
-
attr_accessor :subscriptions
|
177
|
-
|
178
|
-
def initialize
|
179
|
-
super
|
180
|
-
@subscribers = {}
|
181
|
-
end
|
182
|
-
|
183
|
-
def clear
|
184
|
-
super
|
185
|
-
@subscribers.clear
|
186
|
-
end
|
187
|
-
|
188
|
-
def fetch_worker(queue, message)
|
189
|
-
super.tap do |worker|
|
190
|
-
if @subscribers[queue] && worker.respond_to?('job_class=')
|
191
|
-
worker.job_class = @subscribers[queue]
|
192
|
-
end
|
193
|
-
end
|
194
|
-
end
|
195
|
-
|
196
|
-
def register_worker(queue, clazz, subscription = false)
|
197
|
-
if subscription && active_job?
|
198
|
-
@subscribers[queue] = clazz
|
199
|
-
clazz = AnnounceWorker
|
200
|
-
end
|
201
|
-
super(queue, clazz)
|
202
|
-
end
|
203
|
-
|
204
|
-
def active_job?
|
205
|
-
defined?(::ActiveJob) && ::ActiveJob::Base.queue_adapter.to_s == 'shoryuken'
|
206
|
-
end
|
207
|
-
end
|
208
|
-
|
209
194
|
end
|
210
195
|
end
|
211
196
|
end
|
212
197
|
|
213
|
-
Shoryuken.worker_registry = Announce::Adapters::ShoryukenAdapter::AnnounceWorkerRegistry.new
|
214
198
|
Shoryuken::Client.account_id = ENV['AWS_ACCOUNT_ID'] unless Shoryuken::Client.account_id
|
@@ -5,27 +5,50 @@ module Announce
|
|
5
5
|
class Configuration
|
6
6
|
attr_reader :options
|
7
7
|
|
8
|
-
def self.configure(options={})
|
8
|
+
def self.configure(options = {})
|
9
9
|
opts = new(options).configure
|
10
10
|
Announce.options.merge!(opts)
|
11
11
|
end
|
12
12
|
|
13
|
+
def self.default_options
|
14
|
+
{}.tap do |defaults|
|
15
|
+
if defined?(ActiveJob)
|
16
|
+
defaults[:queue_name_prefix] = ::ActiveJob::Base.queue_name_prefix
|
17
|
+
defaults[:queue_name_delimiter] = ::ActiveJob::Base.queue_name_delimiter
|
18
|
+
defaults[:adapter] = aj_queue_adapter_name
|
19
|
+
else
|
20
|
+
defaults[:queue_name_prefix] = ENV['RAILS_ENV'] || ENV['APP_ENV'] || 'development'
|
21
|
+
defaults[:queue_name_delimiter] = '_'
|
22
|
+
defaults[:adapter] = :inline
|
23
|
+
end
|
24
|
+
|
25
|
+
defaults[:app_name] = 'app'
|
26
|
+
defaults[:namespace] = 'announce'
|
27
|
+
end
|
28
|
+
end
|
29
|
+
|
30
|
+
def self.aj_queue_adapter_name
|
31
|
+
ajqa = ::ActiveJob::Base.queue_adapter.name
|
32
|
+
ajqa.match(/ActiveJob::QueueAdapters::(.*)Adapter/)[1].underscore
|
33
|
+
end
|
34
|
+
|
13
35
|
def initialize(options)
|
14
36
|
@options = options
|
15
37
|
base = defined?(Rails) ? Rails.root : Dir.pwd
|
16
|
-
options[:
|
38
|
+
options[:config_file] ||= File.join(base, 'config', 'announce.yml')
|
17
39
|
end
|
18
40
|
|
19
|
-
def
|
20
|
-
options[:
|
41
|
+
def config_file
|
42
|
+
options[:config_file]
|
21
43
|
end
|
22
44
|
|
23
45
|
def configure
|
24
|
-
|
25
|
-
|
46
|
+
defaults = self.class.default_options
|
47
|
+
if File.exist?(config_file)
|
48
|
+
defaults.merge(YAML.load(ERB.new(IO.read(config_file)).result).symbolize_keys)
|
26
49
|
else
|
27
|
-
Announce.logger.warn "PubSub file #{
|
28
|
-
|
50
|
+
Announce.logger.warn "PubSub file #{config_file} does not exist"
|
51
|
+
defaults
|
29
52
|
end
|
30
53
|
end
|
31
54
|
end
|
data/lib/announce/core_ext.rb
CHANGED
@@ -49,9 +49,7 @@ rescue LoadError
|
|
49
49
|
end
|
50
50
|
constant
|
51
51
|
end
|
52
|
-
end if !"".respond_to?(:constantize)
|
53
52
|
|
54
|
-
class String
|
55
53
|
def camelize
|
56
54
|
string = self
|
57
55
|
string = string.sub(/^[a-z\d]*/) { $&.capitalize }
|
@@ -59,5 +57,16 @@ rescue LoadError
|
|
59
57
|
string.gsub!(/\//, '::')
|
60
58
|
string
|
61
59
|
end
|
62
|
-
|
60
|
+
|
61
|
+
def underscore
|
62
|
+
camel_cased_word = self
|
63
|
+
return camel_cased_word unless camel_cased_word =~ /[A-Z-]|::/
|
64
|
+
word = camel_cased_word.to_s.gsub(/::/, '/')
|
65
|
+
word.gsub!(/([A-Z\d]+)([A-Z][a-z])/,'\1_\2')
|
66
|
+
word.gsub!(/([a-z\d])([A-Z])/,'\1_\2')
|
67
|
+
word.tr!("-", "_")
|
68
|
+
word.downcase!
|
69
|
+
word
|
70
|
+
end
|
71
|
+
end if !"".respond_to?(:constantize)
|
63
72
|
end
|
data/lib/announce/publisher.rb
CHANGED
data/lib/announce/subscriber.rb
CHANGED
data/lib/announce/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: announce
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.2.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Andrew Kuklewicz
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2015-05-
|
11
|
+
date: 2015-05-15 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: shoryuken
|
@@ -122,20 +122,6 @@ dependencies:
|
|
122
122
|
- - ">="
|
123
123
|
- !ruby/object:Gem::Version
|
124
124
|
version: '0'
|
125
|
-
- !ruby/object:Gem::Dependency
|
126
|
-
name: codeclimate-test-reporter
|
127
|
-
requirement: !ruby/object:Gem::Requirement
|
128
|
-
requirements:
|
129
|
-
- - ">="
|
130
|
-
- !ruby/object:Gem::Version
|
131
|
-
version: '0'
|
132
|
-
type: :development
|
133
|
-
prerelease: false
|
134
|
-
version_requirements: !ruby/object:Gem::Requirement
|
135
|
-
requirements:
|
136
|
-
- - ">="
|
137
|
-
- !ruby/object:Gem::Version
|
138
|
-
version: '0'
|
139
125
|
- !ruby/object:Gem::Dependency
|
140
126
|
name: simplecov
|
141
127
|
requirement: !ruby/object:Gem::Requirement
|
@@ -193,8 +179,10 @@ files:
|
|
193
179
|
- lib/announce/core_ext.rb
|
194
180
|
- lib/announce/message.rb
|
195
181
|
- lib/announce/publisher.rb
|
182
|
+
- lib/announce/railtie.rb
|
196
183
|
- lib/announce/subscriber.rb
|
197
184
|
- lib/announce/version.rb
|
185
|
+
- lib/tasks/announce.rake
|
198
186
|
homepage: https://github.com/PRX/announce
|
199
187
|
licenses:
|
200
188
|
- MIT
|