chasqui 0.9.2 → 0.9.3
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +67 -87
- data/lib/chasqui.rb +2 -1
- data/lib/chasqui/subscription.rb +9 -2
- data/lib/chasqui/version.rb +1 -1
- data/spec/integration/setup/resque.rb +1 -2
- data/spec/integration/setup/sidekiq.rb +0 -2
- data/spec/integration/setup/subscribers.rb +2 -2
- data/spec/lib/chasqui/multi_broker_spec.rb +3 -3
- data/spec/lib/chasqui/subscription_spec.rb +3 -0
- data/spec/lib/chasqui_spec.rb +21 -10
- data/spec/support/chasqui_spec_helpers.rb +1 -1
- metadata +2 -3
- data/examples/full.rb +0 -56
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: c7b83716114be3ede50a3559cb742d776cdecd1f
|
4
|
+
data.tar.gz: df453dfaa501a592ddc6b3c650fcae09a9343123
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 91b367f3cd65f4bceffd73ebbdd6ef9d9af344741d797ebb8909d7b372c8246c99cee930b3b0a3cf09f316cbd48e84f11f82cfcd16f2251acdaa63c089fba7be
|
7
|
+
data.tar.gz: 420990d6feaa68f81d4db1ec6076a0c2b8fa2e5bd137dcf41a222d89734fa8e560e5d2e8cbb07a455d22a827e020c43c744f886e78a6f17e3f60c9e255f73f55
|
data/README.md
CHANGED
@@ -9,131 +9,111 @@ messaging pattern for service oriented architectures.
|
|
9
9
|
Chasqui delivers messages to subscribers in a Resque-compatible format. If you are already
|
10
10
|
using Resque and/or Sidekiq, Chasqui will make a wonderful companion to your architecture.
|
11
11
|
|
12
|
-
##
|
12
|
+
## Installation
|
13
13
|
|
14
|
-
|
15
|
-
* To process, monitor, and retry messages idependent of request cycles
|
14
|
+
Add this line to your application's Gemfile:
|
16
15
|
|
17
|
-
|
16
|
+
gem 'chasqui'
|
18
17
|
|
19
|
-
|
20
|
-
Failure is expected and planned for in the design.
|
18
|
+
And then execute:
|
21
19
|
|
22
|
-
|
23
|
-
because it cannot ensure delivery of messages, especially when subscribers are not running.
|
20
|
+
$ bundle
|
24
21
|
|
25
|
-
|
26
|
-
The client provides a simple interface for publishing messages. The client places messages
|
27
|
-
on a persistent queue, called an inbox, where they wait for further processing.
|
28
|
-
The server reads messages from the inbox and places them on the queues of each subscriber.
|
29
|
-
The server will create queues for subscribers if the queues do not exist.
|
22
|
+
Or install it yourself as:
|
30
23
|
|
31
|
-
|
32
|
-
chasqui subscribers are not running.
|
24
|
+
$ gem install chasqui
|
33
25
|
|
34
|
-
##
|
26
|
+
## Dependencies
|
35
27
|
|
36
|
-
Chasqui
|
37
|
-
|
28
|
+
Chasqui uses Redis to queue events and manage subscriptions. You can install
|
29
|
+
redis with your favorite package manager, such as homebrew, yum, or apt, or if
|
30
|
+
you prefer, you can run `vagrant up` to run Redis in a virtual machine.
|
38
31
|
|
39
|
-
|
40
|
-
2. You primarily use resque and/or sidekiq to process jobs already.
|
41
|
-
3. You want a simple Pub/Sub solution with minimal setup and maintenance.
|
42
|
-
4. Your organization or traffic volume is not high enough to warrant a more complex solution.
|
32
|
+
## Quick Start
|
43
33
|
|
44
|
-
|
45
|
-
|
34
|
+
Chasqui consistents of two components - a client and a broker. The broker's
|
35
|
+
responsibility is to forward published events to registered subscribers. The
|
36
|
+
client can both publish events and register subscribers.
|
46
37
|
|
47
|
-
|
48
|
-
solve a specific problem for a particular scale and company size.
|
38
|
+
### Start the broker
|
49
39
|
|
50
|
-
|
40
|
+
chasqui -r redis://localhost:6379/0 -q my-app
|
51
41
|
|
52
|
-
|
42
|
+
Your broker must use the same redis connection as your sidekiq (or resque)
|
43
|
+
workers. For a list of available broker options, run `chasqui --help`.
|
53
44
|
|
54
|
-
|
55
|
-
gem 'chasqui'
|
56
|
-
```
|
45
|
+
### Publish events
|
57
46
|
|
58
|
-
|
47
|
+
Publishing events is simple.
|
59
48
|
|
60
|
-
|
49
|
+
# file: publisher.rb
|
50
|
+
require 'chasqui'
|
51
|
+
Chasqui.publish 'user.sign-up', 'Luke Skywalker'
|
52
|
+
Chasqui.publish 'user.cancel', 'Dart Vader', 'invalid use of the force'
|
61
53
|
|
62
|
-
|
54
|
+
Be sure to run the publisher, broker, and subscribers in separate terminal
|
55
|
+
windows.
|
63
56
|
|
64
|
-
|
57
|
+
ruby publisher.rb
|
65
58
|
|
66
|
-
|
59
|
+
### Subscribe to events
|
67
60
|
|
68
|
-
|
61
|
+
Subscribing to events is also simple. The following example tells chasqui to
|
62
|
+
forward events to the subscriber's 'my-app' queue, for which chasqui will
|
63
|
+
generate the appropriate worker class. Within the subscriber block, you define
|
64
|
+
one or more `on` blocks in which you place your application logic for handling
|
65
|
+
an event.
|
69
66
|
|
70
|
-
|
71
|
-
|
72
|
-
```
|
67
|
+
# file: subscriber1.rb
|
68
|
+
require 'chasqui'
|
73
69
|
|
74
|
-
|
70
|
+
Chasqui.subscribe queue: 'my-app' do
|
75
71
|
|
76
|
-
|
77
|
-
#
|
78
|
-
|
79
|
-
config.channel = 'com.example.myapp'
|
80
|
-
end
|
81
|
-
```
|
72
|
+
on 'user.sign-up' do |user_id|
|
73
|
+
# do something when the user signs up
|
74
|
+
end
|
82
75
|
|
83
|
-
|
84
|
-
|
76
|
+
on 'user.cancel' do |user_id, reason|
|
77
|
+
# do something else when user cancels
|
78
|
+
end
|
85
79
|
|
86
|
-
|
80
|
+
end
|
87
81
|
|
88
|
-
|
89
|
-
|
90
|
-
Chasqui.subscribe 'com.example.myapp', queue: 'unique_queue_name_for_app' do
|
82
|
+
You can have as many subscribers as you like, but __each subscriber must have
|
83
|
+
its own unique queue name__.
|
91
84
|
|
92
|
-
|
93
|
-
user = User.find user_id
|
94
|
-
UserMailer.signup(user).deliver
|
95
|
-
end
|
85
|
+
Here is how you can run the subscriber as a sidekiq worker:
|
96
86
|
|
97
|
-
|
98
|
-
user = User.find user_id
|
99
|
-
AdminMailer.user_cancelled(user, reason).deliver
|
100
|
-
user.archive!
|
101
|
-
end
|
87
|
+
sidekiq -r subscriber.rb
|
102
88
|
|
103
|
-
|
104
|
-
```
|
89
|
+
To run the resque worker, you first need to create a Rakefile.
|
105
90
|
|
106
|
-
|
91
|
+
# Rakefile
|
92
|
+
require 'resque'
|
93
|
+
require 'resque/tasks'
|
107
94
|
|
108
|
-
|
109
|
-
Chasqui.configure do |config|
|
110
|
-
config.channel = 'com.example.transcoder'
|
111
|
-
config.redis = ENV.fetch('REDIS_URL')
|
112
|
-
config.workers = :sidekiq # or :resque
|
113
|
-
...
|
114
|
-
end
|
115
|
-
```
|
95
|
+
task 'resque:setup' => ['chasqui:subscriber']
|
116
96
|
|
117
|
-
|
97
|
+
namespace :chasqui do
|
98
|
+
task :subscriber do
|
99
|
+
require './subscriber.rb'
|
100
|
+
end
|
101
|
+
end
|
118
102
|
|
119
|
-
|
103
|
+
Then you can run the resque worker to start processing events.
|
120
104
|
|
121
|
-
|
122
|
-
# install required dependencies
|
123
|
-
$ sudo apt-get install ruby redis-server
|
105
|
+
rake resque:work
|
124
106
|
|
125
|
-
|
126
|
-
$ sudo gem install chasqui --no-ri --no-rdoc
|
107
|
+
## Why Chasqui?
|
127
108
|
|
128
|
-
|
129
|
-
|
130
|
-
|
131
|
-
I, [2015-09-23T00:58:16.631147 #2420] INFO -- chasqui: configured to fetch events from inbox on #<Redis client v3.2.1 for redis://127.0.0.1:6379/0>
|
132
|
-
```
|
109
|
+
* Reduces coupling between applications
|
110
|
+
* Integrates with the popular sidekiq and resque background worker libraries
|
111
|
+
* Queues events for registered subscribers even if a subscriber is unavailable
|
133
112
|
|
134
|
-
|
113
|
+
## Limitations
|
135
114
|
|
136
|
-
|
115
|
+
In order for chasqui to work properly, the publisher, broker, and all
|
116
|
+
subscribers must connect to the same Redis database.
|
137
117
|
|
138
118
|
## Contributing
|
139
119
|
|
data/lib/chasqui.rb
CHANGED
@@ -32,8 +32,9 @@ module Chasqui
|
|
32
32
|
redis.lpush inbox_queue, build_payload(event, *args).to_json
|
33
33
|
end
|
34
34
|
|
35
|
-
def subscribe(
|
35
|
+
def subscribe(options={}, &block)
|
36
36
|
queue = options.fetch :queue
|
37
|
+
channel = options.fetch :channel, config.channel
|
37
38
|
|
38
39
|
create_subscription(queue, channel).tap do |subscription|
|
39
40
|
subscription.subscriber.evaluate(&block) if block_given?
|
data/lib/chasqui/subscription.rb
CHANGED
@@ -18,7 +18,6 @@ module Chasqui
|
|
18
18
|
end
|
19
19
|
|
20
20
|
def worker
|
21
|
-
# TODO How can we store this in an instance variable without breaking things?
|
22
21
|
case worker_backend
|
23
22
|
when :resque
|
24
23
|
Chasqui::ResqueWorker.create subscriber
|
@@ -33,7 +32,15 @@ module Chasqui
|
|
33
32
|
private
|
34
33
|
|
35
34
|
def worker_backend
|
36
|
-
Chasqui.config.worker_backend
|
35
|
+
Chasqui.config.worker_backend || find_first_available_worker_backend
|
36
|
+
end
|
37
|
+
|
38
|
+
def find_first_available_worker_backend
|
39
|
+
if Object.const_defined? :Sidekiq
|
40
|
+
:sidekiq
|
41
|
+
elsif Object.const_defined? :Resque
|
42
|
+
:resque
|
43
|
+
end
|
37
44
|
end
|
38
45
|
|
39
46
|
SUPPORTED_WORKER_BACKENDS = [:resque, :sidekiq].freeze
|
data/lib/chasqui/version.rb
CHANGED
@@ -4,11 +4,11 @@ def log_event subscriber, args
|
|
4
4
|
subscriber.redis.rpush "#{subscriber.queue}:event_log", payload
|
5
5
|
end
|
6
6
|
|
7
|
-
Chasqui.subscribe 'integration', queue: 'app1' do
|
7
|
+
Chasqui.subscribe channel: 'integration', queue: 'app1' do
|
8
8
|
on('user.signup') { |*args| log_event self, args }
|
9
9
|
end
|
10
10
|
|
11
|
-
Chasqui.subscribe 'integration', queue: 'app2' do
|
11
|
+
Chasqui.subscribe channel: 'integration', queue: 'app2' do
|
12
12
|
on('account.*') { |*args| log_event self, args }
|
13
13
|
on('user.cancel') { |*args| log_event self, args }
|
14
14
|
end
|
@@ -13,9 +13,9 @@ describe Chasqui::MultiBroker do
|
|
13
13
|
describe '#forward_event' do
|
14
14
|
before do
|
15
15
|
Chasqui.config.channel = 'app1'
|
16
|
-
Chasqui.subscribe 'app1', queue: 'queue1'
|
17
|
-
Chasqui.subscribe 'app2', queue: 'queue2'
|
18
|
-
Chasqui.subscribe 'app1', queue: 'queue3'
|
16
|
+
Chasqui.subscribe channel: 'app1', queue: 'queue1'
|
17
|
+
Chasqui.subscribe channel: 'app2', queue: 'queue2'
|
18
|
+
Chasqui.subscribe channel: 'app1', queue: 'queue3'
|
19
19
|
end
|
20
20
|
|
21
21
|
it 'places the event on all subscriber queues' do
|
@@ -9,6 +9,9 @@ describe Chasqui::Subscription do
|
|
9
9
|
describe '#worker' do
|
10
10
|
it 'raises when no worker backend configured' do
|
11
11
|
Chasqui.config.worker_backend = nil
|
12
|
+
allow_any_instance_of(Chasqui::Subscription).to receive(
|
13
|
+
:worker_backend).and_return(nil)
|
14
|
+
|
12
15
|
expect(-> {
|
13
16
|
new_subscription.worker
|
14
17
|
}).to raise_error(Chasqui::ConfigurationError)
|
data/spec/lib/chasqui_spec.rb
CHANGED
@@ -134,16 +134,27 @@ describe Chasqui do
|
|
134
134
|
describe '.subscribe' do
|
135
135
|
before do
|
136
136
|
reset_chasqui
|
137
|
+
Resque.redis.namespace = :resque
|
137
138
|
Chasqui.config.worker_backend = :resque
|
138
139
|
end
|
139
140
|
|
141
|
+
context 'with defaults' do
|
142
|
+
it 'subscribes to events on the default channel' do
|
143
|
+
sub = Chasqui.subscribe queue: 'my-queue'
|
144
|
+
|
145
|
+
channel = Chasqui.config.channel
|
146
|
+
queues = Chasqui.redis.smembers Chasqui.subscription_key(channel)
|
147
|
+
expect(queues).to eq(['resque/resque:queue:my-queue'])
|
148
|
+
end
|
149
|
+
end
|
150
|
+
|
140
151
|
context 'resque worker subscriptions' do
|
141
152
|
before { Resque.redis.namespace = 'blah' }
|
142
153
|
|
143
154
|
it 'creates subscriptions using the appropriate redis namespace' do
|
144
|
-
sub1 = Chasqui.subscribe 'com.example.admin', queue: 'app1-queue'
|
145
|
-
sub2 = Chasqui.subscribe 'com.example.admin', queue: 'app2-queue'
|
146
|
-
sub3 = Chasqui.subscribe 'com.example.video', queue: 'app1-queue'
|
155
|
+
sub1 = Chasqui.subscribe channel: 'com.example.admin', queue: 'app1-queue'
|
156
|
+
sub2 = Chasqui.subscribe channel: 'com.example.admin', queue: 'app2-queue'
|
157
|
+
sub3 = Chasqui.subscribe channel: 'com.example.video', queue: 'app1-queue'
|
147
158
|
|
148
159
|
queues = Chasqui.redis.smembers Chasqui.subscription_key("com.example.admin")
|
149
160
|
expect(queues.sort).to eq(['resque/blah:queue:app1-queue', 'resque/blah:queue:app2-queue'])
|
@@ -164,12 +175,12 @@ describe Chasqui do
|
|
164
175
|
end
|
165
176
|
|
166
177
|
it 'creates subscriptions using the appropriate redis namespace' do
|
167
|
-
Chasqui.subscribe 'com.example.admin', queue: 'app1-queue'
|
178
|
+
Chasqui.subscribe channel: 'com.example.admin', queue: 'app1-queue'
|
168
179
|
queues = Chasqui.redis.smembers Chasqui.subscription_key("com.example.admin")
|
169
180
|
expect(queues.sort).to eq(['sidekiq/queue:app1-queue'])
|
170
181
|
|
171
182
|
Sidekiq.redis = { url: redis.client.options[:url], namespace: 'foobar' }
|
172
|
-
Chasqui.subscribe 'com.example.video', queue: 'app2-queue'
|
183
|
+
Chasqui.subscribe channel: 'com.example.video', queue: 'app2-queue'
|
173
184
|
queues = Chasqui.redis.smembers Chasqui.subscription_key("com.example.video")
|
174
185
|
expect(queues.sort).to eq(['sidekiq/foobar:queue:app2-queue'])
|
175
186
|
end
|
@@ -177,13 +188,13 @@ describe Chasqui do
|
|
177
188
|
end
|
178
189
|
|
179
190
|
it 'returns a subscription' do
|
180
|
-
subscription = Chasqui.subscribe 'com.example.admin', queue: 'app1-queue'
|
191
|
+
subscription = Chasqui.subscribe channel: 'com.example.admin', queue: 'app1-queue'
|
181
192
|
expect(subscription.subscriber).to be_kind_of(Chasqui::Subscriber)
|
182
193
|
end
|
183
194
|
|
184
195
|
it 'yields a subscriber configuration context' do
|
185
196
|
$context = nil
|
186
|
-
Chasqui.subscribe 'bar', queue: 'foo' do
|
197
|
+
Chasqui.subscribe channel: 'bar', queue: 'foo' do
|
187
198
|
$context = self
|
188
199
|
end
|
189
200
|
expect($context).to be_kind_of(Chasqui::Subscriber)
|
@@ -195,9 +206,9 @@ describe Chasqui do
|
|
195
206
|
reset_chasqui
|
196
207
|
Chasqui.config.worker_backend = :resque
|
197
208
|
Resque.redis.namespace = 'ns0'
|
198
|
-
Chasqui.subscribe 'com.example.admin', queue: 'app1-queue'
|
199
|
-
Chasqui.subscribe 'com.example.admin', queue: 'app2-queue'
|
200
|
-
Chasqui.subscribe 'com.example.video', queue: 'app1-queue'
|
209
|
+
Chasqui.subscribe channel: 'com.example.admin', queue: 'app1-queue'
|
210
|
+
Chasqui.subscribe channel: 'com.example.admin', queue: 'app2-queue'
|
211
|
+
Chasqui.subscribe channel: 'com.example.video', queue: 'app1-queue'
|
201
212
|
end
|
202
213
|
|
203
214
|
it 'removes the subscription' do
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: chasqui
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.9.
|
4
|
+
version: 0.9.3
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Jordan Bach
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2015-11-
|
11
|
+
date: 2015-11-16 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: redis
|
@@ -156,7 +156,6 @@ files:
|
|
156
156
|
- bin/console
|
157
157
|
- chasqui.gemspec
|
158
158
|
- code-of-conduct.md
|
159
|
-
- examples/full.rb
|
160
159
|
- examples/upstart.conf
|
161
160
|
- lib/chasqui.rb
|
162
161
|
- lib/chasqui/broker.rb
|
data/examples/full.rb
DELETED
@@ -1,56 +0,0 @@
|
|
1
|
-
# file: admin/config/initializers/chasqui.rb
|
2
|
-
Chasqui.configure do |config|
|
3
|
-
config.channel = 'com.example.admin'
|
4
|
-
config.redis = ENV.fetch('REDIS_URL')
|
5
|
-
end
|
6
|
-
|
7
|
-
# file: admin/app/controllers/video_controller.rb
|
8
|
-
class VideosController < ApplicationController
|
9
|
-
|
10
|
-
def upload
|
11
|
-
video = Video.find params[:id]
|
12
|
-
|
13
|
-
if video.upload params[:file]
|
14
|
-
Chasqui.publish 'video.upload', video.id
|
15
|
-
redirect_to upload_complete_url,
|
16
|
-
else
|
17
|
-
redirect_to video_url, alert: "Upload failed."
|
18
|
-
end
|
19
|
-
end
|
20
|
-
|
21
|
-
end
|
22
|
-
|
23
|
-
# file: transcoder/config/initializers/chasqui.rb
|
24
|
-
Chasqui.configure do |config|
|
25
|
-
config.publish 'com.example.transcoder'
|
26
|
-
config.redis ENV.fetch('REDIS_URL')
|
27
|
-
config.workers :sidekiq # or :resque
|
28
|
-
end
|
29
|
-
|
30
|
-
# file: transcoder/app/subscribers/video_subscriber.rb
|
31
|
-
Chasqui.subscribe 'com.example.admin', queue: 'transcoder.video' do
|
32
|
-
on 'video.upload' do |video_id|
|
33
|
-
begin
|
34
|
-
Transcorder.transcode video_url(video_id)
|
35
|
-
Chasqui.publish 'video.complete', video_id
|
36
|
-
rescue => ex
|
37
|
-
Chasqui.publish 'video.error', video_id, ex.message
|
38
|
-
raise
|
39
|
-
end
|
40
|
-
end
|
41
|
-
end
|
42
|
-
|
43
|
-
# file: admin/app/subscribers/video_subscriber.rb
|
44
|
-
Chasqui.subscribe 'com.example.transcoder', queue: 'admin.events' do
|
45
|
-
|
46
|
-
on 'transcoder.video.complete' do |video_id|
|
47
|
-
video = Video.find video_id
|
48
|
-
VideoMailer.transcode_complete(video).deliver
|
49
|
-
end
|
50
|
-
|
51
|
-
on 'transcoder.video.error' do |video_id, error|
|
52
|
-
video = Video.find video_id
|
53
|
-
VideoMailer.transcode_error(video, error).deliver
|
54
|
-
end
|
55
|
-
|
56
|
-
end
|