fairway 0.0.8 → 0.0.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/README.markdown +86 -62
- data/lib/fairway/sidekiq/fetch.rb +1 -1
- data/lib/fairway/version.rb +1 -1
- data/spec/lib/fairway/fetch_spec.rb +16 -0
- metadata +2 -2
data/README.markdown
CHANGED
@@ -1,4 +1,4 @@
|
|
1
|
-
# Fairway - a fair way to queue messages in multi-user systems
|
1
|
+
# Fairway - a fair way to queue messages in multi-user systems
|
2
2
|
|
3
3
|
## Installation
|
4
4
|
|
@@ -10,17 +10,19 @@ Then make sure you `bundle install`.
|
|
10
10
|
|
11
11
|
## Configuration
|
12
12
|
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
|
22
|
-
|
23
|
-
|
13
|
+
```ruby
|
14
|
+
Fairway.configure do |config|
|
15
|
+
config.redis = { host: "localhost", port: 6379 }
|
16
|
+
config.namespace = "fairway"
|
17
|
+
|
18
|
+
config.facet do |message|
|
19
|
+
message[:user]
|
20
|
+
end
|
21
|
+
|
22
|
+
config.register_queue("all_messages")
|
23
|
+
end
|
24
|
+
```
|
25
|
+
|
24
26
|
## What's a facet?
|
25
27
|
|
26
28
|
In many queuing systems, if a single user manages to queue up a lot of messages/jobs at once,
|
@@ -33,11 +35,13 @@ additional messages from a given facet.
|
|
33
35
|
|
34
36
|
You can define how to facet your messages during configuration:
|
35
37
|
|
36
|
-
|
37
|
-
|
38
|
-
|
39
|
-
|
40
|
-
|
38
|
+
```ruby
|
39
|
+
Fairway.configure do |config|
|
40
|
+
config.facet do |message|
|
41
|
+
message[:user]
|
42
|
+
end
|
43
|
+
end
|
44
|
+
```
|
41
45
|
|
42
46
|
Now, any message delivered by fairway, will use the `user` key of the message to determine
|
43
47
|
which facet to use.
|
@@ -52,22 +56,28 @@ and each queue will receive delivered messages.
|
|
52
56
|
|
53
57
|
Registering a queue is part of your fairway configuration:
|
54
58
|
|
55
|
-
|
56
|
-
|
57
|
-
|
58
|
-
|
59
|
+
```ruby
|
60
|
+
Fairway.configure do |config|
|
61
|
+
config.register_queue("myqueue")
|
62
|
+
config.register_queue("yourqueue")
|
63
|
+
end
|
64
|
+
```
|
59
65
|
|
60
66
|
After configuring your queues, just create a fairway connection,
|
61
67
|
and it'll handle persisting your queues in Redis:
|
62
68
|
|
63
|
-
|
69
|
+
```ruby
|
70
|
+
connection = Fairway::Connection.new
|
71
|
+
```
|
64
72
|
|
65
73
|
## Delivering messages
|
66
74
|
|
67
75
|
To add messages to your queues, you deliver them:
|
68
76
|
|
69
|
-
|
70
|
-
|
77
|
+
```ruby
|
78
|
+
connection = Fairway::Connection.new
|
79
|
+
connection.deliver(type: "invite_friends", user: "bob", friends: ["nancy", "john"])
|
80
|
+
```
|
71
81
|
|
72
82
|
Now, any registered queues will receive this message, faceted if you've defined
|
73
83
|
a facet in your configuration.
|
@@ -76,9 +86,11 @@ a facet in your configuration.
|
|
76
86
|
|
77
87
|
Once you have messages on a queue, you can pull them off and process them:
|
78
88
|
|
79
|
-
|
80
|
-
|
81
|
-
|
89
|
+
```ruby
|
90
|
+
connection = Fairway::Connection.new
|
91
|
+
queue = Fairway::Queue.new(connection, "myqueue")
|
92
|
+
message = queue.pull
|
93
|
+
```
|
82
94
|
|
83
95
|
Behind the scenes, fairway uses a round-robin strategy to ensure equal weighting of
|
84
96
|
any facets which contain messages.
|
@@ -94,24 +106,30 @@ You can accomplish this with message channels. By default, all messages use the
|
|
94
106
|
channel. You can customize this by creating a `Fairway::ChanneledConnection` and
|
95
107
|
a block which defines the channel for a given message:
|
96
108
|
|
97
|
-
|
98
|
-
|
99
|
-
|
100
|
-
|
109
|
+
```ruby
|
110
|
+
conn = Fairway::Connection.new
|
111
|
+
conn = Fairway::ChanneledConnection.new(conn) do |message|
|
112
|
+
message[:type]
|
113
|
+
end
|
114
|
+
```
|
101
115
|
|
102
116
|
You can also register queues for a channel:
|
103
117
|
|
104
|
-
|
105
|
-
|
106
|
-
|
107
|
-
|
118
|
+
```ruby
|
119
|
+
Fairway.configure do |config|
|
120
|
+
config.register_queue("invite_queue", "invite_friends")
|
121
|
+
end
|
122
|
+
```
|
123
|
+
|
108
124
|
Now, your queue will only receive messages which have the channel `invite_friends`.
|
109
125
|
|
110
126
|
If you'd like to receive messages with channels that match a pattern:
|
111
127
|
|
112
|
-
|
113
|
-
|
114
|
-
|
128
|
+
```ruby
|
129
|
+
Fairway.configure do |config|
|
130
|
+
config.register_queue("invite_queue", "invite_.*")
|
131
|
+
end
|
132
|
+
```
|
115
133
|
|
116
134
|
Now, messages from the channels `invite_friends`, `invite_pets`, `invite_parents` will
|
117
135
|
be delivered to the `invite_queue`.
|
@@ -120,41 +138,47 @@ be delivered to the `invite_queue`.
|
|
120
138
|
|
121
139
|
To listen for messages without the overhead of queuing them, you can subscribe:
|
122
140
|
|
123
|
-
|
141
|
+
```ruby
|
142
|
+
connection = Fairway::Connection.new
|
124
143
|
|
125
|
-
|
126
|
-
|
127
|
-
|
144
|
+
connection.subscribe do |message|
|
145
|
+
# Do something with each message
|
146
|
+
end
|
147
|
+
```
|
128
148
|
|
129
149
|
If you'd like to only receive some messages, you can subscribe to just a particular channel:
|
130
150
|
|
131
|
-
|
151
|
+
```ruby
|
152
|
+
connection = Fairway::Connection.new
|
132
153
|
|
133
|
-
|
134
|
-
|
135
|
-
|
136
|
-
|
154
|
+
connection.subscribe("invite_*") do |message|
|
155
|
+
# Do something with each message which
|
156
|
+
# has a channel matching "invite_*"
|
157
|
+
end
|
158
|
+
```
|
137
159
|
|
138
160
|
## Fairway and Sidekiq
|
139
161
|
|
140
162
|
Fairway isn't meant to be a robust system for processing queued messages/jobs. To more reliably
|
141
163
|
process queued messages, we've integrated with [Sidekiq](http://sidekiq.org/).
|
142
164
|
|
143
|
-
|
144
|
-
|
145
|
-
|
146
|
-
|
147
|
-
|
148
|
-
|
149
|
-
|
150
|
-
|
151
|
-
|
152
|
-
|
153
|
-
|
154
|
-
|
155
|
-
|
156
|
-
|
157
|
-
|
165
|
+
```ruby
|
166
|
+
require 'fairway/sidekiq'
|
167
|
+
|
168
|
+
connection = Fairway::Connection.new
|
169
|
+
queues = Fairway::Queue.new(connection, "myqueue", "yourqueue")
|
170
|
+
|
171
|
+
Sidekiq.options[:fetch] = Fairway::Sidekiq::Fetch.new do |fetch|
|
172
|
+
fetch.from :sidekiq, 2
|
173
|
+
fetch.from queues, 1 do |queue, message|
|
174
|
+
# translate message to normalized Sidekiq job, if needed
|
175
|
+
{ "queue" => "fairway",
|
176
|
+
"class" => "FairwayMessageJob",
|
177
|
+
"args" => [message],
|
178
|
+
"retry" => true }
|
179
|
+
end
|
180
|
+
end
|
181
|
+
```
|
158
182
|
|
159
183
|
`fetch.from :sidekiq, 2` will fetch from sidekiq queues you have defined through the
|
160
184
|
normal sidekiq configuration.
|
data/lib/fairway/version.rb
CHANGED
@@ -46,6 +46,22 @@ module Fairway::Sidekiq
|
|
46
46
|
end
|
47
47
|
end
|
48
48
|
|
49
|
+
describe "#bulk_requeue" do
|
50
|
+
it "requeues jobs to redis" do
|
51
|
+
uow = Sidekiq::BasicFetch::UnitOfWork
|
52
|
+
q1 = Sidekiq::Queue.new('foo')
|
53
|
+
q2 = Sidekiq::Queue.new('bar')
|
54
|
+
|
55
|
+
q1.size.should == 0
|
56
|
+
q2.size.should == 0
|
57
|
+
|
58
|
+
Fetch.bulk_requeue([uow.new('queue:foo', 'bob'), uow.new('queue:foo', 'bar'), uow.new('queue:bar', 'widget')])
|
59
|
+
|
60
|
+
q1.size.should == 2
|
61
|
+
q2.size.should == 1
|
62
|
+
end
|
63
|
+
end
|
64
|
+
|
49
65
|
describe "#fetch_order" do
|
50
66
|
let(:fetch) { Fetch.new { |f| f.from :fetchA, 10; f.from :fetchB, 1 } }
|
51
67
|
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: fairway
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.0.
|
4
|
+
version: 0.0.9
|
5
5
|
prerelease:
|
6
6
|
platform: ruby
|
7
7
|
authors:
|
@@ -9,7 +9,7 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date: 2013-
|
12
|
+
date: 2013-04-08 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: activesupport
|