redis-call 1.0.2

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: 01b31dcdbab4ab0b47234940ea7a40c1e5cc0af6
4
+ data.tar.gz: db732f64957396a7e855d06756c2c3f51b37bc5b
5
+ SHA512:
6
+ metadata.gz: fa18c36ca2d0f835449636028bb54213c225d2c6627abe5c3af08b4f8d225123060351ce760423be9103a031dd19759a16a772c918570a298e3d34c2f6e8ed16
7
+ data.tar.gz: b64d0a6567542760223a86557f7ee7c680f471e49796be777ea8f4d2f5f8199d751daf5417427a4d6fbfb1620876c731e84fa66a6505b4b0eafea286786357b3
data/CONTRIBUTING.md ADDED
@@ -0,0 +1,5 @@
1
+
2
+ We require our contributors to sign a Contributor License Agreement before
3
+ any contribution can be considered, reviewed, or accepted.
4
+
5
+ Please see https://contributors.senotrusov.com/ for details.
data/ISSUE_TEMPLATE.md ADDED
@@ -0,0 +1,5 @@
1
+ <!--
2
+ Please use the "x" character to declare that you agree with the statement below. Otherwise, remove the said statement and mark your submission as "Not a Contribution.".
3
+ -->
4
+
5
+ [ ] I agree that this is the Contribution as defined in Apache License, Version 2.0 (the "License") and I am submitting this Contribution in compliance with the License. A copy of the License can be obtained at http://www.apache.org/licenses/LICENSE-2
data/LICENSE ADDED
@@ -0,0 +1,202 @@
1
+
2
+ Apache License
3
+ Version 2.0, January 2004
4
+ http://www.apache.org/licenses/
5
+
6
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
7
+
8
+ 1. Definitions.
9
+
10
+ "License" shall mean the terms and conditions for use, reproduction,
11
+ and distribution as defined by Sections 1 through 9 of this document.
12
+
13
+ "Licensor" shall mean the copyright owner or entity authorized by
14
+ the copyright owner that is granting the License.
15
+
16
+ "Legal Entity" shall mean the union of the acting entity and all
17
+ other entities that control, are controlled by, or are under common
18
+ control with that entity. For the purposes of this definition,
19
+ "control" means (i) the power, direct or indirect, to cause the
20
+ direction or management of such entity, whether by contract or
21
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
22
+ outstanding shares, or (iii) beneficial ownership of such entity.
23
+
24
+ "You" (or "Your") shall mean an individual or Legal Entity
25
+ exercising permissions granted by this License.
26
+
27
+ "Source" form shall mean the preferred form for making modifications,
28
+ including but not limited to software source code, documentation
29
+ source, and configuration files.
30
+
31
+ "Object" form shall mean any form resulting from mechanical
32
+ transformation or translation of a Source form, including but
33
+ not limited to compiled object code, generated documentation,
34
+ and conversions to other media types.
35
+
36
+ "Work" shall mean the work of authorship, whether in Source or
37
+ Object form, made available under the License, as indicated by a
38
+ copyright notice that is included in or attached to the work
39
+ (an example is provided in the Appendix below).
40
+
41
+ "Derivative Works" shall mean any work, whether in Source or Object
42
+ form, that is based on (or derived from) the Work and for which the
43
+ editorial revisions, annotations, elaborations, or other modifications
44
+ represent, as a whole, an original work of authorship. For the purposes
45
+ of this License, Derivative Works shall not include works that remain
46
+ separable from, or merely link (or bind by name) to the interfaces of,
47
+ the Work and Derivative Works thereof.
48
+
49
+ "Contribution" shall mean any work of authorship, including
50
+ the original version of the Work and any modifications or additions
51
+ to that Work or Derivative Works thereof, that is intentionally
52
+ submitted to Licensor for inclusion in the Work by the copyright owner
53
+ or by an individual or Legal Entity authorized to submit on behalf of
54
+ the copyright owner. For the purposes of this definition, "submitted"
55
+ means any form of electronic, verbal, or written communication sent
56
+ to the Licensor or its representatives, including but not limited to
57
+ communication on electronic mailing lists, source code control systems,
58
+ and issue tracking systems that are managed by, or on behalf of, the
59
+ Licensor for the purpose of discussing and improving the Work, but
60
+ excluding communication that is conspicuously marked or otherwise
61
+ designated in writing by the copyright owner as "Not a Contribution."
62
+
63
+ "Contributor" shall mean Licensor and any individual or Legal Entity
64
+ on behalf of whom a Contribution has been received by Licensor and
65
+ subsequently incorporated within the Work.
66
+
67
+ 2. Grant of Copyright License. Subject to the terms and conditions of
68
+ this License, each Contributor hereby grants to You a perpetual,
69
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
70
+ copyright license to reproduce, prepare Derivative Works of,
71
+ publicly display, publicly perform, sublicense, and distribute the
72
+ Work and such Derivative Works in Source or Object form.
73
+
74
+ 3. Grant of Patent License. Subject to the terms and conditions of
75
+ this License, each Contributor hereby grants to You a perpetual,
76
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
77
+ (except as stated in this section) patent license to make, have made,
78
+ use, offer to sell, sell, import, and otherwise transfer the Work,
79
+ where such license applies only to those patent claims licensable
80
+ by such Contributor that are necessarily infringed by their
81
+ Contribution(s) alone or by combination of their Contribution(s)
82
+ with the Work to which such Contribution(s) was submitted. If You
83
+ institute patent litigation against any entity (including a
84
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
85
+ or a Contribution incorporated within the Work constitutes direct
86
+ or contributory patent infringement, then any patent licenses
87
+ granted to You under this License for that Work shall terminate
88
+ as of the date such litigation is filed.
89
+
90
+ 4. Redistribution. You may reproduce and distribute copies of the
91
+ Work or Derivative Works thereof in any medium, with or without
92
+ modifications, and in Source or Object form, provided that You
93
+ meet the following conditions:
94
+
95
+ (a) You must give any other recipients of the Work or
96
+ Derivative Works a copy of this License; and
97
+
98
+ (b) You must cause any modified files to carry prominent notices
99
+ stating that You changed the files; and
100
+
101
+ (c) You must retain, in the Source form of any Derivative Works
102
+ that You distribute, all copyright, patent, trademark, and
103
+ attribution notices from the Source form of the Work,
104
+ excluding those notices that do not pertain to any part of
105
+ the Derivative Works; and
106
+
107
+ (d) If the Work includes a "NOTICE" text file as part of its
108
+ distribution, then any Derivative Works that You distribute must
109
+ include a readable copy of the attribution notices contained
110
+ within such NOTICE file, excluding those notices that do not
111
+ pertain to any part of the Derivative Works, in at least one
112
+ of the following places: within a NOTICE text file distributed
113
+ as part of the Derivative Works; within the Source form or
114
+ documentation, if provided along with the Derivative Works; or,
115
+ within a display generated by the Derivative Works, if and
116
+ wherever such third-party notices normally appear. The contents
117
+ of the NOTICE file are for informational purposes only and
118
+ do not modify the License. You may add Your own attribution
119
+ notices within Derivative Works that You distribute, alongside
120
+ or as an addendum to the NOTICE text from the Work, provided
121
+ that such additional attribution notices cannot be construed
122
+ as modifying the License.
123
+
124
+ You may add Your own copyright statement to Your modifications and
125
+ may provide additional or different license terms and conditions
126
+ for use, reproduction, or distribution of Your modifications, or
127
+ for any such Derivative Works as a whole, provided Your use,
128
+ reproduction, and distribution of the Work otherwise complies with
129
+ the conditions stated in this License.
130
+
131
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
132
+ any Contribution intentionally submitted for inclusion in the Work
133
+ by You to the Licensor shall be under the terms and conditions of
134
+ this License, without any additional terms or conditions.
135
+ Notwithstanding the above, nothing herein shall supersede or modify
136
+ the terms of any separate license agreement you may have executed
137
+ with Licensor regarding such Contributions.
138
+
139
+ 6. Trademarks. This License does not grant permission to use the trade
140
+ names, trademarks, service marks, or product names of the Licensor,
141
+ except as required for reasonable and customary use in describing the
142
+ origin of the Work and reproducing the content of the NOTICE file.
143
+
144
+ 7. Disclaimer of Warranty. Unless required by applicable law or
145
+ agreed to in writing, Licensor provides the Work (and each
146
+ Contributor provides its Contributions) on an "AS IS" BASIS,
147
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
148
+ implied, including, without limitation, any warranties or conditions
149
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
150
+ PARTICULAR PURPOSE. You are solely responsible for determining the
151
+ appropriateness of using or redistributing the Work and assume any
152
+ risks associated with Your exercise of permissions under this License.
153
+
154
+ 8. Limitation of Liability. In no event and under no legal theory,
155
+ whether in tort (including negligence), contract, or otherwise,
156
+ unless required by applicable law (such as deliberate and grossly
157
+ negligent acts) or agreed to in writing, shall any Contributor be
158
+ liable to You for damages, including any direct, indirect, special,
159
+ incidental, or consequential damages of any character arising as a
160
+ result of this License or out of the use or inability to use the
161
+ Work (including but not limited to damages for loss of goodwill,
162
+ work stoppage, computer failure or malfunction, or any and all
163
+ other commercial damages or losses), even if such Contributor
164
+ has been advised of the possibility of such damages.
165
+
166
+ 9. Accepting Warranty or Additional Liability. While redistributing
167
+ the Work or Derivative Works thereof, You may choose to offer,
168
+ and charge a fee for, acceptance of support, warranty, indemnity,
169
+ or other liability obligations and/or rights consistent with this
170
+ License. However, in accepting such obligations, You may act only
171
+ on Your own behalf and on Your sole responsibility, not on behalf
172
+ of any other Contributor, and only if You agree to indemnify,
173
+ defend, and hold each Contributor harmless for any liability
174
+ incurred by, or claims asserted against, such Contributor by reason
175
+ of your accepting any such warranty or additional liability.
176
+
177
+ END OF TERMS AND CONDITIONS
178
+
179
+ APPENDIX: How to apply the Apache License to your work.
180
+
181
+ To apply the Apache License to your work, attach the following
182
+ boilerplate notice, with the fields enclosed by brackets "[]"
183
+ replaced with your own identifying information. (Don't include
184
+ the brackets!) The text should be enclosed in the appropriate
185
+ comment syntax for the file format. We also recommend that a
186
+ file or class name and description of purpose be included on the
187
+ same "printed page" as the copyright notice for easier
188
+ identification within third-party archives.
189
+
190
+ Copyright [yyyy] [name of copyright owner]
191
+
192
+ Licensed under the Apache License, Version 2.0 (the "License");
193
+ you may not use this file except in compliance with the License.
194
+ You may obtain a copy of the License at
195
+
196
+ http://www.apache.org/licenses/LICENSE-2.0
197
+
198
+ Unless required by applicable law or agreed to in writing, software
199
+ distributed under the License is distributed on an "AS IS" BASIS,
200
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
201
+ See the License for the specific language governing permissions and
202
+ limitations under the License.
@@ -0,0 +1,5 @@
1
+ <!--
2
+ Please note that we require our contributors to sign a Contributor License Agreement before any contribution can be considered, reviewed, or accepted.
3
+
4
+ Please see our Contributing Guidelines for details.
5
+ -->
data/README.md ADDED
@@ -0,0 +1,283 @@
1
+ # RedisCall
2
+
3
+ A [Redis](https://redis.io) access library, which provides:
4
+
5
+ * Connection management
6
+ * Per-thread connection
7
+ * Shared between threads connection
8
+ * Key names construction
9
+ * Transactions
10
+ * Queued result handling
11
+ * Retry on optimistic lock fail
12
+ * Queues
13
+ * message handling with backup queue
14
+ * restore from backup queue
15
+ * queues in Rails controllers
16
+ * messages as REST resources
17
+ * message handling in separate process
18
+ * graceful stop of blocking operations
19
+ * JSON encode/decode
20
+ * And nice ruby API
21
+
22
+ It use [Hiredis](https://github.com/redis/hiredis) to connect to [Redis](https://redis.io).
23
+
24
+
25
+ ## Basic operations
26
+ ```ruby
27
+ r = RedisCall.new
28
+
29
+ prefix = r.key('foo')
30
+
31
+ # set the 'foo.bar.qux' to 'content' with 1 day expiration
32
+ r.setex prefix+:bar+'qux', 1.day, 'content'
33
+ r.setex prefix/:bar/'qux', 1.day, 'content'
34
+
35
+ # get 'foo.bar.qux'
36
+ r.get prefix/:bar/'qux'
37
+ ```
38
+
39
+
40
+ ## Custom storage class
41
+
42
+ ```ruby
43
+ class MyStorage < RedisCall
44
+ def store key_name, value
45
+ multi do
46
+ del key(:prefix)/key_name/:some_state
47
+ set key(:prefix)/key_name, value
48
+ end
49
+ end
50
+ end
51
+ ```
52
+
53
+
54
+ # Queues
55
+
56
+ ## Basic operations
57
+
58
+ ```ruby
59
+ queue = RedisQueue::Base.new('foo')
60
+ queue.push(hello: 'darling')
61
+
62
+ queue.name
63
+ queue.length
64
+ queue.backup_length
65
+
66
+ queue.pop_all
67
+ queue.backup_elements
68
+ queue.backed_up_pop_all
69
+
70
+ subqueue = RedisQueue::Base.new(queue.name/:subqueue)
71
+ ```
72
+
73
+
74
+ ## Message handler process
75
+
76
+ Here is sample standalone message handler process with backup queue.
77
+ To handle process-related functionality (start/stop/watchdog), [workety](https://github.com/senotrusov/workety) library is used.
78
+
79
+ ```ruby
80
+ class Handler
81
+ def start
82
+ @queue = RedisQueue::Base.new 'queue_name', connect: true
83
+ @queue.restore_backup
84
+
85
+ @handler = Thread.networkety do
86
+ loop do
87
+ @queue.backed_up_blocking_pop do |message|
88
+ puts message.inspect
89
+ end
90
+ end
91
+ end
92
+ end
93
+
94
+ def join
95
+ @handler.log_join "Handler thread joined"
96
+ end
97
+
98
+ def stop
99
+ @queue.disconnect @handler
100
+ end
101
+ end
102
+ ```
103
+
104
+
105
+ ## Serialization and custom queue class
106
+
107
+ Redis operates with string as a queue element. To transfer Ruby classes, serialization technique may be used.
108
+ RedisCall use JSON as the transfer format.
109
+ In case when you want to transfer not Hash but some custom class (say MyElement),
110
+ you may define your own queue class to handle that:
111
+
112
+ ```ruby
113
+ class MyQueue < RedisQueue::Base
114
+ def encode element
115
+ super(element.as_json(:except => [:serialized]))
116
+ end
117
+
118
+ def decode raw
119
+ MyElement.new(super(raw))
120
+ end
121
+ end
122
+ ```
123
+
124
+
125
+ ## Queue metadata
126
+
127
+ You may store queue metadata in ``config/redis_queue.yml`` file:
128
+
129
+ ```yml
130
+ queue_name:
131
+ attr: value
132
+ ```
133
+
134
+ and retrieve it:
135
+
136
+ ``` ruby
137
+ RedisQueue::Base.new('queue_name').config[:attr]
138
+ # => 'value'
139
+ ```
140
+
141
+
142
+ ## Queues in Rails controllers
143
+
144
+ ### A list of queues
145
+
146
+ ```ruby
147
+ class QueueListController
148
+ # Queue names currently are not checked for funny characters by library,
149
+ # so a potential of Redis command injection exists.
150
+ def filtered_id
151
+ params[:id].tr('^A-Za-z0-9.', '')
152
+ end
153
+
154
+ def index
155
+ @queues = RedisQueue::Base.all
156
+ end
157
+
158
+ def destroy
159
+ @queue = RedisQueue::Base.find filtered_id
160
+ @queue.destroy
161
+ end
162
+ end
163
+ ```
164
+
165
+
166
+ ### Messages
167
+
168
+ ```ruby
169
+ class MessagesController
170
+ def queue
171
+ @queue ||= RedisQueue::Base.new('input')
172
+ end
173
+
174
+ def output_queue
175
+ @output_queue ||= RedisQueue::Base.new('output')
176
+ end
177
+
178
+ def raw_message
179
+ params[:message][:serialized]
180
+ end
181
+
182
+ def index
183
+ @messages = queue.elements
184
+ @messages = queue.backup_elements
185
+
186
+ @messages = queue.pop_all
187
+ @messages = queue.backed_up_pop_all
188
+ end
189
+
190
+ def update
191
+ queue.insist do # Retry on optimistic lock fail
192
+ queue.watch_backup # Optimistic locking
193
+
194
+ if queue.raw_backup_elements.include? raw_message
195
+ @message = queue.decode raw_message
196
+
197
+ if @message.valid?
198
+
199
+ queue.multi do
200
+ queue.remove_raw_backup_element raw_message
201
+ output_queue.push "some message"
202
+ end
203
+
204
+ respond_to do |format|
205
+ format.html { render :nothing => true, status: :created }
206
+ end
207
+ else
208
+ respond_to do |format|
209
+ format.html { render partial: 'form', :locals => {message: @message}, status: :unprocessable_entity }
210
+ end
211
+ end
212
+ else
213
+ respond_to do |format|
214
+ format.html { render text: "The message you are trying to update is gone", status: :gone }
215
+ end
216
+ end
217
+ end
218
+ end
219
+
220
+ def destroy
221
+ queue.remove_raw_element raw_message
222
+ end
223
+ end
224
+ ```
225
+
226
+
227
+ # Environment details
228
+
229
+ ## Connection configuration
230
+
231
+ By default, Redis is expected to be found listening the 127.0.0.1:6379, but that can be configured in ``config/redis_call.yml``
232
+
233
+ ```yml
234
+ development:
235
+ host: localhost
236
+ port: 6379
237
+
238
+ production:
239
+ host: localhost
240
+ port: 6379
241
+
242
+ test:
243
+ host: localhost
244
+ port: 6379
245
+ ```
246
+
247
+
248
+ ## Forks and Unicorn webserver
249
+
250
+ Forking the process require reconnection. For Unicorn webserver it may be done by stating in ``config/unicorn.rb``:
251
+
252
+ ```ruby
253
+ before_fork do |server, worker|
254
+ RedisCall.new.disconnect
255
+ end
256
+
257
+ after_fork do |server, worker|
258
+ RedisCall.new.connect
259
+ end
260
+ ```
261
+
262
+
263
+ ## Copyright and License
264
+
265
+ ```
266
+ Copyright 2011 Stanislav Senotrusov
267
+
268
+ Licensed under the Apache License, Version 2.0 (the "License");
269
+ you may not use this file except in compliance with the License.
270
+ You may obtain a copy of the License at
271
+
272
+ http://www.apache.org/licenses/LICENSE-2.0
273
+
274
+ Unless required by applicable law or agreed to in writing, software
275
+ distributed under the License is distributed on an "AS IS" BASIS,
276
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
277
+ See the License for the specific language governing permissions and
278
+ limitations under the License.
279
+ ```
280
+
281
+ ## Contributing
282
+
283
+ Please see our [Contributing Guidelines](CONTRIBUTING.md) for details.
data/Rakefile ADDED
@@ -0,0 +1,36 @@
1
+
2
+ # Copyright 2011 Stanislav Senotrusov <stan@senotrusov.com>
3
+ #
4
+ # Licensed under the Apache License, Version 2.0 (the "License");
5
+ # you may not use this file except in compliance with the License.
6
+ # You may obtain a copy of the License at
7
+ #
8
+ # http://www.apache.org/licenses/LICENSE-2.0
9
+ #
10
+ # Unless required by applicable law or agreed to in writing, software
11
+ # distributed under the License is distributed on an "AS IS" BASIS,
12
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13
+ # See the License for the specific language governing permissions and
14
+ # limitations under the License.
15
+
16
+
17
+ begin
18
+ require 'jeweler'
19
+
20
+ Jeweler::Tasks.new do |gemspec|
21
+ gemspec.name = "redis-call"
22
+ gemspec.licenses = ['Apache-2.0']
23
+ gemspec.summary = "Redis access library for Ruby: threads, handy key names, transactions and queues"
24
+ gemspec.authors = ["Stanislav Senotrusov"]
25
+ gemspec.email = "stan@senotrusov.com"
26
+ gemspec.homepage = "https://github.com/senotrusov/redis-call"
27
+
28
+ gemspec.add_dependency 'hiredis'
29
+ gemspec.add_dependency 'yajl-ruby'
30
+ end
31
+
32
+ Jeweler::GemcutterTasks.new
33
+
34
+ rescue LoadError
35
+ puts "Jeweler not available. Install it with: gem install jeweler"
36
+ end
data/VERSION ADDED
@@ -0,0 +1 @@
1
+ 1.0.2
data/lib/redis-call.rb ADDED
@@ -0,0 +1,23 @@
1
+
2
+ # Copyright 2011 Stanislav Senotrusov <stan@senotrusov.com>
3
+ #
4
+ # Licensed under the Apache License, Version 2.0 (the "License");
5
+ # you may not use this file except in compliance with the License.
6
+ # You may obtain a copy of the License at
7
+ #
8
+ # http://www.apache.org/licenses/LICENSE-2.0
9
+ #
10
+ # Unless required by applicable law or agreed to in writing, software
11
+ # distributed under the License is distributed on an "AS IS" BASIS,
12
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13
+ # See the License for the specific language governing permissions and
14
+ # limitations under the License.
15
+
16
+
17
+ require 'hiredis'
18
+ require 'yajl'
19
+
20
+ require 'redis-call/redis_call.rb'
21
+ require 'redis-call/redis_queue.rb'
22
+ require 'redis-call/railtie.rb' if defined?(Rails)
23
+
@@ -0,0 +1,47 @@
1
+
2
+ # Copyright 2011 Stanislav Senotrusov <stan@senotrusov.com>
3
+ #
4
+ # Licensed under the Apache License, Version 2.0 (the "License");
5
+ # you may not use this file except in compliance with the License.
6
+ # You may obtain a copy of the License at
7
+ #
8
+ # http://www.apache.org/licenses/LICENSE-2.0
9
+ #
10
+ # Unless required by applicable law or agreed to in writing, software
11
+ # distributed under the License is distributed on an "AS IS" BASIS,
12
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13
+ # See the License for the specific language governing permissions and
14
+ # limitations under the License.
15
+
16
+
17
+ class RedisCall::Railtie < Rails::Railtie
18
+ initializer :redis_call do |app|
19
+
20
+ if File.exists?(config_file = Rails.root + 'config' + 'redis_call.yml')
21
+ if (yaml = YAML.load_file(config_file)).kind_of?(Hash)
22
+ if (env = yaml[Rails.env]).kind_of?(Hash)
23
+ RedisCall.config = env.with_indifferent_access
24
+ else
25
+ STDERR.write "WARNING: #{config_file} hash does not contain key for current #{Rails.env} environment\n"
26
+ end
27
+ else
28
+ STDERR.write "WARNING: #{config_file} does not contain a Hash\n"
29
+ end
30
+ end
31
+
32
+ end
33
+ end
34
+
35
+ class RedisQueue::Railtie < Rails::Railtie
36
+ config.to_prepare do
37
+
38
+ if File.exists?(config_file = Rails.root + 'config' + 'redis_queue.yml')
39
+ if (yaml = YAML.load_file(config_file)).kind_of?(Hash)
40
+ RedisQueue.config = yaml.select {|queue, options| queue != "templates"}.with_indifferent_access
41
+ else
42
+ STDERR.write "WARNING: #{config_file} does not contain a Hash\n"
43
+ end
44
+ end
45
+
46
+ end
47
+ end
@@ -0,0 +1,316 @@
1
+
2
+ # Copyright 2011 Stanislav Senotrusov <stan@senotrusov.com>
3
+ #
4
+ # Licensed under the Apache License, Version 2.0 (the "License");
5
+ # you may not use this file except in compliance with the License.
6
+ # You may obtain a copy of the License at
7
+ #
8
+ # http://www.apache.org/licenses/LICENSE-2.0
9
+ #
10
+ # Unless required by applicable law or agreed to in writing, software
11
+ # distributed under the License is distributed on an "AS IS" BASIS,
12
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13
+ # See the License for the specific language governing permissions and
14
+ # limitations under the License.
15
+
16
+
17
+ class RedisCall
18
+ class UnexpectedResult < StandardError; end
19
+ class TransactionAborted < StandardError; end
20
+ class NonTransactionalMethod < StandardError; end
21
+
22
+ class Key
23
+ def initialize name
24
+ @name = name.to_s
25
+ end
26
+
27
+ def + name
28
+ self.class.new(@name + '.' + name.to_s)
29
+ end
30
+
31
+ alias_method "/", "+"
32
+
33
+ def inspect
34
+ @name.inspect
35
+ end
36
+
37
+ def to_str
38
+ @name
39
+ end
40
+
41
+ alias_method :to_s, :to_str
42
+
43
+ def method_missing *args, &block
44
+ @name.__send__ *args, &block
45
+ end
46
+ end
47
+
48
+ def key name
49
+ RedisCall::Key.new name
50
+ end
51
+
52
+
53
+ class Connection
54
+ def initialize(host, port)
55
+ @connection = Hiredis::Connection.new
56
+ @connection.connect(host, port)
57
+
58
+ @multi_depth = 0
59
+ end
60
+
61
+ def connected?
62
+ @connection.connected?
63
+ end
64
+
65
+ def disconnect
66
+ @connection.disconnect
67
+ end
68
+
69
+ def inside_transaction?
70
+ @multi_depth != 0
71
+ end
72
+
73
+
74
+ def call *args
75
+ @connection.write(args)
76
+ result = @connection.read
77
+
78
+ @call_index += 1 if @call_index
79
+
80
+ raise result if result.is_a?(Exception)
81
+
82
+ result
83
+
84
+ rescue RuntimeError => exception
85
+ if exception.message == "not connected"
86
+ raise(IOError, "Not connected")
87
+ else
88
+ raise(exception)
89
+ end
90
+ end
91
+
92
+ alias_method :method_missing, :call
93
+
94
+
95
+ def queued result, &block
96
+ if @queued_handlers
97
+ (@queued_handlers[@call_index] ||= []).push(block)
98
+ else
99
+ yield(result)
100
+ end
101
+ end
102
+
103
+ def exec
104
+ if (@multi_depth -= 1) == 0
105
+ begin
106
+ unless result = call(:EXEC)
107
+ raise RedisCall::TransactionAborted
108
+ end
109
+
110
+ drop_results = []
111
+
112
+ @queued_handlers.each do |index, handlers|
113
+ result[index] = handlers.inject(result[index]) do |data, handler|
114
+ if handler
115
+ handler.call(data)
116
+ else
117
+ drop_results.push index
118
+ data
119
+ end
120
+ end
121
+ end
122
+
123
+ drop_results.each {|index| result.delete_at index }
124
+
125
+ (result.length == 1) ? result.first : result
126
+
127
+ ensure
128
+ @call_index = @queued_handlers = nil
129
+ end
130
+ end
131
+ end
132
+
133
+
134
+ def discard
135
+ if (@multi_depth -= 1) == 0
136
+ begin
137
+ call(:DISCARD) if @connection.connected?
138
+ ensure
139
+ @call_index = @queued_handlers = nil
140
+ end
141
+ end
142
+ end
143
+
144
+
145
+ def multi
146
+ call(:MULTI) if (@multi_depth += 1) == 1
147
+
148
+ @call_index = -1
149
+ @queued_handlers = {}
150
+
151
+ if block_given?
152
+ begin
153
+ yield
154
+ rescue ScriptError, StandardError => exception
155
+ begin
156
+ discard
157
+ rescue ScriptError, StandardError => discard_exception
158
+ # It is not important to report this error
159
+ discard_exception.report! if discard_exception.respond_to? :report!
160
+ ensure
161
+ raise exception
162
+ end
163
+ end
164
+ exec
165
+ end
166
+ end
167
+
168
+ end
169
+
170
+
171
+ def self.query(*args, &block)
172
+ self.new(*args).instance_exec(&block)
173
+ end
174
+
175
+
176
+ @@config = {}
177
+
178
+ def self.config= conf
179
+ @@config = conf
180
+ end
181
+
182
+ DEFAULT_HOST = "127.0.0.1"
183
+ DEFAULT_PORT = 6379
184
+
185
+ def initialize(args = {})
186
+ @host = args[:host] || @@config[:host] || DEFAULT_HOST
187
+ @port = args[:port] || @@config[:port] || DEFAULT_PORT
188
+
189
+ if args[:connect]
190
+ @connection = Connection.new(@host, @port)
191
+ else
192
+ @pool_key = "redis_#{@host}:#{@port}".to_sym
193
+ end
194
+ end
195
+
196
+ def connection
197
+ @connection || (Thread.current[@pool_key] ||= Connection.new(@host, @port))
198
+ end
199
+
200
+ alias_method :connect, :connection
201
+
202
+ def disconnect(thread = nil, limit = 10)
203
+ begin
204
+ connection.disconnect
205
+ rescue RuntimeError => exception
206
+ raise(exception) if exception.message != "not connected"
207
+ end
208
+
209
+ Thread.current[@pool_key] = nil if @pool_key
210
+
211
+ if thread
212
+ begin
213
+ thread.run
214
+ rescue ThreadError => exception
215
+ raise exception if exception.message != "killed thread"
216
+ end
217
+ thread.join(limit)
218
+ end
219
+ end
220
+
221
+ def method_missing *args, &block
222
+ connection.__send__ *args, &block
223
+ end
224
+
225
+ def insist(retries = 42, *exceptions)
226
+ exceptions.push RedisCall::TransactionAborted
227
+ yield
228
+ rescue *exceptions => exception
229
+ if (retries -= 1) > 0
230
+ retry
231
+ else
232
+ raise exception
233
+ end
234
+ end
235
+
236
+
237
+ def rpushex key, ttl, value
238
+ multi do
239
+ queued(rpush key, value) {|result| result}
240
+ queued(expire key, ttl)
241
+ end
242
+ end
243
+
244
+ def decrzerodelex key, ttl
245
+ multi do
246
+ queued(decr key) do |result|
247
+ del(key) if result <= 0
248
+ result
249
+ end
250
+ queued(expire key, ttl)
251
+ end
252
+ end
253
+
254
+ def llen key
255
+ queued(call :LLEN, key) {|result| result.to_i}
256
+ end
257
+
258
+ def getnnil key
259
+ queued(get key) do |result|
260
+ raise(RedisCall::UnexpectedResult, "Key #{key.inspect} expected to be not nil") if result == nil
261
+ result
262
+ end
263
+ end
264
+
265
+ def getnnili key
266
+ queued(getnnil key) {|result| result.to_i}
267
+ end
268
+
269
+ def geti key
270
+ queued(get key) {|result| result.to_i}
271
+ end
272
+
273
+ def lgetall key
274
+ lrange key, 0, -1
275
+ end
276
+
277
+ def hgetallarr key
278
+ queued(hgetall key) do |raw|
279
+ result = []
280
+ Hash[*raw].each {|k, v| result[k.to_i] = v}
281
+ result
282
+ end
283
+ end
284
+
285
+
286
+ module JSON
287
+ def encode element
288
+ Yajl::Encoder.encode(element)
289
+ end
290
+
291
+ def decode raw
292
+ (result = Yajl::Parser.new.parse(raw)).is_a?(Hash) ? result.with_indifferent_access : result
293
+ end
294
+
295
+ alias_method :encode_json, :encode
296
+ alias_method :decode_json, :decode
297
+ end
298
+
299
+ module KeepSerializedElement
300
+ def encode element
301
+ if element.is_a?(Hash)
302
+ element = element.dup
303
+ element.delete :serialized
304
+ end
305
+ super(element)
306
+ end
307
+
308
+ def decode raw
309
+ result = super(raw)
310
+ result[:serialized] = raw if result.is_a?(Hash)
311
+ result
312
+ end
313
+ end
314
+
315
+ end
316
+
@@ -0,0 +1,259 @@
1
+
2
+ # Copyright 2011 Stanislav Senotrusov <stan@senotrusov.com>
3
+ #
4
+ # Licensed under the Apache License, Version 2.0 (the "License");
5
+ # you may not use this file except in compliance with the License.
6
+ # You may obtain a copy of the License at
7
+ #
8
+ # http://www.apache.org/licenses/LICENSE-2.0
9
+ #
10
+ # Unless required by applicable law or agreed to in writing, software
11
+ # distributed under the License is distributed on an "AS IS" BASIS,
12
+ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13
+ # See the License for the specific language governing permissions and
14
+ # limitations under the License.
15
+
16
+
17
+ module RedisQueue
18
+ class ElementNotFound < StandardError; end
19
+
20
+ @config = {}
21
+
22
+ def self.config= conf
23
+ @config = conf
24
+ end
25
+
26
+ def self.config
27
+ @config
28
+ end
29
+
30
+ class Simple < RedisCall
31
+ class << self
32
+ alias_method :find, :new
33
+
34
+ def all
35
+ klass = self
36
+ query do
37
+ queued(keys "queue.*") do |result|
38
+ (result.collect {|name| name.gsub /\Aqueue\./, ''} | RedisQueue.config.keys).sort.collect {|name| klass.new name}
39
+ end
40
+ end
41
+ end
42
+
43
+ def delete *names
44
+ query do
45
+ del *(names.map{|name| key(:queue)/name})
46
+ end
47
+ end
48
+ end
49
+
50
+ attr_reader :name, :config
51
+
52
+ def initialize(name = nil, args = {})
53
+ super(args)
54
+
55
+ @name = key(name)
56
+ @key = key(:queue)/name
57
+
58
+ @config = args[:config] || RedisQueue.config[name] || {}
59
+ end
60
+
61
+
62
+ def encode element
63
+ element
64
+ end
65
+
66
+ def decode element
67
+ element
68
+ end
69
+
70
+
71
+ # Returns the number of elements inside the queue after the push operation.
72
+ def push element
73
+ lpush(@key, encode(element))
74
+ end
75
+
76
+ def error_push element
77
+ lpush(@key/:error, encode(element))
78
+ end
79
+
80
+ def error_push_raw element
81
+ lpush(@key/:error, element)
82
+ end
83
+
84
+
85
+ # Returns element
86
+ def pop
87
+ queued(rpop @key) {|result| decode(result) if result}
88
+ end
89
+
90
+ # Returns element
91
+ def blocking_pop timeout = 0
92
+ if result = brpop(@key, timeout)
93
+ decode(result.last)
94
+ end
95
+ end
96
+
97
+ def backed_up_pop
98
+ queued(rpoplpush(@key, @key/:backup)) {|result| decode(result) if result}
99
+ end
100
+
101
+ # Returns element
102
+ def backed_up_blocking_pop timeout = 0
103
+ if raw_element = brpoplpush(@key, @key/:backup, timeout)
104
+ element = decode(raw_element)
105
+
106
+ if block_given?
107
+ yield(element)
108
+ remove_raw_backup_element raw_element
109
+ else
110
+ return element
111
+ end
112
+ end
113
+ end
114
+
115
+ def remove_raw_backup_element element
116
+ queued(lrem(@key/:backup, -1, element)) do |result|
117
+ raise(RedisQueue::ElementNotFound, "Not found element #{element.inspect} in queue #{@key/:backup}") if result != 1
118
+ end
119
+ end
120
+
121
+ def remove_raw_element element
122
+ queued(lrem(@key, -1, element)) do |result|
123
+ raise(RedisQueue::ElementNotFound, "Not found element #{element.inspect} in queue #{@key}") if result != 1
124
+ end
125
+ end
126
+
127
+
128
+ def backed_up_pop_all
129
+ raise(NonTransactionalMethod) if inside_transaction?
130
+ result = []
131
+ # We does not call backed_up_pop here, because of the edge case, when element is a string "null" which JSON-decoded as nil
132
+ while element = rpoplpush(@key, @key/:backup)
133
+ result.push decode(element)
134
+ end
135
+ result
136
+ end
137
+
138
+ def pop_all
139
+ multi do
140
+ queued(lgetall(@key)) {|result| result.map {|element| decode(element)}.reverse }
141
+ queued(del @key)
142
+ end
143
+ end
144
+
145
+
146
+ def elements
147
+ queued(lgetall(@key)) {|result| result.map {|element| decode(element)}.reverse }
148
+ end
149
+
150
+ def backup_elements
151
+ queued(lgetall(@key/:backup)) {|result| result.map {|element| decode(element)}.reverse }
152
+ end
153
+
154
+ def raw_backup_elements
155
+ queued(lgetall(@key/:backup)) {|result| result.reverse }
156
+ end
157
+
158
+
159
+ def watch *keys
160
+ keys.empty? ? super(@key) : super(*keys)
161
+ end
162
+
163
+ def watch_backup
164
+ watch @key/:backup
165
+ end
166
+
167
+ # NOTE: Make sure your redis is 2.2.13 or higher to use this method
168
+ # https://github.com/antirez/redis/commit/c47d152c8d96415de1af994b1a4bb3e0347caef3
169
+ # http://code.google.com/p/redis/issues/detail?id=593
170
+ def blocking_redirect to, timeout = 0
171
+ brpoplpush(@key, key(:queue)/(to.kind_of?(RedisQueue::Simple) ? to.name : to), timeout)
172
+ end
173
+
174
+ def restore_backup
175
+ raise(NonTransactionalMethod) if inside_transaction?
176
+
177
+ while element = rpop(@key/:backup)
178
+ if element = filter_backup_element(element)
179
+ lpush(@key, element)
180
+ end
181
+ end
182
+ end
183
+
184
+ def filter_backup_element element
185
+ element
186
+ end
187
+
188
+ def length
189
+ llen(@key)
190
+ end
191
+
192
+ def error_length
193
+ llen(@key/:error)
194
+ end
195
+
196
+ def backup_length
197
+ llen(@key/:backup)
198
+ end
199
+
200
+ def delete
201
+ del(@key)
202
+ end
203
+
204
+ alias_method :destroy, :delete
205
+ end
206
+
207
+
208
+ module RestoreBackupLimit
209
+ BACKUP_LIMIT = 3
210
+ BACKUP_COUNT_KEY = :redis_queue_backup_retry_count
211
+
212
+ def filter_backup_element element
213
+ result = decode_json(element)
214
+
215
+ if result.is_a?(Hash)
216
+ result[BACKUP_COUNT_KEY] ||= 0
217
+ result[BACKUP_COUNT_KEY] += 1
218
+
219
+ if result[BACKUP_COUNT_KEY] > BACKUP_LIMIT
220
+ error_push_raw encode_json(result)
221
+ return nil
222
+ else
223
+ return encode_json(result)
224
+ end
225
+
226
+ else
227
+ element
228
+ end
229
+ end
230
+ end
231
+
232
+ class Base < Simple
233
+ include RedisCall::JSON
234
+ include RedisCall::KeepSerializedElement
235
+
236
+ include RedisQueue::RestoreBackupLimit
237
+
238
+ extend ActiveModel::Naming
239
+ include ActiveModel::Conversion
240
+
241
+ def persisted?
242
+ true
243
+ end
244
+
245
+ def id
246
+ @name
247
+ end
248
+
249
+ def action name
250
+ @config[:actions] && @config[:actions].find {|action| action[:name] = name.to_s}
251
+ end
252
+
253
+ def requested_action params
254
+ @config[:actions] && @config[:actions].find {|action| params[action[:name]]}
255
+ end
256
+ end
257
+
258
+ end
259
+
@@ -0,0 +1,54 @@
1
+ # Generated by jeweler
2
+ # DO NOT EDIT THIS FILE DIRECTLY
3
+ # Instead, edit Jeweler::Tasks in Rakefile, and run 'rake gemspec'
4
+ # -*- encoding: utf-8 -*-
5
+ # stub: redis-call 1.0.2 ruby lib
6
+
7
+ Gem::Specification.new do |s|
8
+ s.name = "redis-call"
9
+ s.version = "1.0.2"
10
+
11
+ s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
12
+ s.require_paths = ["lib"]
13
+ s.authors = ["Stanislav Senotrusov"]
14
+ s.date = "2017-04-23"
15
+ s.email = "stan@senotrusov.com"
16
+ s.extra_rdoc_files = [
17
+ "LICENSE",
18
+ "README.md"
19
+ ]
20
+ s.files = [
21
+ "CONTRIBUTING.md",
22
+ "ISSUE_TEMPLATE.md",
23
+ "LICENSE",
24
+ "PULL_REQUEST_TEMPLATE.md",
25
+ "README.md",
26
+ "Rakefile",
27
+ "VERSION",
28
+ "lib/redis-call.rb",
29
+ "lib/redis-call/railtie.rb",
30
+ "lib/redis-call/redis_call.rb",
31
+ "lib/redis-call/redis_queue.rb",
32
+ "redis-call.gemspec"
33
+ ]
34
+ s.homepage = "https://github.com/senotrusov/redis-call"
35
+ s.licenses = ["Apache-2.0"]
36
+ s.rubygems_version = "2.5.1"
37
+ s.summary = "Redis access library for Ruby: threads, handy key names, transactions and queues"
38
+
39
+ if s.respond_to? :specification_version then
40
+ s.specification_version = 4
41
+
42
+ if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then
43
+ s.add_runtime_dependency(%q<hiredis>, [">= 0"])
44
+ s.add_runtime_dependency(%q<yajl-ruby>, [">= 0"])
45
+ else
46
+ s.add_dependency(%q<hiredis>, [">= 0"])
47
+ s.add_dependency(%q<yajl-ruby>, [">= 0"])
48
+ end
49
+ else
50
+ s.add_dependency(%q<hiredis>, [">= 0"])
51
+ s.add_dependency(%q<yajl-ruby>, [">= 0"])
52
+ end
53
+ end
54
+
metadata ADDED
@@ -0,0 +1,86 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: redis-call
3
+ version: !ruby/object:Gem::Version
4
+ version: 1.0.2
5
+ platform: ruby
6
+ authors:
7
+ - Stanislav Senotrusov
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+ date: 2017-04-23 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: hiredis
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - ">="
18
+ - !ruby/object:Gem::Version
19
+ version: '0'
20
+ type: :runtime
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - ">="
25
+ - !ruby/object:Gem::Version
26
+ version: '0'
27
+ - !ruby/object:Gem::Dependency
28
+ name: yajl-ruby
29
+ requirement: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - ">="
32
+ - !ruby/object:Gem::Version
33
+ version: '0'
34
+ type: :runtime
35
+ prerelease: false
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - ">="
39
+ - !ruby/object:Gem::Version
40
+ version: '0'
41
+ description:
42
+ email: stan@senotrusov.com
43
+ executables: []
44
+ extensions: []
45
+ extra_rdoc_files:
46
+ - LICENSE
47
+ - README.md
48
+ files:
49
+ - CONTRIBUTING.md
50
+ - ISSUE_TEMPLATE.md
51
+ - LICENSE
52
+ - PULL_REQUEST_TEMPLATE.md
53
+ - README.md
54
+ - Rakefile
55
+ - VERSION
56
+ - lib/redis-call.rb
57
+ - lib/redis-call/railtie.rb
58
+ - lib/redis-call/redis_call.rb
59
+ - lib/redis-call/redis_queue.rb
60
+ - redis-call.gemspec
61
+ homepage: https://github.com/senotrusov/redis-call
62
+ licenses:
63
+ - Apache-2.0
64
+ metadata: {}
65
+ post_install_message:
66
+ rdoc_options: []
67
+ require_paths:
68
+ - lib
69
+ required_ruby_version: !ruby/object:Gem::Requirement
70
+ requirements:
71
+ - - ">="
72
+ - !ruby/object:Gem::Version
73
+ version: '0'
74
+ required_rubygems_version: !ruby/object:Gem::Requirement
75
+ requirements:
76
+ - - ">="
77
+ - !ruby/object:Gem::Version
78
+ version: '0'
79
+ requirements: []
80
+ rubyforge_project:
81
+ rubygems_version: 2.5.1
82
+ signing_key:
83
+ specification_version: 4
84
+ summary: 'Redis access library for Ruby: threads, handy key names, transactions and
85
+ queues'
86
+ test_files: []