loops 2.0.7 → 2.0.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/.travis.yml +10 -0
- data/Gemfile +4 -0
- data/README.rdoc +33 -27
- data/lib/loops/base.rb +1 -1
- data/lib/loops/logger.rb +4 -0
- data/lib/loops/version.rb +9 -2
- data/loops.gemspec +2 -2
- data/spec/loop_lock_spec.rb +1 -1
- data/spec/loops/base/with_lock_spec.rb +92 -0
- data/spec/loops/base_spec.rb +10 -80
- data/spec/loops/cli_spec.rb +1 -1
- data/spec/loops_spec.rb +2 -2
- metadata +10 -6
data/.travis.yml
ADDED
data/Gemfile
CHANGED
data/README.rdoc
CHANGED
@@ -76,11 +76,12 @@ Here is a simple loop scaffold for you to start from (put this file to
|
|
76
76
|
end
|
77
77
|
|
78
78
|
When you have your loop ready to use, add the following lines to your (maybe empty yet)
|
79
|
-
<tt>config/loops.yml</tt> file:
|
79
|
+
<tt>config/loops.yml</tt> file +loops+ section:
|
80
80
|
|
81
|
-
|
82
|
-
|
83
|
-
|
81
|
+
loops:
|
82
|
+
hello_world:
|
83
|
+
type: simple
|
84
|
+
sleep_period: 10
|
84
85
|
|
85
86
|
This is it! To manage your loop, just run one of the following commands:
|
86
87
|
|
@@ -116,10 +117,11 @@ could use instead of your <tt>./script/loops<tt> binary:
|
|
116
117
|
If you want to have more than one copy of your worker running, that is as simple as adding one
|
117
118
|
option to your loop configuration:
|
118
119
|
|
119
|
-
|
120
|
-
|
121
|
-
|
122
|
-
|
120
|
+
loops:
|
121
|
+
hello_world:
|
122
|
+
type: simple
|
123
|
+
sleep_period: 10
|
124
|
+
workers_number: 1
|
123
125
|
|
124
126
|
This <tt>workers_number</tt> option would tell loops manager to spawn more than one copy of
|
125
127
|
your loop and run them in parallel. The only thing you'd need to do is to remember about
|
@@ -133,15 +135,16 @@ memcache-based locks.
|
|
133
135
|
You can run the same loop class with different configuration parameters by explicitly identifying
|
134
136
|
the loop class to execute:
|
135
137
|
|
136
|
-
|
137
|
-
|
138
|
-
|
139
|
-
|
138
|
+
loops:
|
139
|
+
hello:
|
140
|
+
type: simple
|
141
|
+
loop_name: some_module/my_worker
|
142
|
+
language: English
|
140
143
|
|
141
|
-
|
142
|
-
|
143
|
-
|
144
|
-
|
144
|
+
salut:
|
145
|
+
type: simple
|
146
|
+
loop_name: some_module/my_worker
|
147
|
+
language: French
|
145
148
|
|
146
149
|
Now two independent sets of loops are using the same class <tt>SomeModule::MyWorkerLoop</tt>
|
147
150
|
customized by the language parameter.
|
@@ -199,12 +202,13 @@ code and as the result, you can create a loops like this:
|
|
199
202
|
|
200
203
|
With configs like this:
|
201
204
|
|
202
|
-
|
203
|
-
|
204
|
-
|
205
|
-
|
206
|
-
|
207
|
-
|
205
|
+
loops:
|
206
|
+
# An example of a STOMP queue-based loop
|
207
|
+
my_queue:
|
208
|
+
type: queue
|
209
|
+
host: 127.0.0.1
|
210
|
+
port: 61613
|
211
|
+
queue_name: blah
|
208
212
|
|
209
213
|
Of course, this solution scales perfectly and to make your queue processing faster you just
|
210
214
|
need to add more workers (by adding <tt>workers_number: N</tt> option).
|
@@ -228,10 +232,12 @@ The default engine is <tt>fork</tt>.
|
|
228
232
|
|
229
233
|
== What Ruby implementations does it work for?
|
230
234
|
|
231
|
-
We've tested and used the plugin on MRI 1.8.6/1.8.7 and on JRuby 1.4.0. At this point we do
|
232
|
-
not support demonization in JRuby
|
233
|
-
|
234
|
-
|
235
|
+
We've tested and used the plugin on MRI 1.8.6/1.8.7/1.9.3 and on JRuby 1.4.0. At this point we do
|
236
|
+
not support demonization in JRuby. Obviously because of JVM limitations you won't be able to
|
237
|
+
use <tt>fork</tt> workers engine in JRuby, but threaded workers do pretty well.
|
238
|
+
|
239
|
+
We have a continuous integration server configured for the project:
|
240
|
+
{<img src="https://secure.travis-ci.org/kovyrin/loops.png?branch=master" alt="Build Status" />}[http://travis-ci.org/kovyrin/loops]
|
235
241
|
|
236
242
|
Recommended version of ruby ro run loops is Ruby Enterprise Edition. This is because we
|
237
243
|
have a support for their Copy-On-Write friendly garbage collector that makes your loops
|
@@ -270,6 +276,6 @@ Use <tt>loops help</tt> to get help.
|
|
270
276
|
== Who are the authors?
|
271
277
|
|
272
278
|
This plugin has been created in Scribd.com for our internal use and then the sources were opened
|
273
|
-
for other people to use. All the code in this package has been developed by
|
279
|
+
for other people to use. All the code in this package has been developed by Oleksiy Kovyrin,
|
274
280
|
Dmytro Shteflyuk and Alexey Verkhovsky for Scribd.com and is released under the MIT license.
|
275
281
|
For more details, see LICENSE file.
|
data/lib/loops/base.rb
CHANGED
data/lib/loops/logger.rb
CHANGED
@@ -129,6 +129,10 @@ class Loops::Logger < ::Delegator
|
|
129
129
|
@implementation or raise "Logger implementation not initialized"
|
130
130
|
end
|
131
131
|
|
132
|
+
def __setobj__(obj)
|
133
|
+
@implementation = obj
|
134
|
+
end
|
135
|
+
|
132
136
|
# @private
|
133
137
|
# Delegator's method_missing ignores the &block argument (!!!?)
|
134
138
|
def method_missing(m, *args, &block)
|
data/lib/loops/version.rb
CHANGED
@@ -1,8 +1,15 @@
|
|
1
1
|
# Contains information about currently used Loops version.
|
2
2
|
#
|
3
3
|
# @example
|
4
|
-
# puts "Loops #{Loops::
|
4
|
+
# puts "Loops #{Loops::Version::STRING}"
|
5
5
|
#
|
6
6
|
module Loops
|
7
|
-
|
7
|
+
module Version
|
8
|
+
MAJOR = 2
|
9
|
+
MINOR = 0
|
10
|
+
PATCH = 8
|
11
|
+
BUILD = nil
|
12
|
+
|
13
|
+
STRING = [MAJOR, MINOR, PATCH, BUILD].compact.join('.')
|
14
|
+
end
|
8
15
|
end
|
data/loops.gemspec
CHANGED
@@ -4,9 +4,9 @@ require 'loops/version'
|
|
4
4
|
|
5
5
|
Gem::Specification.new do |s|
|
6
6
|
s.name = 'loops'
|
7
|
-
s.version = Loops::
|
7
|
+
s.version = Loops::Version::STRING
|
8
8
|
s.platform = Gem::Platform::RUBY
|
9
|
-
s.authors = ['
|
9
|
+
s.authors = [ 'Oleksiy Kovyrin', 'Dmytro Shteflyuk' ]
|
10
10
|
s.email = %q{alexey@kovyrin.net}
|
11
11
|
s.homepage = %q{http://github.com/kovyrin/loops}
|
12
12
|
s.summary = %q{Simple background loops framework for ruby}
|
data/spec/loop_lock_spec.rb
CHANGED
@@ -0,0 +1,92 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
describe Loops::Base, '#with_lock' do
|
4
|
+
before :each do
|
5
|
+
@logger = mock('Logger').as_null_object
|
6
|
+
@loop = Loops::Base.new(@logger, 'simple', {})
|
7
|
+
end
|
8
|
+
|
9
|
+
context 'when an entity is not locked' do
|
10
|
+
it 'should create lock on an item' do
|
11
|
+
called = false
|
12
|
+
@loop.with_lock(1, 'rspec', 60) do
|
13
|
+
called = true
|
14
|
+
LoopLock.locked?(:loop => 'rspec', :entity_id => 1).should be_true
|
15
|
+
end
|
16
|
+
called.should be_true
|
17
|
+
end
|
18
|
+
|
19
|
+
it 'should release lock on an item' do
|
20
|
+
called = false
|
21
|
+
@loop.with_lock(1, 'rspec', 60) do
|
22
|
+
called = true
|
23
|
+
end
|
24
|
+
LoopLock.locked?(:loop => 'rspec', :entity_id => 1).should be_false
|
25
|
+
called.should be_true
|
26
|
+
end
|
27
|
+
|
28
|
+
it 'should release lock on an item in case of error' do
|
29
|
+
called = false
|
30
|
+
expect {
|
31
|
+
@loop.with_lock(1, 'rspec', 60) do
|
32
|
+
called = true
|
33
|
+
raise 'ouch'
|
34
|
+
end
|
35
|
+
}.to raise_error('ouch')
|
36
|
+
called.should be_true
|
37
|
+
LoopLock.locked?(:loop => 'rspec', :entity_id => 1).should be_false
|
38
|
+
end
|
39
|
+
|
40
|
+
it 'should pass the lock timeout' do
|
41
|
+
called = false
|
42
|
+
@loop.with_lock(1, 'rspec', 0.2) do
|
43
|
+
called = true
|
44
|
+
LoopLock.lock(:loop => 'rspec', :entity_id => 1).should be_false
|
45
|
+
sleep(0.2)
|
46
|
+
LoopLock.lock(:loop => 'rspec', :entity_id => 1).should be_true
|
47
|
+
end
|
48
|
+
called.should be_true
|
49
|
+
end
|
50
|
+
|
51
|
+
it 'should release the lock on an item' do
|
52
|
+
called = false
|
53
|
+
@loop.with_lock(1, 'rspec', 60) do
|
54
|
+
called = true
|
55
|
+
end
|
56
|
+
called.should be_true
|
57
|
+
LoopLock.locked?(:loop => 'rspec', :entity_id => 1).should be_false
|
58
|
+
end
|
59
|
+
|
60
|
+
it 'should yield with entity_id value if block accepts the argument' do
|
61
|
+
called = false
|
62
|
+
@loop.with_lock(1, 'rspec', 60) do |entity_id|
|
63
|
+
called = true
|
64
|
+
entity_id.should == 1
|
65
|
+
end
|
66
|
+
called.should be_true
|
67
|
+
end
|
68
|
+
end
|
69
|
+
|
70
|
+
context 'when an entity is already locked' do
|
71
|
+
before :each do
|
72
|
+
LoopLock.lock(:loop => 'rspec', :entity_id => 1)
|
73
|
+
end
|
74
|
+
|
75
|
+
it 'should should not yield' do
|
76
|
+
called = false
|
77
|
+
@loop.with_lock(1, 'rspec', 60) do
|
78
|
+
called = true
|
79
|
+
end
|
80
|
+
called.should be_false
|
81
|
+
end
|
82
|
+
|
83
|
+
it 'should should not touch the lock object' do
|
84
|
+
called = false
|
85
|
+
@loop.with_lock(1, 'rspec', 60) do
|
86
|
+
called = true
|
87
|
+
end
|
88
|
+
LoopLock.locked?(:loop => 'rspec', :entity_id => 1).should be_true
|
89
|
+
called.should be_false
|
90
|
+
end
|
91
|
+
end
|
92
|
+
end
|
data/spec/loops/base_spec.rb
CHANGED
@@ -1,92 +1,22 @@
|
|
1
|
-
require '
|
1
|
+
require 'spec_helper'
|
2
|
+
require 'active_support/core_ext/numeric/time'
|
2
3
|
|
3
|
-
describe Loops::Base
|
4
|
+
describe Loops::Base do
|
4
5
|
before :each do
|
5
6
|
@logger = mock('Logger').as_null_object
|
6
7
|
@loop = Loops::Base.new(@logger, 'simple', {})
|
7
8
|
end
|
8
|
-
|
9
|
-
context 'when an entity is not locked' do
|
10
|
-
it 'should create lock on an item' do
|
11
|
-
called = false
|
12
|
-
@loop.with_lock(1, 'rspec', 60) do
|
13
|
-
called = true
|
14
|
-
LoopLock.locked?(:loop => 'rspec', :entity_id => 1).should be_true
|
15
|
-
end
|
16
|
-
called.should be_true
|
17
|
-
end
|
18
|
-
|
19
|
-
it 'should release lock on an item' do
|
20
|
-
called = false
|
21
|
-
@loop.with_lock(1, 'rspec', 60) do
|
22
|
-
called = true
|
23
|
-
end
|
24
|
-
LoopLock.locked?(:loop => 'rspec', :entity_id => 1).should be_false
|
25
|
-
called.should be_true
|
26
|
-
end
|
27
9
|
|
28
|
-
|
29
|
-
|
30
|
-
|
31
|
-
@
|
32
|
-
|
33
|
-
|
10
|
+
context "in #with_period_of method" do
|
11
|
+
it "should coerce parameters to integer before passing them to sleep" do
|
12
|
+
[ 1, 1.seconds, "1" ].each do |period|
|
13
|
+
@test_iterations = 0
|
14
|
+
@loop.with_period_of(period) do
|
15
|
+
break if @test_iterations > 0
|
16
|
+
@test_iterations += 1
|
34
17
|
end
|
35
|
-
}.to raise_error('ouch')
|
36
|
-
called.should be_true
|
37
|
-
LoopLock.locked?(:loop => 'rspec', :entity_id => 1).should be_false
|
38
|
-
end
|
39
|
-
|
40
|
-
it 'should pass the lock timeout' do
|
41
|
-
called = false
|
42
|
-
@loop.with_lock(1, 'rspec', 0.2) do
|
43
|
-
called = true
|
44
|
-
LoopLock.lock(:loop => 'rspec', :entity_id => 1).should be_false
|
45
|
-
sleep(0.2)
|
46
|
-
LoopLock.lock(:loop => 'rspec', :entity_id => 1).should be_true
|
47
|
-
end
|
48
|
-
called.should be_true
|
49
|
-
end
|
50
|
-
|
51
|
-
it 'should release the lock on an item' do
|
52
|
-
called = false
|
53
|
-
@loop.with_lock(1, 'rspec', 60) do
|
54
|
-
called = true
|
55
18
|
end
|
56
|
-
called.should be_true
|
57
|
-
LoopLock.locked?(:loop => 'rspec', :entity_id => 1).should be_false
|
58
|
-
end
|
59
|
-
|
60
|
-
it 'should yield with entity_id value if block accepts the argument' do
|
61
|
-
called = false
|
62
|
-
@loop.with_lock(1, 'rspec', 60) do |entity_id|
|
63
|
-
called = true
|
64
|
-
entity_id.should == 1
|
65
|
-
end
|
66
|
-
called.should be_true
|
67
19
|
end
|
68
20
|
end
|
69
21
|
|
70
|
-
context 'when an entity is already locked' do
|
71
|
-
before :each do
|
72
|
-
LoopLock.lock(:loop => 'rspec', :entity_id => 1)
|
73
|
-
end
|
74
|
-
|
75
|
-
it 'should should not yield' do
|
76
|
-
called = false
|
77
|
-
@loop.with_lock(1, 'rspec', 60) do
|
78
|
-
called = true
|
79
|
-
end
|
80
|
-
called.should be_false
|
81
|
-
end
|
82
|
-
|
83
|
-
it 'should should not touch the lock object' do
|
84
|
-
called = false
|
85
|
-
@loop.with_lock(1, 'rspec', 60) do
|
86
|
-
called = true
|
87
|
-
end
|
88
|
-
LoopLock.locked?(:loop => 'rspec', :entity_id => 1).should be_true
|
89
|
-
called.should be_false
|
90
|
-
end
|
91
|
-
end
|
92
22
|
end
|
data/spec/loops/cli_spec.rb
CHANGED
data/spec/loops_spec.rb
CHANGED
@@ -1,4 +1,4 @@
|
|
1
|
-
require '
|
1
|
+
require 'spec_helper'
|
2
2
|
|
3
3
|
describe Loops do
|
4
4
|
describe '.load_config' do
|
@@ -6,7 +6,7 @@ describe Loops do
|
|
6
6
|
Loops.root = RAILS_ROOT
|
7
7
|
@engine = Loops::Engine.new
|
8
8
|
end
|
9
|
-
|
9
|
+
|
10
10
|
it 'should load and parse Loops configuration file' do
|
11
11
|
@engine.config.should be_an_instance_of(Hash)
|
12
12
|
@engine.global_config.should be_an_instance_of(Hash)
|
metadata
CHANGED
@@ -1,22 +1,22 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: loops
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
hash:
|
4
|
+
hash: 31
|
5
5
|
prerelease:
|
6
6
|
segments:
|
7
7
|
- 2
|
8
8
|
- 0
|
9
|
-
-
|
10
|
-
version: 2.0.
|
9
|
+
- 8
|
10
|
+
version: 2.0.8
|
11
11
|
platform: ruby
|
12
12
|
authors:
|
13
|
-
-
|
13
|
+
- Oleksiy Kovyrin
|
14
14
|
- Dmytro Shteflyuk
|
15
15
|
autorequire:
|
16
16
|
bindir: bin
|
17
17
|
cert_chain: []
|
18
18
|
|
19
|
-
date:
|
19
|
+
date: 2012-05-18 00:00:00 Z
|
20
20
|
dependencies:
|
21
21
|
- !ruby/object:Gem::Dependency
|
22
22
|
name: rspec
|
@@ -59,6 +59,7 @@ extra_rdoc_files:
|
|
59
59
|
files:
|
60
60
|
- .gitignore
|
61
61
|
- .rspec
|
62
|
+
- .travis.yml
|
62
63
|
- Gemfile
|
63
64
|
- LICENSE
|
64
65
|
- README.rdoc
|
@@ -95,6 +96,7 @@ files:
|
|
95
96
|
- lib/loops/worker_pool.rb
|
96
97
|
- loops.gemspec
|
97
98
|
- spec/loop_lock_spec.rb
|
99
|
+
- spec/loops/base/with_lock_spec.rb
|
98
100
|
- spec/loops/base_spec.rb
|
99
101
|
- spec/loops/cli_spec.rb
|
100
102
|
- spec/loops_spec.rb
|
@@ -136,12 +138,13 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
136
138
|
requirements: []
|
137
139
|
|
138
140
|
rubyforge_project:
|
139
|
-
rubygems_version: 1.8.
|
141
|
+
rubygems_version: 1.8.23
|
140
142
|
signing_key:
|
141
143
|
specification_version: 3
|
142
144
|
summary: Simple background loops framework for ruby
|
143
145
|
test_files:
|
144
146
|
- spec/loop_lock_spec.rb
|
147
|
+
- spec/loops/base/with_lock_spec.rb
|
145
148
|
- spec/loops/base_spec.rb
|
146
149
|
- spec/loops/cli_spec.rb
|
147
150
|
- spec/loops_spec.rb
|
@@ -154,3 +157,4 @@ test_files:
|
|
154
157
|
- spec/rails/config/init.rb
|
155
158
|
- spec/rails/config/loops.yml
|
156
159
|
- spec/spec_helper.rb
|
160
|
+
has_rdoc:
|