deferrer 0.0.2 → 0.1.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 3177ff9473fe3ddecdc6579a2adcde3fa02112f5
4
- data.tar.gz: de0bbbd3540ec14e50061ed917d633dd27be62d1
3
+ metadata.gz: 47517fb6ffc62ca2b600c64f1a0a5ae5e2379239
4
+ data.tar.gz: 06110ea8a1dbdf20002f5627798a33ac5fd550e8
5
5
  SHA512:
6
- metadata.gz: 5db8302690c0707b3d0ce6ae0ef03c87e26be05024565bd579121deb222cc4987788fd832d7366842a1d56e32acaea1abfc9702fef965c5c513e05427522f04c
7
- data.tar.gz: f3cd63d60f6d37c4275781fb58a0ace53dcb730b5af152744de7059000ec4d2f4b9fc84785e5bbe45efc05d51ace50bf893a84938b18b8ddee1d1ee99f8620a2
6
+ metadata.gz: 47f4549a640efb122f2726e3fdf052c7f9998f748d978f608cd440f74c8b5b352a157918423a5001d4f5688c617ca30a3fa89256c5d879e80e4cbece5cade884
7
+ data.tar.gz: 1f330b0d1a8e6bae1a27dbb66fb0c69b5c0b83ca1818d4a3012ff2ef11738f34ff032753f55701d39548ce887cff9082a2f0a50cea7af87e620f0ed875f0ad3f
data/README.md CHANGED
@@ -2,64 +2,86 @@
2
2
 
3
3
  # Deferrer
4
4
 
5
- Defer executions and run only the last update at the scheduled time
6
-
5
+ Deferrer is a library for deferring work units for a time period or to a specific time. When time reaches, only the last work unit will be processed. Usually, the last work unit should be the one that summarizes all the previous ones. An example scenario would be sending live updates that happen *very* frequently and we want to limit them by sending an update every x seconds.
7
6
 
8
7
  ## Installation
9
8
 
10
9
  Add this line to your application's Gemfile:
11
10
 
12
- gem 'deferrer'
11
+ ```
12
+ gem 'deferrer'
13
+ ```
13
14
 
14
15
  And then execute:
15
16
 
16
- $ bundle install
17
+ ```
18
+ $ bundle install
19
+ ```
17
20
 
18
21
  Or install it yourself as:
19
22
 
20
- $ gem install deferrer
21
-
23
+ ```
24
+ $ gem install deferrer
25
+ ```
22
26
 
23
27
 
24
28
  ## Usage
25
29
 
26
- Configure redis
30
+ Configure deferrer (redis connection, logger)
31
+
32
+ ```ruby
33
+ Deferrer.redis_config = { :host => "localhost", :port => 6379 }
34
+ Deferrer.logger = Logger.new(STDOUT)
35
+ ```
36
+
37
+
38
+ Define deferrer worker that must respond to call method
39
+
40
+ ```ruby
41
+ Deferrer.worker = lambda do |klass, *args|
42
+ # do some work
43
+ # Resque.enqueue(klass, *args)
44
+ end
45
+ ```
27
46
 
28
- Deferrer.redis_config = { :host => "localhost", :port => 6379 }
47
+ Deferrer is usually used in combination with background processing tools like sidekiq and resque. If that's the case, Deferrer.worker can be light-weight and responsible only for pushing work to a background job.
29
48
 
30
49
 
31
- Define deferrer class (must have perform class method)
50
+ Start a worker process.
32
51
 
33
- class NameDeferrer
34
- def self.perform(first_name, last_name)
35
- puts "#{first_name} #{last_name}".upcase
36
- end
37
- end
52
+ ```ruby
53
+ Deferrer.run(options = {})
38
54
 
55
+ # Following `options` are available:
56
+ # loop_frequency - sleep between loops, default to 0.1 seconds
57
+ # single_run - process items only for a single loop, useful for testing
58
+ # ignore_time - don't wait for time period to expire, useful for testing
59
+ ```
39
60
 
40
- Start a worker process. It needs to have redis configured and access to deferrer classes.
41
61
 
42
- Deferrer.run(options = {})
62
+ Defer some executions:
43
63
 
44
- # Following `options` are available:
45
- # loop_frequency - sleep between loops, default to 0.1 seconds
46
- # logger - logging mechanism, needs to respond to `info` and `error`
47
- # before_each - callback to run before processing an item, needs to respond to `call`
48
- # after_each - callback to run after processing an item, needs to respond to `call`
49
- # single_run - process items only for a single loop, useful for testing
64
+ ```ruby
65
+ Deferrer.defer_in(5, 'identifier', Worker, 'update 1')
66
+ Deferrer.defer_in(6, 'identifier', Worker, 'update 2')
67
+ Deferrer.defer_in(9, 'identifier', Worker, 'update 3')
68
+ ```
50
69
 
51
70
 
52
- Defer some executions
71
+ It will stack all defered executions per identifier until first timeout expires (5 seconds) and then it will only execute the last update for the expired identifier, calling the deferrer worker:
53
72
 
54
- Deferrer.defer_in(5, 'identifier', NameDeferrer, 'User', '1')
55
- Deferrer.defer_in(6, 'identifier', NameDeferrer, 'User', '2')
56
- Deferrer.defer_in(9, 'identifier', NameDeferrer, 'User', '3')
73
+ ```ruby
74
+ Deferrer.worker.call('Worker', 'update 3')
75
+ ```
57
76
 
58
77
 
59
- It will stack all defered executions per identifier until first timeout expires (5 seconds) and then it will only execute the last update for the expired identifier:
78
+ ## Testing
60
79
 
61
- NameDeferrer.perform('User', '3') => USER 3
80
+ For testing, two options of the `run` method are useful. `single_run` will run the loop only once and `ignore_time` will not wait for time period to expire but execute to job now.
62
81
 
82
+ ```ruby
83
+ Deferrer.run(single_run: true, ignore_time: true)
84
+ ```
63
85
 
64
86
 
65
87
  ## Contributing
@@ -8,8 +8,8 @@ Gem::Specification.new do |spec|
8
8
  spec.version = Deferrer::VERSION
9
9
  spec.authors = ["Dalibor Nasevic"]
10
10
  spec.email = ["dalibor.nasevic@gmail.com"]
11
- spec.description = %q{Defer executions and run only the last update}
12
- spec.summary = %q{Defer executions and run only the last update at the scheduled time}
11
+ spec.description = %q{Defer work units and process only the last one}
12
+ spec.summary = %q{Defer work units and process only the last work unit when time comes}
13
13
  spec.homepage = ""
14
14
  spec.license = "MIT"
15
15
 
@@ -22,5 +22,5 @@ Gem::Specification.new do |spec|
22
22
  spec.add_dependency "multi_json"
23
23
  spec.add_development_dependency "bundler", "~> 1.3"
24
24
  spec.add_development_dependency "rake"
25
- spec.add_development_dependency "rspec", "~> 2.14.1"
25
+ spec.add_development_dependency "rspec", "~> 3.0.0"
26
26
  end
@@ -1,8 +1,6 @@
1
1
  require 'deferrer'
2
- require_relative './name_deferrer'
2
+ require_relative './common'
3
3
 
4
- Deferrer.redis_config = { :host => "localhost", :port => 6379 }
5
-
6
- Deferrer.defer_in(5, 'identifier', NameDeferrer, 'User', '1')
7
- Deferrer.defer_in(6, 'identifier', NameDeferrer, 'User', '2')
8
- Deferrer.defer_in(9, 'identifier', NameDeferrer, 'User', '3')
4
+ 1.upto(5) do |i|
5
+ Deferrer.defer_in(i + 3, 'identifier', "update #{i}")
6
+ end
@@ -0,0 +1,2 @@
1
+ # setup redis
2
+ Deferrer.redis_config = { :host => "localhost", :port => 6379, db: 15 }
@@ -1,21 +1,14 @@
1
1
  require 'deferrer'
2
- require_relative './name_deferrer'
3
-
4
- Deferrer.redis_config = { :host => "localhost", :port => 6379 }
5
-
6
- class Logger
7
- def self.info(message)
8
- puts "INFO: #{message}"
9
- end
10
-
11
- def self.error(message)
12
- puts "ERROR: #{message}"
13
- end
14
- end
2
+ require 'logger'
3
+ require_relative './common'
15
4
 
16
5
  puts 'Runner started'
17
6
 
7
+ Deferrer.redis_config = { :host => "localhost", :port => 6379, :db => 15 }
8
+ Deferrer.logger = Logger.new(STDOUT)
9
+ Deferrer.worker = lambda do |*args|
10
+ p args
11
+ end
18
12
  Deferrer.run({
19
13
  :loop_frequency => 0.5,
20
- :logger => Logger
21
14
  })
@@ -2,12 +2,21 @@ require 'redis'
2
2
  require "deferrer/version"
3
3
 
4
4
  module Deferrer
5
-
6
5
  autoload :Configuration, 'deferrer/configuration'
7
6
  autoload :JsonEncoding, 'deferrer/json_encoding'
8
- autoload :Deferral, 'deferrer/deferral'
7
+ autoload :Runner, 'deferrer/runner'
8
+ autoload :Job, 'deferrer/job'
9
+
10
+ LIST_KEY = 'deferred_list'
11
+ ITEM_KEY_PREFIX = 'deferred'
9
12
 
10
13
  extend Configuration
11
14
  extend JsonEncoding
12
- extend Deferral
15
+ extend Runner
16
+
17
+ class WorkerNotConfigured < NotImplementedError
18
+ def initialize
19
+ super("Deferrer worker not configured")
20
+ end
21
+ end
13
22
  end
@@ -1,14 +1,13 @@
1
1
  module Deferrer
2
2
  module Configuration
3
3
 
4
+ attr_reader :redis
5
+ attr_accessor :logger
6
+ attr_accessor :worker
7
+
4
8
  # Deferrer.redis_config = { :host => "localhost", :port => 6379 }
5
9
  def redis_config=(config)
6
10
  @redis = Redis.new(config)
7
11
  end
8
-
9
- # Returns the configured Redis instance
10
- def redis
11
- @redis
12
- end
13
12
  end
14
13
  end
@@ -3,18 +3,12 @@ require 'multi_json'
3
3
  module Deferrer
4
4
  module JsonEncoding
5
5
 
6
- class DecodeException < StandardError; end
7
-
8
6
  def encode(item)
9
7
  MultiJson.dump(item)
10
8
  end
11
9
 
12
10
  def decode(item)
13
- begin
14
- MultiJson.load(item)
15
- rescue ::MultiJson::DecodeError => e
16
- raise DecodeException, e.message, e.backtrace
17
- end
11
+ MultiJson.load(item)
18
12
  end
19
13
  end
20
14
  end
@@ -0,0 +1,103 @@
1
+ module Deferrer
2
+ module Runner
3
+ def run(options = {})
4
+ loop_frequency = options.fetch(:loop_frequency, 0.1)
5
+ single_run = options.fetch(:single_run, false)
6
+ @ignore_time = options.fetch(:ignore_time, false)
7
+
8
+ raise WorkerNotConfigured unless worker
9
+
10
+ loop do
11
+ begin
12
+ while item = next_item
13
+ process_item(item)
14
+ end
15
+ rescue StandardError => e
16
+ log(:error, "Error: #{e.class}: #{e.message}")
17
+ rescue Exception => e
18
+ log(:error, "Error: #{e.class}: #{e.message}")
19
+ raise
20
+ end
21
+
22
+ break if single_run
23
+ sleep loop_frequency
24
+ end
25
+ end
26
+
27
+ def next_item
28
+ item = nil
29
+ decoded_item = nil
30
+ key = next_key
31
+
32
+ if key
33
+ item = redis.rpop(key)
34
+ if item
35
+ decoded_item = decode(item)
36
+ decoded_item['key'] = key
37
+ end
38
+
39
+ remove(key)
40
+ end
41
+
42
+ decoded_item
43
+ end
44
+
45
+ def defer_in(number_of_seconds_from_now, identifier, *args)
46
+ timestamp = Time.now + number_of_seconds_from_now
47
+ defer_at(timestamp, identifier, *args)
48
+ end
49
+
50
+ def defer_at(timestamp, identifier, *args)
51
+ key = item_key(identifier)
52
+ item = { 'args' => args }
53
+
54
+ push_item(key, item, timestamp)
55
+ end
56
+
57
+ private
58
+ def process_item(item)
59
+ log(:info, "Processing: #{item['key']}")
60
+ worker.call(*item['args'])
61
+ end
62
+
63
+ def item_key(identifier)
64
+ "#{ITEM_KEY_PREFIX}:#{identifier}"
65
+ end
66
+
67
+ def push_item(key, item, timestamp)
68
+ count = redis.rpush(key, encode(item))
69
+
70
+ # set score only on first update
71
+ if count == 1
72
+ score = calculate_score(timestamp)
73
+ redis.zadd(LIST_KEY, score, key)
74
+ end
75
+ end
76
+
77
+ def next_key
78
+ if @ignore_time
79
+ redis.zrange(LIST_KEY, 0, 1).first
80
+ else
81
+ score = calculate_score(Time.now)
82
+ redis.zrangebyscore(LIST_KEY, '-inf', score, :limit => [0, 1]).first
83
+ end
84
+ end
85
+
86
+ def calculate_score(timestamp)
87
+ timestamp.to_f
88
+ end
89
+
90
+ def remove(key)
91
+ redis.watch(key)
92
+ redis.multi do
93
+ redis.del(key)
94
+ redis.zrem(LIST_KEY, key)
95
+ end
96
+ redis.unwatch
97
+ end
98
+
99
+ def log(type, message)
100
+ logger.send(type, message) if logger
101
+ end
102
+ end
103
+ end
@@ -1,3 +1,3 @@
1
1
  module Deferrer
2
- VERSION = "0.0.2"
2
+ VERSION = "0.1.0"
3
3
  end
@@ -0,0 +1,145 @@
1
+ require 'spec_helper'
2
+ require 'logger'
3
+
4
+ describe Deferrer::Runner do
5
+ let(:identifier) { 'some_identifier' }
6
+ let(:redis) { Deferrer.redis }
7
+ let(:list_key) { Deferrer::LIST_KEY }
8
+ let(:logger) { Logger.new(STDOUT) }
9
+
10
+ describe ".run" do
11
+ it "processes jobs" do
12
+ expect(Deferrer.worker).to receive(:call).with('TestWorker', 'test')
13
+
14
+ Deferrer.defer_in(-1, identifier, 'TestWorker', 'test')
15
+ Deferrer.run(single_run: true)
16
+ end
17
+
18
+ it "correctly sets arguments and converts symbols to strings for hashes" do
19
+ expect(Deferrer.worker).to receive(:call).with('TestWorker', 1, 'arg1', { "a" => "b"})
20
+
21
+ Deferrer.defer_in(-1, identifier, 'TestWorker', 1, 'arg1', { a: :b })
22
+ Deferrer.run(single_run: true)
23
+ end
24
+
25
+ it "rescues standard errors" do
26
+ allow(Deferrer).to receive(:next_item) { raise RuntimeError.new('error') }
27
+
28
+ Deferrer.defer_in(-1, identifier, 'TestWorker', 'test')
29
+ Deferrer.run(single_run: true)
30
+ end
31
+
32
+ it "rescues exceptions and logs error messages" do
33
+ expect(logger).to receive(:error).with("Error: Exception: error")
34
+ allow(Deferrer).to receive(:next_item) { raise Exception.new('error') }
35
+
36
+ Deferrer.logger = logger
37
+ Deferrer.defer_in(-1, identifier, 'TestWorker', 'test')
38
+ expect { Deferrer.run(single_run: true) }.to raise_error(Exception)
39
+ end
40
+
41
+ it "raises error if worker not configured" do
42
+ Deferrer.worker = nil
43
+ Deferrer.defer_in(-1, identifier, 'TestWorker', 'test')
44
+
45
+ expect { Deferrer.run(single_run: true) }.to raise_error(Deferrer::WorkerNotConfigured)
46
+ end
47
+
48
+ it "ignores time to wait and performs jobs" do
49
+ expect(Deferrer.worker).to receive(:call).with('Worker', { "c" => "d"})
50
+
51
+ Deferrer.defer_in(100, identifier, 'Worker', { a: :b })
52
+ Deferrer.defer_in(100, identifier, 'Worker', { c: :d })
53
+
54
+ Deferrer.run(single_run: true, ignore_time: true)
55
+ end
56
+ end
57
+
58
+ describe ".defer_at" do
59
+ it "deferrs at given time" do
60
+ Deferrer.defer_at(Time.now, identifier, 'TestWorker', 'test')
61
+
62
+ expect(redis.zrangebyscore(list_key, '-inf', Time.now.to_f, :limit => [0, 1]).first).not_to be_nil
63
+ expect(redis.exists(item_key(identifier))).to be_truthy
64
+ end
65
+ end
66
+
67
+ describe ".defer_in" do
68
+ it "defers in given interval" do
69
+ Deferrer.defer_in(1, identifier, 'TestWorker', 'test')
70
+
71
+ expect(redis.zrangebyscore(list_key, '-inf', (Time.now + 1).to_f, :limit => [0, 1]).first).not_to be_nil
72
+ expect(redis.exists(item_key(identifier))).to be_truthy
73
+ end
74
+ end
75
+
76
+ describe ".next_item" do
77
+ it "returns the next item" do
78
+ Deferrer.defer_at(Time.now, identifier, 'TestWorker', 'test')
79
+
80
+ item = Deferrer.next_item
81
+
82
+ expect(item['args']).to eq(['TestWorker', 'test'])
83
+ end
84
+
85
+ it "returns last update of an item" do
86
+ Deferrer.defer_at(Time.now - 3, identifier, 'TestWorker', 'update1')
87
+ Deferrer.defer_at(Time.now - 2, identifier, 'TestWorker', 'update2')
88
+
89
+ item = Deferrer.next_item
90
+
91
+ expect(item['args']).to eq(['TestWorker', 'update2'])
92
+ end
93
+
94
+ it "returns nil when no next item" do
95
+ expect(Deferrer.next_item).to be_nil
96
+ end
97
+
98
+ it "removes values from redis" do
99
+ Deferrer.defer_at(Time.now, identifier, 'TestWorker', 'test')
100
+
101
+ item = Deferrer.next_item
102
+
103
+ expect(redis.zrangebyscore(list_key, '-inf', Time.now.to_f, :limit => [0, 1]).first).to be_nil
104
+ expect(redis.exists(item_key(identifier))).to be_falsey
105
+ expect(Deferrer.next_item).to be_nil
106
+ end
107
+
108
+ it "doesn't block on empty lists" do
109
+ Deferrer.defer_in(-1, identifier, 'TestWorker', 'test')
110
+ redis.del(item_key(identifier))
111
+
112
+ expect(Deferrer.next_item).to be_nil
113
+ expect(redis.zrangebyscore(list_key, '-inf', 'inf', :limit => [0, 1]).first).to be_nil
114
+ end
115
+ end
116
+
117
+ describe ".logger" do
118
+ before :each do
119
+ Deferrer.logger = logger
120
+ end
121
+
122
+ it "logs info messages" do
123
+ expect(logger).to receive(:info).with("Processing: deferred:#{identifier}")
124
+
125
+ Deferrer.defer_in(-1, identifier, 'TestWorker', 'test')
126
+ Deferrer.run(single_run: true)
127
+ end
128
+
129
+ it "logs error messages" do
130
+ expect(logger).to receive(:error).with("Error: RuntimeError: error")
131
+ allow(Deferrer).to receive(:next_item) { raise RuntimeError.new('error') }
132
+
133
+ Deferrer.defer_in(-1, identifier, 'TestWorker', 'test')
134
+ Deferrer.run(single_run: true)
135
+ end
136
+
137
+ it "logs error messages on exceptions" do
138
+ expect(logger).to receive(:error).with("Error: Exception: error")
139
+ allow(Deferrer).to receive(:next_item) { raise Exception.new('error') }
140
+
141
+ Deferrer.defer_in(-1, identifier, 'TestWorker', 'test')
142
+ expect { Deferrer.run(single_run: true) }.to raise_error
143
+ end
144
+ end
145
+ end
@@ -1,3 +1,21 @@
1
1
  require 'deferrer'
2
2
 
3
3
  Deferrer.redis_config = { :host => "localhost", :port => 6379 }
4
+
5
+
6
+ ROOT = File.expand_path('../', File.dirname(__FILE__))
7
+
8
+ # Load support files
9
+ Dir["#{ROOT}/spec/support/**/*.rb"].each { |f| require f }
10
+
11
+
12
+ RSpec.configure do |config|
13
+ config.include Helpers
14
+
15
+ config.before :each do
16
+ Deferrer.redis.flushdb
17
+ Deferrer.logger = nil
18
+ Deferrer.worker = lambda { |klass, *args| }
19
+ end
20
+ end
21
+
@@ -0,0 +1,5 @@
1
+ module Helpers
2
+ def item_key(identifier)
3
+ "#{Deferrer::ITEM_KEY_PREFIX}:#{identifier}"
4
+ end
5
+ end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: deferrer
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.2
4
+ version: 0.1.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Dalibor Nasevic
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2014-07-08 00:00:00.000000000 Z
11
+ date: 2014-07-28 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: redis
@@ -72,15 +72,15 @@ dependencies:
72
72
  requirements:
73
73
  - - "~>"
74
74
  - !ruby/object:Gem::Version
75
- version: 2.14.1
75
+ version: 3.0.0
76
76
  type: :development
77
77
  prerelease: false
78
78
  version_requirements: !ruby/object:Gem::Requirement
79
79
  requirements:
80
80
  - - "~>"
81
81
  - !ruby/object:Gem::Version
82
- version: 2.14.1
83
- description: Defer executions and run only the last update
82
+ version: 3.0.0
83
+ description: Defer work units and process only the last one
84
84
  email:
85
85
  - dalibor.nasevic@gmail.com
86
86
  executables: []
@@ -97,15 +97,16 @@ files:
97
97
  - deferrer.gemspec
98
98
  - example/README.md
99
99
  - example/client.rb
100
- - example/name_deferrer.rb
100
+ - example/common.rb
101
101
  - example/runner.rb
102
102
  - lib/deferrer.rb
103
103
  - lib/deferrer/configuration.rb
104
- - lib/deferrer/deferral.rb
105
104
  - lib/deferrer/json_encoding.rb
105
+ - lib/deferrer/runner.rb
106
106
  - lib/deferrer/version.rb
107
- - spec/deferrer/deferral_spec.rb
107
+ - spec/deferrer/runner_spec.rb
108
108
  - spec/spec_helper.rb
109
+ - spec/support/helpers.rb
109
110
  homepage: ''
110
111
  licenses:
111
112
  - MIT
@@ -129,8 +130,9 @@ rubyforge_project:
129
130
  rubygems_version: 2.2.0
130
131
  signing_key:
131
132
  specification_version: 4
132
- summary: Defer executions and run only the last update at the scheduled time
133
+ summary: Defer work units and process only the last work unit when time comes
133
134
  test_files:
134
- - spec/deferrer/deferral_spec.rb
135
+ - spec/deferrer/runner_spec.rb
135
136
  - spec/spec_helper.rb
137
+ - spec/support/helpers.rb
136
138
  has_rdoc:
@@ -1,9 +0,0 @@
1
- # setup redis
2
- Deferrer.redis_config = { :host => "localhost", :port => 6379 }
3
-
4
- # define deferrer class (must have perform class method)
5
- class NameDeferrer
6
- def self.perform(first_name, last_name)
7
- puts "#{first_name} #{last_name}".upcase
8
- end
9
- end
@@ -1,100 +0,0 @@
1
- module Deferrer
2
- module Deferral
3
-
4
- LIST_KEY = :deferred_list
5
-
6
- def run(options = {})
7
- loop_frequency = options.fetch(:loop_frequency, 0.1)
8
- logger = options.fetch(:logger, nil)
9
- before_each = options.fetch(:before_each, nil)
10
- after_each = options.fetch(:after_each, nil)
11
- single_run = options.fetch(:single_run, false)
12
-
13
- loop do
14
- while item = next_item
15
- process_item(item, logger, before_each, after_each)
16
- end
17
-
18
- break if single_run
19
- sleep loop_frequency
20
- end
21
- end
22
-
23
- def next_item
24
- item = nil
25
- decoded_item = nil
26
- score = calculate_score(Time.now)
27
-
28
- key = redis.zrangebyscore(LIST_KEY, '-inf', score, :limit => [0, 1]).first
29
-
30
- if key
31
- item = redis.rpop(key)
32
- if item
33
- decoded_item = decode(item)
34
- decoded_item['key'] = key
35
- end
36
-
37
- remove(key)
38
- end
39
-
40
- decoded_item
41
- end
42
-
43
- def constantize(klass_string)
44
- klass_string.split('::').inject(Object) {|memo,name| memo = memo.const_get(name); memo}
45
- end
46
-
47
- def defer_in(number_of_seconds_from_now, identifier, klass, *args)
48
- timestamp = Time.now + number_of_seconds_from_now
49
- defer_at(timestamp, identifier, klass, *args)
50
- end
51
-
52
- def defer_at(timestamp, identifier, klass, *args)
53
- item = build_item(klass, args)
54
- key = item_key(identifier)
55
- score = calculate_score(timestamp)
56
-
57
- count = redis.rpush(key, encode(item))
58
-
59
- # set score only on first update
60
- if count == 1
61
- redis.zadd(LIST_KEY, score, key)
62
- end
63
- end
64
-
65
- def item_key(identifier)
66
- "deferred:#{identifier}"
67
- end
68
-
69
- private
70
- def process_item(item, logger, before_each, after_each)
71
- before_each.call if before_each
72
- klass = constantize(item['class'])
73
- args = item['args']
74
-
75
- logger.info("Executing: #{item['key']}") if logger
76
-
77
- klass.send(:perform, *args)
78
- after_each.call if after_each
79
- rescue Exception => e
80
- logger.error("Error: #{e.class}: #{e.message}") if logger
81
- end
82
-
83
- def build_item(klass, args)
84
- {'class' => klass.to_s, 'args' => args}
85
- end
86
-
87
- def calculate_score(timestamp)
88
- timestamp.to_f
89
- end
90
-
91
- def remove(key)
92
- redis.watch(key)
93
- redis.multi do
94
- redis.del(key)
95
- redis.zrem(LIST_KEY, key)
96
- end
97
- redis.unwatch
98
- end
99
- end
100
- end
@@ -1,133 +0,0 @@
1
- require 'spec_helper'
2
- require 'timeout'
3
-
4
- class CarDeferrer
5
- def self.perform(car)
6
- end
7
-
8
- def self.callback
9
- end
10
- end
11
-
12
- class Logger
13
- def self.info(message)
14
- end
15
-
16
- def self.error(message)
17
- end
18
- end
19
-
20
- class InvalidLogger
21
- def self.error(message)
22
- end
23
- end
24
-
25
- describe Deferrer::Deferral do
26
- let(:car) { 'car' }
27
- let(:car2) { 'car2' }
28
- let(:identifier) { 'car1' }
29
- let(:redis) { Deferrer.redis }
30
- let(:list_key) { Deferrer::Deferral::LIST_KEY }
31
-
32
- before :each do
33
- redis.flushdb
34
- end
35
-
36
- describe "run" do
37
- it "processes jobs" do
38
- CarDeferrer.should_receive(:perform).with(car)
39
- Deferrer.defer_in(-1, identifier, CarDeferrer, car)
40
- Deferrer.run(single_run: true)
41
- end
42
-
43
- it "logs info messages if logger provided" do
44
- Logger.should_receive(:info).with("Executing: deferred:#{identifier}")
45
- Deferrer.defer_in(-1, identifier, CarDeferrer, car)
46
- Deferrer.run(single_run: true, logger: Logger)
47
- end
48
-
49
- it "logs error messages if logger provided" do
50
- InvalidLogger.should_receive(:error).with("Error: NoMethodError: undefined method `info' for InvalidLogger:Class")
51
- Deferrer.defer_in(-1, identifier, CarDeferrer, car)
52
- Deferrer.run(single_run: true, logger: InvalidLogger)
53
- end
54
-
55
- it "runs before callback" do
56
- CarDeferrer.should_receive(:callback)
57
- Deferrer.defer_in(-1, identifier, CarDeferrer, car)
58
- Deferrer.run(single_run: true, before_each: Proc.new { CarDeferrer.callback })
59
- end
60
-
61
- it "runs after callback" do
62
- CarDeferrer.should_receive(:callback)
63
- Deferrer.defer_in(-1, identifier, CarDeferrer, car)
64
- Deferrer.run(single_run: true, after_each: Proc.new { CarDeferrer.callback })
65
- end
66
- end
67
-
68
- describe ".defer_at" do
69
- it "deferrs at given time" do
70
- Deferrer.defer_at(Time.now, identifier, CarDeferrer, car)
71
-
72
- redis.zrangebyscore(list_key, '-inf', Time.now.to_f, :limit => [0, 1]).first.should_not be_nil
73
- redis.exists(Deferrer.item_key(identifier)).should be_true
74
- end
75
-
76
- it "defers in given interval" do
77
- Deferrer.defer_in(1, identifier, CarDeferrer, car)
78
-
79
- redis.zrangebyscore(list_key, '-inf', (Time.now + 1).to_f, :limit => [0, 1]).first.should_not be_nil
80
- redis.exists(Deferrer.item_key(identifier)).should be_true
81
- end
82
- end
83
-
84
- describe ".next_item" do
85
- it "returns the next item" do
86
- Deferrer.defer_at(Time.now, identifier, CarDeferrer, car)
87
-
88
- item = Deferrer.next_item
89
-
90
- item['class'].should == CarDeferrer.to_s
91
- item['args'].should == [car]
92
- end
93
-
94
- it "returns last update of an item" do
95
- Deferrer.defer_at(Time.now - 3, identifier, CarDeferrer, car)
96
- Deferrer.defer_at(Time.now - 2, identifier, CarDeferrer, car2)
97
-
98
- item = Deferrer.next_item
99
-
100
- item['class'].should == CarDeferrer.to_s
101
- item['args'].should == [car2]
102
- end
103
-
104
- it "keep the old score value" do
105
- Deferrer.defer_at(Time.now - 3, identifier, CarDeferrer, car)
106
- Deferrer.defer_at(Time.now + 1, identifier, CarDeferrer, car2)
107
-
108
- Deferrer.next_item.should_not be_nil
109
- end
110
-
111
- it "returns nil when no next item" do
112
- Deferrer.next_item.should be_nil
113
- end
114
-
115
- it "removes values from redis" do
116
- Deferrer.defer_at(Time.now, identifier, CarDeferrer, car)
117
-
118
- item = Deferrer.next_item
119
-
120
- redis.zrangebyscore(list_key, '-inf', Time.now.to_f, :limit => [0, 1]).first.should be_nil
121
- redis.exists(Deferrer.item_key(identifier)).should be_false
122
- Deferrer.next_item.should be_nil
123
- end
124
-
125
- it "doesn't block on empty lists" do
126
- Deferrer.defer_in(-1, identifier, CarDeferrer, car)
127
- redis.del Deferrer.item_key(identifier)
128
-
129
- Timeout::timeout(2) { Deferrer.next_item.should be_nil }
130
- redis.zrangebyscore(list_key, '-inf', 'inf', :limit => [0, 1]).first.should be_nil
131
- end
132
- end
133
- end