wind_up 0.0.2 → 0.0.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 79c283c998f19e33759901fca41cfb5c00219db2
4
- data.tar.gz: 739bf5be6f00edba35343cc8f1fbf9a15b909ad8
3
+ metadata.gz: b7c5a7df34c4d2083d84ab27b16a1535a645d12a
4
+ data.tar.gz: dbde5970c7f5437e9d38755b454d552f6cdfd92a
5
5
  SHA512:
6
- metadata.gz: 6ae64913bcdd2eebcc0bd26f12a8c0649d259b44cb3bb191a35f631c842604791d7111a36a60c62f28d336ca17ed221304f55aad8bcdf83e5085defc0be1c98e
7
- data.tar.gz: e13d341414cb2b0ad0c65cdf7757648bbd37110dcc4e5435c782205e81d01153f9c42cab2907d32b38709574218155f138085100c0ae5b915917f4dea820a92a
6
+ metadata.gz: 1e44db4d841a2dfd63495b9918eacb9a498826b641812f62a99ed04abd7e802adcf9d721c3f3e74b330174248fd98d17226fe252281cc04f821e7dc7f8a4b27e
7
+ data.tar.gz: 7b476af5f07282981a26aa38bab7e415ca5afd618b6c523293273a461aa7edf5296eac2e3d0a1d6dff04f997f3ed5b7553cd83a9da3a033655650c441be3e638
data/README.md CHANGED
@@ -1,87 +1,67 @@
1
- Wind Up
2
- =======
3
- WindUp is a drop-in replacement for Celluloid's `PoolManager` class. So why
4
- would you use WindUp?
5
-
6
- * Asynchronous message passing - get all the nice worker-level concurrency
7
- Celluloid give you (#sleep, #Celluloid::IO, #future, etc)
8
- * Separate proxies for QueueManager and queues - no more unexpected behavior
9
- between #is_a? and #class
10
- * Single queue handles multiple workers - Extend WindUp with
11
- [MultiWindUp](https://www.github.com/ryanlchan/multi_wind_up) and you can
12
- have one pool execute multiple types of workers simultaneously
1
+ WindUp
2
+ ===========
13
3
 
14
- Usage
15
- -----
16
-
17
- WindUp `Queues` are almost drop-in replacements for Celluloid pools.
4
+ WindUp allows you to do super simple background processing using Celluloid
5
+ Actors.
18
6
 
7
+ Installation
8
+ ------------
19
9
  ```ruby
20
- q = AnyCelluloidClass.queue size: 3 # size defaults to number of cores
21
- q.any_method # perform synchronously
22
- q.async.long_running_method # perform asynchronously
23
- q.future.i_want_this_back # perform as a future
10
+ # Gemspec
11
+ gem 'wind_up'
24
12
  ```
25
13
 
26
- `Queues` use two separate proxies to control `Queue` commands vs
27
- `QueueManager` commands.
28
- ```ruby
29
- # .queue returns the proxy for the queue (i.e. workers)
30
- q = AnyCelluloidClass.queue # => WindUp::QueueProxy(AnyCelluloidClass)
14
+ Usage
15
+ -----
31
16
 
32
- # Get the proxy for the manager from the QueueProxy
33
- q.__manager__ # => Celluloid::ActorProxy(WindUp::QueueManager)
17
+ WindUp's `Delegator` allows us to use one queue to process multiple job types.
18
+ `Delegator#perform_with` will instantiate a class and run its `#perform`
19
+ method, passing any additional arguments provided.
34
20
 
35
- # Return to the queue from the manager
36
- q.__manager__.queue # WindUp::QueueProxy(AnyCelluloidClass)
37
- ```
21
+ Create a new `Delegator` queue using the WindUp Queue method. Use
22
+ `Delegator#perform_with` to send tasks to this queue. Asynchronous processing
23
+ is accomplished the same way you would with a WindUp Queue or Celluloid Pool;
24
+ `#sync`, `#async`, and `#future` all continue to work as expected.
38
25
 
39
- You may store these `Queue` object in the registry as any actor
40
26
  ```ruby
41
- Celluloid::Actor[:queue] = q
42
- ```
43
-
44
- Changing Routing Behavior (Advanced)
45
- ------------------------------------
46
-
47
- WindUp accepts multiple types of routing behavior. Supported behaviors include:
27
+ # Create a Delegator queue; these are equivalent
28
+ queue = WindUp::Delegator.queue size: 3
29
+ queue = WindUp.queue size: 3
30
+
31
+ # Create a job class
32
+ class GreetingJob
33
+ def perform(name = "Bob")
34
+ "Hello, #{name}!"
35
+ end
36
+ end
48
37
 
49
- * :random - Route messages to workers randomly
50
- * :round_robin - Route messages to each worker sequentially
51
- * :smallest_mailbox - Route messages to the worker with the smallest mailbox
52
- * :first_available - Route messages to the first available worker (Default)
38
+ # Send the delayed action to the Delegator queue
39
+ queue.perform_with GreetingJob, "Hurried Harry" # => "Hello, Hurried Harry!", completed synchronously
40
+ queue.async.perform_with GreetingJob, "Mellow Mary" # => nil, work completed in background
41
+ queue.future.perform_with GreetingJob, "Telepathic Tim" # => Celluloid::Future, with value "Hello, Telepathic Tim!"
53
42
 
54
- To configure your queue to use a specific routing style, specify the routing
55
- behavior when calling .queue:
43
+ # Store this queue for later usage
44
+ Celluloid::Actor[:background] = queue
45
+ # Later...
46
+ Celluloid::Actor[:background].async.perform_with GreetingJob, "Tina"
56
47
 
57
- ```ruby
58
- # Use a random router
59
- Klass.queue router: :random
60
48
  ```
61
49
 
62
- You can specify your own custom routing behavior as well:
63
- ```ruby
64
- class FirstRouter < WindUp::Router::Base
65
- # Returns the next subscriber to route a message to
66
- def next_subscriber
67
- subscribers.first
68
- end
69
- end
50
+ Tips
51
+ ----
70
52
 
71
- # Register this router
72
- WindUp::Routers.register :first, FirstRouter
53
+ * Don't share state from your current thread with your Jobs!
73
54
 
74
- # Use this router
75
- q = Klass.queue router: :first
76
-
77
- ```
55
+ With WindUp, your jobs will be *eventually* processed, but we have no idea
56
+ how long it'll be. As such, your jobs and messages should not share state.
57
+ I.e., don't pass a User object; pass a user.id and recreates the user from
58
+ the database within your job.
78
59
 
79
- Multiple worker types per queue
80
- -------------------------------
60
+ * Use JSON serializable arguments for your jobs
81
61
 
82
- This part of WindUp has been extracted to its own gem,
83
- [MultiWindUp](https://www.github.com/ryanlchan/multi_wind_up).
84
- WindUp now only contains the pooling implementation.
62
+ Not only is it a good way to prevent yourself from accidentally sharing
63
+ state (see previous point), but if you want to use a persistant job store
64
+ you cannot pass in any objects that can't be dumped to JSON.
85
65
 
86
66
  ## Contributing
87
67
 
data/lib/wind_up.rb CHANGED
@@ -1,20 +1,8 @@
1
- require 'celluloid'
2
- require 'wind_up/exceptions'
3
- require 'wind_up/calls'
4
- require 'wind_up/routers'
5
- require 'wind_up/queue_proxy'
6
- require 'wind_up/queue_manager'
1
+ require 'wind_up/delegator'
7
2
  require 'wind_up/version'
8
- require 'wind_up/celluloid_ext'
9
3
 
10
4
  module WindUp
11
- def self.logger
12
- Celluloid.logger
13
- end
14
-
15
- def self.logger=(logger)
16
- Celluloid.logger = logger
5
+ def self.queue(opts = {})
6
+ Delegator.queue opts
17
7
  end
18
8
  end
19
-
20
- require 'wind_up/railtie' if defined?(::Rails)
@@ -0,0 +1,50 @@
1
+ # A worker to enable flexible, queue-based background processing
2
+ # ---
3
+ # Ever wish you could reuse the same background worker pool for multiple types
4
+ # of work? WindUp's `Delegator` was designed to solve this problem.
5
+ #
6
+ # `Delegator#perform_with` will instantiate the class and run its
7
+ # #perform method with any additional arguments provided
8
+ # * Gracefully handles errors/crashes
9
+ # * Use just like a WindUp Queue or Celluloid Pool; `#sync`, `#async`, and
10
+ # `#future` all work
11
+ #
12
+ # Usage
13
+ # -----
14
+ # Create a new `Delegator` queue using the WindUp Queue method. Use
15
+ # `Delegator#perform_with` to perform tasks in the background.
16
+ #
17
+ # ```ruby
18
+ # # Create a Delegator queue
19
+ # queue = WindUp::Delegator.queue size: 3
20
+ #
21
+ # # Create a job class
22
+ # class GreetingJob
23
+ # def perform(name = "Bob")
24
+ # "Hello, #{name}!"
25
+ # end
26
+ # end
27
+ #
28
+ # # Send the delayed action to the Delegator queue
29
+ # queue.async.perform_with GreetingJob, "Mary" # => nil, work completed in background
30
+ # queue.future.perform_with GreetingJob, "Tim" # => Celluloid::Future, with value "Hello, Tim!"
31
+ #
32
+ # # Store your queue for future usage
33
+ # Celluloid::Actor[:greeting_queue] = queue
34
+ # ```
35
+ module WindUp
36
+ class InvalidDelegatee < StandardError; end
37
+
38
+ class Delegator
39
+ include Celluloid
40
+
41
+ # Instantiate the referenced class and run the #perform action
42
+ # @param delegatee [Class] the class to delegate to
43
+ # @return [Object] the return value of the action
44
+ def perform_with(delegatee, *args)
45
+ handler = delegatee.new
46
+ raise InvalidDelegatee, "#{delegatee} does not have a #perform method defined" unless handler.respond_to?(:perform)
47
+ handler.perform *args
48
+ end
49
+ end
50
+ end
@@ -1,3 +1,3 @@
1
1
  module WindUp
2
- VERSION = "0.0.2"
2
+ VERSION = "0.0.3"
3
3
  end
@@ -0,0 +1,70 @@
1
+ require 'spec_helper'
2
+
3
+ class MissingPerform; end
4
+
5
+ class TestHandler
6
+ def perform(*args)
7
+ if args.size > 0
8
+ :args
9
+ else
10
+ :done
11
+ end
12
+ end
13
+ end
14
+
15
+ class TestQueuer
16
+ def perform(queue)
17
+ queue << :done
18
+ end
19
+ end
20
+
21
+
22
+ describe WindUp::Delegator do
23
+ describe '#perform_with' do
24
+ let(:worker) { WindUp::Delegator.new }
25
+ context 'when not passed a delegatee' do
26
+ it 'raises an exception' do
27
+ mute_celluloid_logging do
28
+ expect{ worker.perform_with }.to raise_exception
29
+ end
30
+ end
31
+ end
32
+
33
+ context 'when passed a delegatee' do
34
+ context 'but does not have a #perform method' do
35
+ it 'raises an exception' do
36
+ mute_celluloid_logging do
37
+ expect{ worker.perform_with MissingPerform }.to raise_exception(WindUp::InvalidDelegatee)
38
+ end
39
+ end
40
+ end
41
+ context 'and has a #perform method' do
42
+ it 'initializes the handler class' do
43
+ worker.perform_with(TestHandler).should eq :done
44
+ end
45
+
46
+ it 'passes the arguments from the method' do
47
+ worker.perform_with(TestHandler, :argument).should eq :args
48
+ end
49
+ end
50
+ end
51
+ end
52
+
53
+ context 'when queued' do
54
+ let(:queue) { WindUp::Delegator.queue }
55
+ it 'returns synchronously' do
56
+ queue.perform_with(TestHandler).should eq :done
57
+ end
58
+
59
+ it 'returns asynchronously' do
60
+ q = Queue.new
61
+ queue.async.perform_with TestQueuer, q
62
+ q.pop.should eq :done
63
+ end
64
+
65
+ it 'returns as a future' do
66
+ f = queue.future.perform_with TestHandler
67
+ f.value.should eq :done
68
+ end
69
+ end
70
+ end
data/spec/spec_helper.rb CHANGED
@@ -4,14 +4,20 @@ rescue LoadError
4
4
  end
5
5
 
6
6
  require 'celluloid'
7
+ require 'wind_up_queue'
7
8
  require 'wind_up'
8
9
 
9
10
  Celluloid.shutdown; Celluloid.boot
10
11
 
12
+ LOGGING_MUTEX = Mutex.new
11
13
  def mute_celluloid_logging
12
- # Temporarily mute celluloid logger
13
- Celluloid.logger.level = Logger::FATAL
14
- yield if block_given?
15
- # Restore celluloid logger
16
- Celluloid.logger.level = Logger::DEBUG
14
+ LOGGING_MUTEX.synchronize do
15
+ # Temporarily mute celluloid logger
16
+ Celluloid.logger.level = Logger::FATAL
17
+ yield if block_given?
18
+ # Allow any async messages to process before completing
19
+ sleep 0.01
20
+ # Restore celluloid logger
21
+ Celluloid.logger.level = Logger::DEBUG
22
+ end
17
23
  end
data/spec/wind_up_spec.rb CHANGED
@@ -1,14 +1,9 @@
1
1
  require 'spec_helper'
2
2
 
3
3
  describe WindUp do
4
- describe '.logger' do
5
- it "delegates get to Celluloid's logger" do
6
- WindUp.logger.should == Celluloid.logger
7
- end
8
-
9
- it "delegates set to Celluloid's logger" do
10
- Celluloid.should_receive(:logger=)
11
- WindUp.logger = nil
4
+ describe '.queue' do
5
+ it 'creates a delegator queue' do
6
+ WindUp.queue.should be_a(WindUp::Delegator)
12
7
  end
13
8
  end
14
9
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: wind_up
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.2
4
+ version: 0.0.3
5
5
  platform: ruby
6
6
  authors:
7
7
  - Ryan Chan
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2013-08-20 00:00:00.000000000 Z
11
+ date: 2013-08-21 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: rspec
@@ -39,50 +39,35 @@ dependencies:
39
39
  - !ruby/object:Gem::Version
40
40
  version: '0'
41
41
  - !ruby/object:Gem::Dependency
42
- name: celluloid
42
+ name: wind_up_queue
43
43
  requirement: !ruby/object:Gem::Requirement
44
44
  requirements:
45
45
  - - ~>
46
46
  - !ruby/object:Gem::Version
47
- version: 0.14.1
47
+ version: 0.0.1
48
48
  type: :runtime
49
49
  prerelease: false
50
50
  version_requirements: !ruby/object:Gem::Requirement
51
51
  requirements:
52
52
  - - ~>
53
53
  - !ruby/object:Gem::Version
54
- version: 0.14.1
55
- description: WindUp is a simple background processing library meant to improve on
56
- Celluloid's pools
54
+ version: 0.0.1
55
+ description: WindUp enables simple background processing using Celluloid Actors
57
56
  email:
58
57
  - ryan@ryanlchan.com
59
58
  executables: []
60
59
  extensions: []
61
60
  extra_rdoc_files: []
62
61
  files:
63
- - .ruby-gemset
64
- - .ruby-version
65
- - CHANGES.md
66
- - Gemfile
67
- - Gemfile.lock
68
- - LICENSE.txt
69
62
  - README.md
70
- - Rakefile
71
- - lib/wind_up.rb
72
- - lib/wind_up/calls.rb
73
- - lib/wind_up/celluloid_ext.rb
74
- - lib/wind_up/exceptions.rb
75
- - lib/wind_up/queue_manager.rb
76
- - lib/wind_up/queue_proxy.rb
77
- - lib/wind_up/railtie.rb
63
+ - lib/wind_up/delegator.rb
78
64
  - lib/wind_up/routers.rb
79
65
  - lib/wind_up/version.rb
66
+ - lib/wind_up.rb
67
+ - spec/lib/delegator_spec.rb
80
68
  - spec/spec_helper.rb
81
- - spec/wind_up/queue_manager_spec.rb
82
- - spec/wind_up/queue_spec.rb
83
69
  - spec/wind_up/routers_spec.rb
84
70
  - spec/wind_up_spec.rb
85
- - wind_up.gemspec
86
71
  homepage: https://github.com/ryanlchan/wind_up
87
72
  licenses: []
88
73
  metadata: {}
@@ -105,10 +90,9 @@ rubyforge_project:
105
90
  rubygems_version: 2.0.3
106
91
  signing_key:
107
92
  specification_version: 4
108
- summary: A drop-in replacement for Celluloid pools using the message API
93
+ summary: Super simple background processing
109
94
  test_files:
95
+ - spec/lib/delegator_spec.rb
110
96
  - spec/spec_helper.rb
111
- - spec/wind_up/queue_manager_spec.rb
112
- - spec/wind_up/queue_spec.rb
113
97
  - spec/wind_up/routers_spec.rb
114
98
  - spec/wind_up_spec.rb
data/.ruby-gemset DELETED
@@ -1 +0,0 @@
1
- wind_up
data/.ruby-version DELETED
@@ -1 +0,0 @@
1
- ruby-2.0.0-p0
data/CHANGES.md DELETED
@@ -1,9 +0,0 @@
1
- CHANGELOG
2
- =========
3
-
4
- ## 0.0.2 ##
5
- * Use Akka-style routers
6
-
7
-
8
- ## 0.0.1 ##
9
- * Initial release
data/Gemfile DELETED
@@ -1,4 +0,0 @@
1
- source 'https://rubygems.org'
2
-
3
- # Specify your gem's dependencies in wind_up.gemspec
4
- gemspec
data/Gemfile.lock DELETED
@@ -1,30 +0,0 @@
1
- PATH
2
- remote: .
3
- specs:
4
- wind_up (0.0.2)
5
- celluloid (~> 0.14.1)
6
-
7
- GEM
8
- remote: https://rubygems.org/
9
- specs:
10
- celluloid (0.14.1)
11
- timers (>= 1.0.0)
12
- diff-lcs (1.2.4)
13
- rake (10.1.0)
14
- rspec (2.13.0)
15
- rspec-core (~> 2.13.0)
16
- rspec-expectations (~> 2.13.0)
17
- rspec-mocks (~> 2.13.0)
18
- rspec-core (2.13.1)
19
- rspec-expectations (2.13.0)
20
- diff-lcs (>= 1.1.3, < 2.0)
21
- rspec-mocks (2.13.1)
22
- timers (1.1.0)
23
-
24
- PLATFORMS
25
- ruby
26
-
27
- DEPENDENCIES
28
- rake
29
- rspec
30
- wind_up!
data/LICENSE.txt DELETED
@@ -1,22 +0,0 @@
1
- Copyright (c) 2013 Ryan Chan
2
-
3
- MIT License
4
-
5
- Permission is hereby granted, free of charge, to any person obtaining
6
- a copy of this software and associated documentation files (the
7
- "Software"), to deal in the Software without restriction, including
8
- without limitation the rights to use, copy, modify, merge, publish,
9
- distribute, sublicense, and/or sell copies of the Software, and to
10
- permit persons to whom the Software is furnished to do so, subject to
11
- the following conditions:
12
-
13
- The above copyright notice and this permission notice shall be
14
- included in all copies or substantial portions of the Software.
15
-
16
- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
17
- EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
18
- MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
19
- NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
20
- LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
21
- OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
22
- WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/Rakefile DELETED
@@ -1,16 +0,0 @@
1
- require "bundler/gem_tasks"
2
- require 'rspec/core/rake_task'
3
-
4
- RSpec::Core::RakeTask.new('spec')
5
-
6
- # If you want to make this the default task
7
- task :default => :spec
8
- task :test => :spec
9
-
10
- task :console do
11
- exec "irb -r wind_up -I ./lib"
12
- end
13
-
14
- task :rspec do
15
- exec "rspec -f doc --color"
16
- end
data/lib/wind_up/calls.rb DELETED
@@ -1,27 +0,0 @@
1
- module WindUp
2
- # WindUp's ForwardedCall tells an actor to pull a message from a source
3
- # mailbox when processed
4
- class ForwardedCall < Celluloid::Call
5
-
6
- # Do not block if no work found
7
- TIMEOUT = 0
8
-
9
- def initialize(source)
10
- @source = source
11
- end
12
-
13
- # Pull the next message from the source, if available
14
- def dispatch(obj)
15
- msg = @source.receive(TIMEOUT)
16
- ::Celluloid.mailbox << msg if msg
17
- end
18
- end
19
-
20
- # Wraps a TerminationRequest for standard ordering within a mailbox
21
- class DelayedTerminationRequest < Celluloid::Call
22
- def initialize
23
- @method = :terminate
24
- end
25
- end
26
- end
27
-
@@ -1,10 +0,0 @@
1
- # Extend Celluloid with the ability to use Klass.queue
2
- module WindUp
3
- module CelluloidExts
4
- def queue(options = {})
5
- WindUp::QueueManager.new(self, options).queue
6
- end
7
- end
8
- end
9
-
10
- Celluloid::ClassMethods.send :include, WindUp::CelluloidExts
@@ -1,3 +0,0 @@
1
- module WindUp
2
- class MissingWorkerClass < StandardError; end
3
- end
@@ -1,138 +0,0 @@
1
- # Manages a queue of workers
2
- # Accumulates/stores messages and supervises a group of workers
3
- # WindUp `Queues` are almost drop-in replacements for Celluloid pools.
4
- #
5
- # ```ruby
6
- # q = AnyCelluloidClass.queue size: 3 # size defaults to number of cores
7
- # q.any_method # perform synchronously
8
- # q.async.long_running_method # perform asynchronously
9
- # q.future.i_want_this_back # perform as a future
10
- # ```
11
- #
12
- # `Queues` use two separate proxies to control `Queue` commands vs
13
- # `QueueManager` commands.
14
- # ```ruby
15
- # # .queue returns the proxy for the queue (i.e. workers)
16
- # q = AnyCelluloidClass.queue # => WindUp::QueueProxy(AnyCelluloidClass)
17
- #
18
- # # Get the proxy for the manager from the QueueProxy
19
- # q.__manager__ # => Celluloid::ActorProxy(WindUp::QueueManager)
20
- #
21
- # # Return to the queue from the manager
22
- # q.__manager__.queue # WindUp::QueueProxy(AnyCelluloidClass)
23
- # ```
24
- #
25
- # You may store these `Queue` object in the registry as any actor
26
- # ```ruby
27
- # Celluloid::Actor[:queue] = q
28
- # ```
29
- module WindUp
30
- class QueueManager
31
- include Celluloid
32
- attr_reader :size, :router, :worker_class
33
-
34
- trap_exit :restart_actor
35
-
36
- # Don't use QueueManager.new, use Klass.queue instead
37
- def initialize(worker_class, options = {})
38
- defaults = { :size => [Celluloid.cores, 2].max,
39
- :router => :first_available }
40
- options = defaults.merge options
41
-
42
- @worker_class = worker_class
43
- @args = options[:args] ? Array(options[:args]) : []
44
- @size = options[:size]
45
-
46
- router_class = Routers[options[:router]]
47
- raise ArgumentError, "Router class not recognized" unless router_class
48
- @router = router_class.new
49
-
50
- @registry = Celluloid::Registry.root
51
- @group = []
52
- resize_group
53
- end
54
-
55
- # Terminate our supervised group on finalization
56
- finalizer :__shutdown__
57
- def __shutdown__
58
- @router.shutdown
59
- group.reverse_each(&:terminate)
60
- end
61
-
62
- ###########
63
- # Helpers #
64
- ###########
65
-
66
- # Access the Queue's proxy
67
- def queue
68
- WindUp::QueueProxy.new Actor.current
69
- end
70
-
71
- # Resize this queue's worker group
72
- # NOTE: Using this to down-size your queue CAN truncate ongoing work!
73
- # Workers which are waiting on blocks/sleeping will receive a termination
74
- # request prematurely!
75
- # @param num [Integer] Number of workers to use
76
- def size=(num)
77
- @size = num
78
- resize_group
79
- end
80
-
81
- # Return the size of the queue backlog
82
- # @return [Integer] the number of messages queueing
83
- def backlog
84
- @router.size
85
- end
86
-
87
- def inspect
88
- "<Celluloid::ActorProxy(#{self.class}) @size=#{@size} @worker_class=#{@worker_class} @backlog=#{backlog}>"
89
- end
90
-
91
- ####################
92
- # Group Management #
93
- ####################
94
-
95
- # Restart a crashed actor
96
- def restart_actor(actor, reason)
97
- member = group.find do |_member|
98
- _member.actor == actor
99
- end
100
- raise "A group member went missing. This shouldn't be!" unless member
101
-
102
- if reason
103
- member.restart(reason)
104
- else
105
- # Remove from group on clean shutdown
106
- group.delete_if do |_member|
107
- _member.actor == actor
108
- end
109
- end
110
- end
111
-
112
- private
113
- def group
114
- @group ||= []
115
- end
116
-
117
- # Resize the worker group in this queue
118
- # You should probably be using #size=
119
- # @param target [Integer] the targeted number of workers to grow to
120
- def resize_group(target = size)
121
- delta = target - group.size
122
- if delta == 0
123
- # *Twiddle thumbs*
124
- return
125
- elsif delta > 0
126
- # Increase pool size
127
- delta.times do
128
- worker = Celluloid::SupervisionGroup::Member.new @registry, @worker_class, :args => @args
129
- group << worker
130
- @router.add_subscriber(worker.actor.mailbox)
131
- end
132
- else
133
- # Truncate pool
134
- delta.abs.times { @router << DelayedTerminationRequest.new }
135
- end
136
- end
137
- end
138
- end
@@ -1,37 +0,0 @@
1
- # A proxy object which sends calls to a Queue mailbox
2
- module WindUp
3
- class QueueProxy < Celluloid::ActorProxy
4
- def initialize(manager)
5
- @mailbox = manager.router
6
- @klass = manager.worker_class.to_s
7
- @sync_proxy = ::Celluloid::SyncProxy.new(@mailbox, @klass)
8
- @async_proxy = ::Celluloid::AsyncProxy.new(@mailbox, @klass)
9
- @future_proxy = ::Celluloid::FutureProxy.new(@mailbox, @klass)
10
-
11
- @manager_proxy = manager
12
- end
13
-
14
- # Escape route to access the QueueManager actor from the QueueProxy
15
- def __manager__
16
- @manager_proxy
17
- end
18
-
19
- # Reroute termination/alive? to the queue manager
20
- def terminate
21
- __manager__.terminate
22
- end
23
-
24
- def terminate!
25
- __manager__.terminate!
26
- end
27
-
28
- def alive?
29
- __manager__.alive?
30
- end
31
-
32
- def inspect
33
- orig = super
34
- orig.sub("Celluloid::ActorProxy", "WindUp::QueueProxy")
35
- end
36
- end
37
- end
@@ -1,7 +0,0 @@
1
- module WindUp
2
- class Railtie < ::Rails::Railtie
3
- initializer "wind_up.logger" do
4
- WindUp.logger = Rails.logger
5
- end
6
- end
7
- end
@@ -1,105 +0,0 @@
1
- require 'spec_helper'
2
-
3
- class FakeWorker
4
- include Celluloid
5
- def process(queue = nil)
6
- if queue
7
- queue << :done
8
- else
9
- :done
10
- end
11
- end
12
-
13
- def crash
14
- raise StandardError, "zomgcrash"
15
- end
16
- end
17
-
18
- class SleepyWorker < FakeWorker
19
- def sleepy
20
- sleep 0.2
21
- :done
22
- end
23
- end
24
-
25
-
26
- describe WindUp::QueueManager do
27
- let(:queue) { FakeWorker.queue size: 2 }
28
- describe '#initialize' do
29
- it 'creates a supervision group of workers' do
30
- expect { FakeWorker.queue size: 1 }.to change { Celluloid::Actor.all.size }.by(2)
31
- end
32
-
33
- it 'creates as many workers as number of cores on the system' do
34
- cores = FakeWorker.queue
35
- cores.__manager__.size.should eq Celluloid.cores
36
- end
37
-
38
- it 'requires a worker class' do
39
- mute_celluloid_logging do
40
- expect { WindUp::QueueManager.new }.to raise_exception
41
- sleep 0.1
42
- end
43
- end
44
- end
45
-
46
- describe '#terminate' do
47
- it 'terminates the manager' do
48
- queue.terminate
49
- queue.should_not be_alive
50
- end
51
-
52
- it 'terminates the pool' do
53
- expect{ queue.terminate }.to change { Celluloid::Actor.all.size }.by(0)
54
- end
55
- end
56
-
57
- describe '#sync' do
58
- it 'processs calls synchronously' do
59
- queue.process.should be :done
60
- end
61
- end
62
-
63
- describe '#async' do
64
- it 'processs calls asynchronously' do
65
- q = Queue.new
66
- queue.async.process(q)
67
- q.pop.should be :done
68
- end
69
-
70
- it 'processes additional work when workers as sleeping' do
71
- sleepy_queue = SleepyWorker.queue size: 1
72
- start_time = Time.now
73
- vals = 2.times.map { sleepy_queue.future.sleepy }
74
- vals.each { |v| v.value }
75
- (start_time - Time.now).should be < 0.3
76
- end
77
-
78
- it 'handles crashed calls gracefully' do
79
- mute_celluloid_logging do
80
- queue.async.crash
81
- queue.should be_alive
82
- end
83
- end
84
- end
85
-
86
- describe '#future' do
87
- it 'processes calls as futures' do
88
- f = queue.future.process
89
- f.value.should be :done
90
- end
91
- end
92
-
93
- describe '#size=' do
94
- let(:manager) { queue.__manager__ }
95
- it 'increases the size of the pool' do
96
- manager.size.should eq 2
97
- expect { manager.size = 3 }.to change{ Celluloid::Actor.all.size }.by(1)
98
- end
99
-
100
- it 'reduces the size of the pool' do
101
- manager.size.should eq 2
102
- expect { manager.size = 1 }.to change{ Celluloid::Actor.all.size }.by(-1)
103
- end
104
- end
105
- end
@@ -1,372 +0,0 @@
1
- require 'spec_helper'
2
-
3
- class FakeWorker
4
- include Celluloid
5
- def perform(*args); end;
6
- end
7
-
8
- # Because we use a Singleton model, it's tough to change configurations
9
- # without redefining a ton of classes. This class just helps us dynamically
10
- # create Queue classes using a block.
11
- class QueueFactory
12
- ALPHABET = ('a'..'z').to_a
13
- def self.bake(&block)
14
- name = 10.times.map{ ALPHABET.sample }.join.capitalize
15
- c = WindUp::Queue.new name
16
-
17
- if block_given?
18
- c.instance_eval &block
19
- else
20
- c.instance_eval {
21
- worker_class FakeWorker
22
- }
23
- end
24
- return c
25
- end
26
- end
27
-
28
- # describe WindUp::Queue, pending: "rewrite" do
29
- # let(:queue) { QueueFactory.bake }
30
- # describe "#store" do
31
- # context "by default" do
32
- # it "sets the InMemory store" do
33
- # queue.store.should be_a(WindUp::Store::InMemory)
34
- # end
35
- # end
36
- # context "when passed :memory or 'memory'" do
37
- # it "sets the InMemory store" do
38
- # queue.store(:memory).should be_a(WindUp::Store::InMemory)
39
- # queue.store("memory").should be_a(WindUp::Store::InMemory)
40
- # queue.store.should be_a(WindUp::Store::InMemory)
41
- # end
42
- # end
43
- # context "when passed :redis or 'redis'" do
44
- # it "sets the Redis store" do
45
- # queue.store(:redis).should be_a(WindUp::Store::Redis)
46
- # queue.store("redis").should be_a(WindUp::Store::Redis)
47
- # queue.store.should be_a(WindUp::Store::Redis)
48
- # end
49
-
50
- # context "with a :url provided" do
51
- # it "uses the url to connect to Redis" do
52
- # queue.store(:redis, url: "redis://127.0.0.1:6379")
53
- # queue.store.should be
54
- # end
55
- # end
56
-
57
- # context 'with :connection set to a Redis client instance' do
58
- # it 'works' do
59
- # redis = Redis.new
60
- # queue.store(:redis, connection: redis)
61
- # queue.store.should be
62
- # end
63
- # end
64
-
65
- # context 'with :connection set to a Redis connection_pool' do
66
- # it 'works' do
67
- # redis = ConnectionPool.new(size: 2, timeout: 5) { Redis.new }
68
- # queue.store(:redis, connection: redis)
69
- # queue.store.should be
70
- # end
71
- # end
72
- # end
73
- # end
74
-
75
- # context '#pool' do
76
- # context 'configured using :workers/:worker' do
77
- # let(:queue) do
78
- # QueueFactory.bake do
79
- # worker_class FakeWorker
80
- # workers 3
81
- # end
82
- # end
83
-
84
- # it "sets up the pool" do
85
- # queue.pool.should be
86
- # queue.pool.size.should eq 3
87
- # end
88
- # end
89
-
90
- # context ':pool_name' do
91
- # context 'when given an existing pool' do
92
- # it 'uses the existing pool' do
93
- # Celluloid::Actor[:pool_name_spec] = FakeWorker.pool size: 2
94
- # queue = QueueFactory.bake do
95
- # pool_name :pool_name_spec
96
- # end
97
- # queue.pool.should be Celluloid::Actor[:pool_name_spec]
98
- # end
99
- # end
100
- # end
101
- # end
102
-
103
- # context '#priority_level' do
104
- # context 'that are equal' do
105
- # it 'creates a queue with priority levels' do
106
- # queue = QueueFactory.bake do
107
- # worker_class FakeWorker
108
- # priority_level :clone1
109
- # priority_level :clone2
110
- # end
111
-
112
- # queue.should be
113
- # queue.priority_levels.should_not be_empty
114
- # end
115
- # end
116
-
117
- # context 'that are weighted' do
118
- # it 'creates a queue with priority levels' do
119
- # queue = QueueFactory.bake do
120
- # worker_class FakeWorker
121
- # priority_level :high, weight: 10
122
- # priority_level :low, weight: 1
123
- # end
124
-
125
- # queue.should be
126
- # queue.priority_levels.should_not be_empty
127
- # queue.priority_level_weights.should eq({high: 10, low: 1})
128
- # end
129
- # end
130
-
131
- # context 'that are strictly ordered' do
132
- # it 'creates a queue with priority levels' do
133
- # queue = QueueFactory.bake do
134
- # worker_class FakeWorker
135
- # strict true
136
- # priority_level :clone1
137
- # priority_level :clone2
138
- # end
139
-
140
- # queue.should be
141
- # queue.priority_levels.should_not be_empty
142
- # queue.should be_strict
143
- # end
144
- # end
145
-
146
- # it 'does not create priority levels with the same name' do
147
- # queue = QueueFactory.bake do
148
- # worker_class FakeWorker
149
- # strict true
150
- # priority_level :clone1
151
- # priority_level :clone1
152
- # end
153
-
154
- # queue.priority_levels.should eq Set[:clone1]
155
- # end
156
- # end # with priority leels
157
-
158
- # describe '#push' do
159
- # context 'when not given a priority level' do
160
- # it 'pushes the argument to the store with default priority' do
161
- # queue.store.should_receive(:push).with("test", priority_level: nil)
162
- # queue.push "test"
163
- # end
164
- # end
165
- # context 'when given a priority level' do
166
- # it 'pushes the argument to the store with specified priority' do
167
- # queue.store.should_receive(:push).with("test", priority_level: "high")
168
- # queue.push "test", priority_level: "high"
169
- # end
170
- # end
171
- # end
172
-
173
- # describe '#pop' do
174
- # context 'without a priority level argument' do
175
- # context 'with a strictly ordered queue' do
176
- # it 'pops priority levels in order' do
177
- # queue = QueueFactory.bake do
178
- # worker_class FakeWorker
179
- # strict true
180
- # priority_level :high
181
- # priority_level :low
182
- # end
183
- # queue.store.should_receive(:pop).with([:high, :low]).at_least(1).times
184
- # queue.pop
185
- # end
186
- # end
187
- # context 'with a loosely ordered queue' do
188
- # it 'pops priority levels in proportion' do
189
- # queue = QueueFactory.bake do
190
- # worker_class FakeWorker
191
- # priority_level :high
192
- # priority_level :low
193
- # end
194
- # queue.store.stub(:pop) { nil }
195
- # queue.store.stub(:pop).with { [:high, :low] }.and_return { "success" }
196
- # queue.store.stub(:pop).with { [:low, :high] }.and_return { "success" }
197
-
198
- # queue.pop.should be
199
- # end
200
- # end
201
- # end
202
- # context 'with a priority_level argument' do
203
- # it 'pops the specified priority' do
204
- # queue.store.stub(:pop) { nil }
205
- # queue.store.stub(:pop).with(["queue"]) { "work" }
206
- # queue.pop(["queue"]).should be
207
- # end
208
- # end
209
- # end
210
-
211
- # describe '#workers' do
212
- # it 'returns the number of workers in the pool' do
213
- # queue = QueueFactory.bake do
214
- # worker_class FakeWorker
215
- # workers 2
216
- # end
217
- # queue.workers.should eq 2
218
- # end
219
- # end
220
-
221
- # describe '#busy_workers' do
222
- # it 'returns the number of busy_workers in the pool' do
223
- # queue = QueueFactory.bake do
224
- # worker_class FakeWorker
225
- # workers 2
226
- # end
227
- # queue.busy_workers.should eq 0
228
- # end
229
- # end
230
-
231
- # describe '#idle_workers' do
232
- # it 'returns the number of idle_workers in the pool' do
233
- # queue = QueueFactory.bake do
234
- # worker_class FakeWorker
235
- # workers 2
236
- # end
237
- # queue.idle_workers.should eq 2
238
- # end
239
- # end
240
-
241
- # describe '#backlog?' do
242
- # it 'returns true if the pool is fully utilized' do
243
- # queue = QueueFactory.bake do
244
- # worker_class FakeWorker
245
- # end
246
- # queue.should_not be_backlog
247
- # end
248
- # end
249
-
250
- # describe '#size' do
251
- # it 'returns the number of jobs in the queue store' do
252
- # queue.store.should_receive(:size).and_return(4)
253
- # queue.size.should eq 4
254
- # end
255
- # end
256
-
257
- # describe '#strict?' do
258
- # context 'when the queue is strictly ordered' do
259
- # it 'returns true' do
260
- # queue = QueueFactory.bake do
261
- # worker_class FakeWorker
262
- # strict true
263
- # end
264
- # queue.should be_strict
265
- # end
266
- # end
267
- # context 'when the queue is not strictly ordered' do
268
- # it 'returns false' do
269
- # queue = QueueFactory.bake do
270
- # worker_class FakeWorker
271
- # end
272
- # queue.should_not be_strict
273
- # end
274
- # end
275
- # end
276
-
277
- # describe '#priority_levels' do
278
- # context 'with a queue with priority levels' do
279
- # it 'returns the priority levels for this queue' do
280
- # queue = QueueFactory.bake do
281
- # worker_class FakeWorker
282
- # priority_level :high
283
- # priority_level :low
284
- # end
285
- # queue.priority_levels.should eq Set[:high, :low]
286
- # end
287
-
288
- # it 'does not return duplicates' do
289
- # queue = QueueFactory.bake do
290
- # worker_class FakeWorker
291
- # priority_level :high, weight: 10
292
- # priority_level :low, weight: 1
293
- # end
294
- # queue.priority_levels.should eq Set[:high, :low]
295
- # end
296
- # end
297
- # context 'with a queue without priority levels' do
298
- # it 'returns an empty array' do
299
- # queue = QueueFactory.bake do
300
- # worker_class FakeWorker
301
- # end
302
- # queue.priority_levels.should eq Set[]
303
- # end
304
- # end
305
- # end
306
-
307
- # describe '#priority_level_weights' do
308
- # context 'when run on a queue without priority levels' do
309
- # it 'returns an empty hash' do
310
- # queue = QueueFactory.bake do
311
- # worker_class FakeWorker
312
- # end
313
- # queue.priority_level_weights.should eq({})
314
- # end
315
- # end
316
-
317
- # context 'when run on a queue with unweighted priority levels' do
318
- # it "returns the priority levels with their weightings" do
319
- # queue = QueueFactory.bake do
320
- # worker_class FakeWorker
321
- # priority_level :high
322
- # priority_level :low
323
- # end
324
- # queue.priority_level_weights.should eq({high: 1, low: 1})
325
- # end
326
- # end
327
-
328
- # context 'when run on a queue with unweighted priority levels' do
329
- # it "returns the priority levels with their weightings" do
330
- # queue = QueueFactory.bake do
331
- # worker_class FakeWorker
332
- # priority_level :high, weight: 10
333
- # priority_level :low, weight: 1
334
- # end
335
- # queue.priority_level_weights.should eq({high: 10, low: 1})
336
- # end
337
- # end
338
- # end
339
-
340
- # describe '#default_priority_level' do
341
- # context 'when run on a queue without priority levels' do
342
- # it 'returns nil' do
343
- # queue = QueueFactory.bake do
344
- # worker_class FakeWorker
345
- # end
346
- # queue.default_priority_level.should_not be
347
- # end
348
- # end
349
-
350
- # context 'when run on a queue with priority levels' do
351
- # it "returns the first priority level" do
352
- # queue = QueueFactory.bake do
353
- # worker_class FakeWorker
354
- # priority_level :high
355
- # priority_level :low
356
- # end
357
- # queue.default_priority_level.should eq(:high)
358
- # end
359
- # end
360
-
361
- # context 'when run on a queue with a default priority level explicitly set' do
362
- # it "returns the set priority level" do
363
- # queue = QueueFactory.bake do
364
- # worker_class FakeWorker
365
- # priority_level :high, weight: 10
366
- # priority_level :low, weight: 1, default: true
367
- # end
368
- # queue.default_priority_level.should eq(:low)
369
- # end
370
- # end
371
- # end
372
- # end
data/wind_up.gemspec DELETED
@@ -1,24 +0,0 @@
1
- # -*- encoding: utf-8 -*-
2
- lib = File.expand_path('../lib', __FILE__)
3
- $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
4
- require 'wind_up/version'
5
-
6
- Gem::Specification.new do |gem|
7
- gem.name = "wind_up"
8
- gem.version = WindUp::VERSION
9
- gem.authors = ["Ryan Chan"]
10
- gem.email = ["ryan@ryanlchan.com"]
11
- gem.summary = %q{A drop-in replacement for Celluloid pools using the message API}
12
- gem.description = %q{WindUp is a simple background processing library meant to improve on Celluloid's pools}
13
- gem.homepage = "https://github.com/ryanlchan/wind_up"
14
-
15
- gem.files = `git ls-files`.split($/)
16
- gem.executables = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
17
- gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
18
- gem.require_paths = ["lib"]
19
-
20
- gem.add_development_dependency "rspec"
21
- gem.add_development_dependency "rake"
22
-
23
- gem.add_dependency "celluloid", "~> 0.14.1"
24
- end