workhorse 0.0.2 → 0.0.3

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: cf7da416379ae17a4defae96289beca3c4dd1d58
4
- data.tar.gz: 5b4e9cb886632f4a78982506747e1bcbdef1a7f3
3
+ metadata.gz: 1a6e6801d5faf896ac2c0ac4fdee3b769239c475
4
+ data.tar.gz: 68d7918cc11af8be1961c807dd6230dd7158ae0a
5
5
  SHA512:
6
- metadata.gz: c7f8b1730ada8f2613b63ee87bf570acae14fe057bfd2650ba1ea4daa479f641a99f6d77e74b2f9484cd62aedc2899a2fd5fed8b138b09b52a2191855439d884
7
- data.tar.gz: ee19f3e7a11eafda737333a7c41b6a83444e927c37033d8230adb26987c8bbd60c17346da5e4abfa61521500991b9856dc5fba91f5f37368fc72941a8fce60d6
6
+ metadata.gz: 1a7d958df73684464a58f66b8a6ceedfa464601ebf7ecb7de5a31ebd77c00ba9d71aa7d5b9581eca68567066e3e4f1696eadb46a2c2aec40d8cb2837b28ff88e
7
+ data.tar.gz: fbec1a172427cfccb678cf56ea5a688c68af5efc9040b0139e73e9445a7c27b6a0a04418f4dde89eb9891814c095e0f454349c16c2772cf2087bd024e15c4eb3
data/FAQ.md ADDED
@@ -0,0 +1,59 @@
1
+ # Workhorse FAQ
2
+
3
+ ## Why should I use workhorse over *X*?
4
+
5
+ There exist a variety of job backends for ruby,
6
+ [delayed_job](https://github.com/collectiveidea/delayed_job) probably being the
7
+ closest one to workhorse.
8
+
9
+ Some key advantages we feel workhorse has over other Gems:
10
+
11
+ - Workhorse is less than 500 lines of code at its core. The code is supposed to
12
+ be easily readable, understandable, and modifiable.
13
+
14
+ - Workhorse allows you to run multiple jobs simultaneously *in the same
15
+ process*.
16
+ This capability is what inspired the creation of workhorse in the first place.
17
+
18
+ We encourage you to have a look at the other projects as well and carefully
19
+ figure out which one best suits your needs.
20
+
21
+ ## My code is not thread safe. How can I use workhorse safely?
22
+
23
+ Job code that is not thread safe can not be executed safely in multiple threads
24
+ of the same process. In these cases, set the `pool_size` to `1` and, if you
25
+ still want to execute multiple jobs simultaneously, the daemon `count` to a
26
+ number greater than `1`:
27
+
28
+ ```ruby
29
+ Workhorse::Daemon::ShellHandler.run count: 5 do
30
+ Workhorse::Worker.start_and_wait(pool_size: 1)
31
+ end
32
+ ```
33
+
34
+ ## I'm using jRuby. How can I use the daemon handler?
35
+
36
+ As java processes in general can not be forked safely, the daemon handler
37
+ provided with this Gem does not support jRuby platforms.
38
+
39
+ If your jRuby application consists of one single application process, it is
40
+ recommended to just start the job backend in the same process:
41
+
42
+ ```ruby
43
+ worker = Workhorse::Worker.new(pool_size: 5)
44
+ worker.start
45
+ ```
46
+
47
+ This code is non-blocking, which means that it will run as long as the process
48
+ is up. Make sure to trap `INT` and `TERM` and call `worker.shutdown` when the
49
+ process stops.
50
+
51
+ If you have multiple application processes though, you may want to start the
52
+ worker in a separate process. For this purpose, adapt the startup script
53
+ `bin/workhorse.rb` so that it is blocking:
54
+
55
+ ```ruby
56
+ Workhorse::Worker.start_and_wait(pool_size: 5)
57
+ ```
58
+
59
+ This can then be started and "daemonized" using standard linux tools.
data/README.md CHANGED
@@ -3,34 +3,38 @@
3
3
 
4
4
  # Workhorse
5
5
 
6
- **This Gem is still in an early stage of development. Please not use this in production yet.**
7
-
8
6
  Multi-threaded job backend with database queuing for ruby.
9
7
 
10
8
  ## Introduction
11
9
 
12
- What it is:
10
+ How it works:
13
11
 
14
12
  * Jobs are instances of classes that support the `perform` method.
15
13
  * Jobs are persisted in the database using ActiveRecord.
16
14
  * You can start one or more worker processes.
17
- * Each worker is configurable as to which queue(s) it processes.
15
+ * Each worker is configurable as to which queue(s) it processes. Jobs in the
16
+ same queue never run simultaneously. Jobs with no queue can always run in
17
+ parallel.
18
18
  * Each worker polls the database and spawns a number of threads to execute jobs
19
19
  of different queues simultaneously.
20
20
 
21
- What it isn't:
21
+ What it does not do:
22
22
 
23
- * It cannot spawn new processes. Jobs are run in separate threads but not in
24
- separate processes (unless you start multiple worker processes manually).
25
- * It does not support retries.
23
+ * It does not spawn new processes on the fly. Jobs are run in separate threads
24
+ but not in separate processes (unless you manually start multiple worker
25
+ processes).
26
+ * It does not support retries, timeouts, and timed execution.
26
27
 
27
28
  ## Installation
28
29
 
29
30
  ### Requirements
30
31
 
31
32
  * A database and table handler that properly supports row-level locking (such as
32
- MySQL with InnoDB, PostgreSQL or Oracle).
33
- * An operating system and file system that supports file locking.
33
+ MySQL with InnoDB, PostgreSQL, or Oracle).
34
+ * If you are planning on using the daemons handler:
35
+ * An operating system and file system that supports file locking.
36
+ * MRI ruby (aka "c ruby") as jRuby does not support `fork`. See the
37
+ [FAQ](FAQ.md##im-using-jruby-how-can-i-use-the-daemon-handler) for possible workarounds.
34
38
 
35
39
  ### Installing under Rails
36
40
 
@@ -40,6 +44,8 @@ What it isn't:
40
44
  gem 'workhorse'
41
45
  ```
42
46
 
47
+ Install it using `bundle install` as usual.
48
+
43
49
  2. Run the install generator:
44
50
 
45
51
  ```bash
@@ -48,18 +54,19 @@ What it isn't:
48
54
 
49
55
  This generates:
50
56
 
51
- * A database migration for creating the `jobs` table
52
- * An initializer `config/initializers/workhorse.rb` for global configuration
53
- * A daemon worker script under `bin/workhorse.rb`
57
+ * A database migration for creating a table named `jobs`
58
+ * The initializer `config/initializers/workhorse.rb` for global configuration
59
+ * The daemon worker script `bin/workhorse.rb`
54
60
 
55
- Please customize the configuration files to your liking.
61
+ Please customize the initializer and worker script to your liking.
56
62
 
57
63
  ## Queuing jobs
58
64
 
59
65
  ### Basic jobs
60
66
 
61
67
  Workhorse can handle any jobs that support the `perform` method and are
62
- serializable. To queue a basic job, use the static method `Workhorse.enqueue`:
68
+ serializable. To queue a basic job, use the static method `Workhorse.enqueue`.
69
+ You can optionally pass a queue name.
63
70
 
64
71
  ```ruby
65
72
  class MyJob
@@ -75,11 +82,6 @@ end
75
82
  Workhorse.enqueue MyJob.new('John'), queue: :test
76
83
  ```
77
84
 
78
- In the above example, we also specify a queue named `:test`. This means that
79
- this job will never run simoultaneously with other jobs in the same queue. If no
80
- queue is given, the job can always be executed simoultaneously with any other
81
- job.
82
-
83
85
  ### RailsOps operations
84
86
 
85
87
  Workhorse allows you to easily queue
@@ -87,16 +89,16 @@ Workhorse allows you to easily queue
87
89
  method `Workhorse.enqueue_op`:
88
90
 
89
91
  ```ruby
90
- Workhorse.enqueue Operations::Jobs::CleanUpDatabase, quiet: true
92
+ Workhorse.enqueue_op Operations::Jobs::CleanUpDatabase, { quiet: true }, queue: :maintenance
91
93
  ```
92
94
 
93
95
  Params passed using the second argument will be used for operation instantiation
94
96
  at job execution.
95
97
 
96
- You can also specify a queue:
98
+ If you do not want to pass any params to the operation, just omit the second hash:
97
99
 
98
100
  ```ruby
99
- Workhorse.enqueue Operations::Jobs::CleanUpDatabase, { quiet: true }, queue: :maintenance
101
+ Workhorse.enqueue_op Operations::Jobs::CleanUpDatabase, queue: :maintenance
100
102
  ```
101
103
 
102
104
  ## Configuring and starting workers
@@ -105,12 +107,12 @@ Workers poll the database for new jobs and execute them in one or more threads.
105
107
  Typically, one worker is started per process. While you can start workers
106
108
  manually, either in your main application process(es) or in a separate one,
107
109
  workhorse also provides you with a convenient way of starting one or multiple
108
- worker processes as a daemon.
110
+ worker processes as daemons.
109
111
 
110
112
  ### Start workers manually
111
113
 
112
- Workers are created by instantiatating, configuring and starting a new
113
- `Workhorse::Worker` instance.
114
+ Workers are created by instantiating, configuring, and starting a new
115
+ `Workhorse::Worker` instance:
114
116
 
115
117
  ```ruby
116
118
  Workhorse::Worker.start_and_wait(
@@ -121,38 +123,30 @@ Workhorse::Worker.start_and_wait(
121
123
  )
122
124
  ```
123
125
 
124
- See [code
125
- documentation](http://www.rubydoc.info/github/sitrox/workhorse/Workhorse%2FWorker:initialize)
126
- for more information on the arguments.
126
+ See [code documentation](http://www.rubydoc.info/github/sitrox/workhorse/Workhorse%2FWorker:initialize)
127
+ for more information on the arguments. All arguments passed to `start_and_wait`
128
+ are passed to the initialize. All arguments passed to `start_and_wait` are
129
+ in turn passed to the initializer of `Workhorse::Worker`.
127
130
 
128
131
  ### Start workers using a daemon script
129
132
 
130
- Using `Workhorse::Daemon` (`Workhorse::Daemon::ShellHandler`), you can spawn one
131
- or multiple worker processes automatically. This is useful for cases where you
132
- want the workers to exist in separate processes as opposed to in your main
133
- application process(es).
133
+ Using `Workhorse::Daemon::ShellHandler`, you can spawn one or multiple worker
134
+ processes automatically. This is useful for cases where you want the workers to
135
+ exist in separate processes as opposed to your main application process(es).
134
136
 
135
- For this case, the workhorse install routine automatically creates a file called
136
- `bin/workhorse.rb` which can be used to start one or more worker processes.
137
+ For this case, the workhorse install routine automatically creates the file
138
+ `bin/workhorse.rb`, which can be used to start one or more worker processes.
137
139
 
138
- To start the daemon:
140
+ The script can be called as follows:
139
141
 
140
142
  ```bash
141
- bin/workhorse.rb start[|stop|status|watch|restart|usage]
143
+ bin/workhorse.rb start|stop|status|watch|restart|usage
142
144
  ```
143
145
 
144
146
  #### Background and customization
145
147
 
146
- The daemon-part allows you to run arbitrary code as a daemon:
147
-
148
- ```ruby
149
- Workhorse::Daemon::ShellHandler.run count: 5 do
150
- # This runs as a daemon and will be started 5 times
151
- end
152
- ```
153
-
154
- Within this shell handler, you can now instantiate, configure and start a worker
155
- as described under *Start workers manually*:
148
+ Within the shell handler, you can instantiate, configure, and start a worker as
149
+ described under [Start workers manually](#start-workers-manually):
156
150
 
157
151
  ```ruby
158
152
  Workhorse::Daemon::ShellHandler.run count: 5 do
@@ -162,11 +156,16 @@ Workhorse::Daemon::ShellHandler.run count: 5 do
162
156
  end
163
157
  ```
164
158
 
159
+ ## Frequently asked questions
160
+
161
+ Please consult the [FAQ](FAQ.md).
162
+
165
163
  ## Roadmap
166
164
 
167
165
  * [ ] ActiveJob integration for Rails
168
166
  * [ ] Job timeouts
167
+ * [ ] Job priorities
169
168
 
170
169
  ## Copyright
171
170
 
172
- Copyright (c) 2017 Sitrox. See `LICENSE` for further details.
171
+ Copyright © 2017 Sitrox. See `LICENSE` for further details.
data/VERSION CHANGED
@@ -1 +1 @@
1
- 0.0.2
1
+ 0.0.3
@@ -2,9 +2,12 @@ Workhorse.setup do |config|
2
2
  # Set this to false in order to prevent jobs from being automatically
3
3
  # wrapped into a transaction. The built-in workhorse logic will still run
4
4
  # in transactions.
5
- config.perform_jobs_in_tx = true
5
+ #
6
+ # config.perform_jobs_in_tx = true
6
7
 
7
- # Enable this to specify an alternative callback for handling transactions.
8
+ # Enable and configure this to specify an alternative callback for handling
9
+ # transactions.
10
+ #
8
11
  # config.tx_callback = proc do |&block|
9
12
  # ActiveRecord::Base.transaction&(&block)
10
13
  # end
data/lib/workhorse.rb CHANGED
@@ -2,7 +2,11 @@ require 'socket'
2
2
  require 'active_support/all'
3
3
  require 'active_record'
4
4
 
5
+ require 'workhorse/enqueuer'
6
+
5
7
  module Workhorse
8
+ extend Workhorse::Enqueuer
9
+
6
10
  @set_up = false
7
11
 
8
12
  # Returns the performer currently performing the active job. This can only be
@@ -28,7 +32,6 @@ module Workhorse
28
32
  end
29
33
 
30
34
  require 'workhorse/db_job'
31
- require 'workhorse/enqueuer'
32
35
  require 'workhorse/performer'
33
36
  require 'workhorse/poller'
34
37
  require 'workhorse/pool'
@@ -1,7 +1,7 @@
1
1
  module Workhorse
2
- class Enqueuer
2
+ module Enqueuer
3
3
  # Enqueue any object that is serializable and has a `perform` method
4
- def self.enqueue(job, queue: nil)
4
+ def enqueue(job, queue: nil)
5
5
  return DbJob.create!(
6
6
  queue: queue,
7
7
  handler: Marshal.dump(job)
@@ -9,14 +9,27 @@ module Workhorse
9
9
  end
10
10
 
11
11
  # Enqueue an ActiveJob job
12
- def self.enqueue_active_job(job)
12
+ def enqueue_active_job(job)
13
13
  enqueue job, queue: job.queue_name
14
14
  end
15
15
 
16
16
  # Enqueue the execution of an operation by its class and params
17
- def self.enqueue_op(cls, params, queue: nil)
18
- job = Workhorse::Jobs::RunRailsOp.new(cls, params)
19
- enqueue job, queue: queue
17
+ def enqueue_op(cls, *args)
18
+ case args.size
19
+ when 0
20
+ workhorse_args = {}
21
+ op_args = {}
22
+ when 1
23
+ workhorse_args = args.first
24
+ op_args = {}
25
+ when 2
26
+ workhorse_args, op_args = *args
27
+ else
28
+ fail ArgumentError, "wrong number of arguments (#{args.size + 1} for 2..3)"
29
+ end
30
+
31
+ job = Workhorse::Jobs::RunRailsOp.new(cls, op_args)
32
+ enqueue job, workhorse_args
20
33
  end
21
34
  end
22
35
  end
@@ -51,7 +51,12 @@ module Workhorse
51
51
 
52
52
  def poll
53
53
  Workhorse.tx_callback.call do
54
+ # As we are the only thread posting into the worker pool, it is safe to
55
+ # get the number of idle threads without mutex synchronization. The
56
+ # actual number of idle workers at time of posting can only be larger
57
+ # than or equal to the number we get here.
54
58
  idle = worker.idle
59
+
55
60
  worker.log "Polling DB for jobs (#{idle} available threads)...", :debug
56
61
 
57
62
  unless idle.zero?
@@ -70,7 +75,7 @@ module Workhorse
70
75
  is_oracle = ActiveRecord::Base.connection.adapter_name == 'OracleEnhanced'
71
76
 
72
77
  # ---------------------------------------------------------------
73
- # Lock all queued jobs that are not complete
78
+ # Lock all queued jobs that are waiting
74
79
  # ---------------------------------------------------------------
75
80
  Workhorse::DbJob.connection.execute(
76
81
  Workhorse::DbJob.select('null').where(
@@ -93,11 +98,17 @@ module Workhorse
93
98
  select = select.where(table[:queue].not_in(bad_queries_select))
94
99
 
95
100
  # Restrict queues to "open" ones
96
- if worker.queues.any?
97
- where = table[:queue].in(worker.queues.reject(&:nil?))
101
+ unless worker.queues.empty?
98
102
  if worker.queues.include?(nil)
99
- where = where.or(table[:queue].eq(nil))
103
+ where = table[:queue].eq(nil)
104
+ remaining_queues = worker.queues.reject(&:nil?)
105
+ unless remaining_queues.empty?
106
+ where = where.or(table[:queue].in(remaining_queues))
107
+ end
108
+ else
109
+ where = table[:queue].in(worker.queues)
100
110
  end
111
+
101
112
  select = select.where(where)
102
113
  end
103
114
 
@@ -19,7 +19,7 @@ module Workhorse
19
19
  # Posts a new work unit to the pool.
20
20
  def post
21
21
  mutex.synchronize do
22
- if @active_threads.value >= @size
22
+ if idle.zero?
23
23
  fail 'All threads are busy.'
24
24
  end
25
25
 
@@ -42,10 +42,14 @@ module Workhorse
42
42
  @size - @active_threads.value
43
43
  end
44
44
 
45
- # Shuts down the pool
45
+ def wait
46
+ @executor.wait_for_termination
47
+ end
48
+
49
+ # Shuts down the pool and waits for termination.
46
50
  def shutdown
47
51
  @executor.shutdown
48
- @executor.wait_for_termination
52
+ wait
49
53
  end
50
54
  end
51
55
  end
@@ -88,9 +88,13 @@ module Workhorse
88
88
  # properly finished before this method returns. Subsequent calls to this
89
89
  # method are ignored.
90
90
  def shutdown
91
+ # This is safe to be checked outside of the mutex as 'shutdown' is the
92
+ # final state this worker can be in.
93
+ return if @state == :shutdown
94
+
91
95
  mutex.synchronize do
92
- return if @state == :shutdown
93
96
  assert_state! :running
97
+
94
98
  log 'Shutting down'
95
99
  @state = :shutdown
96
100
 
@@ -107,6 +111,7 @@ module Workhorse
107
111
  def wait
108
112
  assert_state! :running
109
113
  @poller.wait
114
+ @pool.wait
110
115
  end
111
116
 
112
117
  def idle
@@ -133,10 +138,14 @@ module Workhorse
133
138
  def trap_termination
134
139
  SHUTDOWN_SIGNALS.each do |signal|
135
140
  Signal.trap(signal) do
141
+ # Start a new thread as certain functionality (such as logging) is not
142
+ # available from within a trap context. As "shutdown" terminates
143
+ # quickly when called multiple times, this does not pose a risk of
144
+ # keeping open a big number of "shutdown threads".
136
145
  Thread.new do
137
146
  log "\nCaught #{signal}, shutting worker down..."
138
147
  shutdown
139
- end
148
+ end.join
140
149
  end
141
150
  end
142
151
  end
@@ -22,7 +22,6 @@ class WorkhorseTest < ActiveSupport::TestCase
22
22
  end
23
23
  end
24
24
 
25
- ActiveRecord::Base.logger = Logger.new('debug.log')
26
25
  ActiveRecord::Base.establish_connection adapter: 'mysql2', database: 'workhorse', username: 'travis', password: '', pool: 10, host: :localhost
27
26
 
28
27
  require 'db_schema'
@@ -3,7 +3,7 @@ require 'test_helper'
3
3
  class Workhorse::EnqueuerTest < WorkhorseTest
4
4
  def test_basic
5
5
  assert_equal 0, Workhorse::DbJob.all.count
6
- Workhorse::Enqueuer.enqueue BasicJob.new
6
+ Workhorse.enqueue BasicJob.new
7
7
  assert_equal 1, Workhorse::DbJob.all.count
8
8
 
9
9
  db_job = Workhorse::DbJob.first
@@ -20,7 +20,7 @@ class Workhorse::EnqueuerTest < WorkhorseTest
20
20
 
21
21
  def test_with_queue
22
22
  assert_equal 0, Workhorse::DbJob.all.count
23
- Workhorse::Enqueuer.enqueue BasicJob.new, queue: :q1
23
+ Workhorse.enqueue BasicJob.new, queue: :q1
24
24
  assert_equal 1, Workhorse::DbJob.all.count
25
25
 
26
26
  db_job = Workhorse::DbJob.first
@@ -28,15 +28,25 @@ class Workhorse::EnqueuerTest < WorkhorseTest
28
28
  end
29
29
 
30
30
  def test_op
31
- Workhorse::Enqueuer.enqueue_op DummyRailsOpsOp, { foo: :bar }, queue: :q1
31
+ Workhorse.enqueue_op DummyRailsOpsOp, { queue: :q1 }, foo: :bar
32
32
 
33
33
  w = Workhorse::Worker.new(queues: [:q1])
34
34
  w.start
35
- sleep 1.5
35
+ sleep 1
36
36
  w.shutdown
37
37
 
38
38
  assert_equal 'succeeded', Workhorse::DbJob.first.state
39
39
 
40
40
  assert_equal [{ foo: :bar }], DummyRailsOpsOp.results
41
41
  end
42
+
43
+ def test_op_without_params
44
+ Workhorse.enqueue_op DummyRailsOpsOp, queue: :q1
45
+ assert_equal 'q1', Workhorse::DbJob.first.queue
46
+ end
47
+
48
+ def test_op_without_params_and_queue
49
+ Workhorse.enqueue_op DummyRailsOpsOp
50
+ assert_nil Workhorse::DbJob.first.queue
51
+ end
42
52
  end
@@ -6,10 +6,10 @@ class Workhorse::WorkerTest < WorkhorseTest
6
6
  def test_db_connections
7
7
  w = Workhorse::Worker.new polling_interval: 1, pool_size: 5
8
8
  5.times do
9
- Workhorse::Enqueuer.enqueue DbConnectionTestJob.new
9
+ Workhorse.enqueue DbConnectionTestJob.new
10
10
  end
11
11
  w.start
12
- sleep 2
12
+ sleep 1
13
13
  w.shutdown
14
14
 
15
15
  assert_equal 5, DbConnectionTestJob.db_connections.count
@@ -4,7 +4,7 @@ class Workhorse::PollerTest < WorkhorseTest
4
4
  def test_interruptable_sleep
5
5
  w = Workhorse::Worker.new(polling_interval: 60)
6
6
  w.start
7
- sleep 2
7
+ sleep 0.5
8
8
 
9
9
  Timeout.timeout(1.5) do
10
10
  w.shutdown
@@ -7,21 +7,21 @@ class Workhorse::PoolTest < WorkhorseTest
7
7
 
8
8
  4.times do |_i|
9
9
  p.post do
10
- sleep 1
10
+ sleep 0.5
11
11
  end
12
12
  end
13
13
 
14
- sleep 0.5
14
+ sleep 0.1
15
15
  assert_equal 1, p.idle
16
16
 
17
- sleep 1
17
+ sleep 0.5
18
18
  assert_equal 5, p.idle
19
19
  end
20
20
  end
21
21
 
22
22
  def test_overflow
23
23
  with_pool 5 do |p|
24
- 5.times { p.post { sleep 1 } }
24
+ 5.times { p.post { sleep 0.5 } }
25
25
 
26
26
  exception = assert_raises do
27
27
  p.post { sleep 1 }
@@ -37,23 +37,21 @@ class Workhorse::PoolTest < WorkhorseTest
37
37
 
38
38
  5.times do
39
39
  p.post do
40
- sleep 1
41
40
  counter.increment
42
41
  end
43
42
  end
44
43
 
45
- sleep 1.2
44
+ sleep 0.01
46
45
 
47
46
  assert_equal 5, counter.value
48
47
 
49
48
  2.times do
50
49
  p.post do
51
- sleep 1
52
50
  counter.increment
53
51
  end
54
52
  end
55
53
 
56
- sleep 1.2
54
+ sleep 0.01
57
55
 
58
56
  assert_equal 7, counter.value
59
57
  end
@@ -6,12 +6,12 @@ class Workhorse::WorkerTest < WorkhorseTest
6
6
  w.start
7
7
  assert_equal 5, w.idle
8
8
 
9
- Workhorse::Enqueuer.enqueue BasicJob.new(sleep_time: 3)
9
+ Workhorse.enqueue BasicJob.new(sleep_time: 0.5)
10
10
 
11
- sleep 2
11
+ sleep 0.1
12
12
  assert_equal 4, w.idle
13
13
 
14
- sleep 3
14
+ sleep 0.5
15
15
  assert_equal 5, w.idle
16
16
 
17
17
  w.shutdown
@@ -29,16 +29,16 @@ class Workhorse::WorkerTest < WorkhorseTest
29
29
  w.shutdown
30
30
  w.shutdown # Should be ignored
31
31
 
32
- Workhorse::Enqueuer.enqueue BasicJob.new
32
+ Workhorse.enqueue BasicJob.new
33
33
  end
34
34
 
35
35
  def test_perform
36
36
  w = Workhorse::Worker.new polling_interval: 1
37
- Workhorse::Enqueuer.enqueue BasicJob.new
37
+ Workhorse.enqueue BasicJob.new(sleep_time: 0.1)
38
38
  assert_equal 'waiting', Workhorse::DbJob.first.state
39
39
 
40
40
  w.start
41
- sleep 2
41
+ sleep 1
42
42
  w.shutdown
43
43
 
44
44
  assert_equal 'succeeded', Workhorse::DbJob.first.state
@@ -47,10 +47,10 @@ class Workhorse::WorkerTest < WorkhorseTest
47
47
  def test_params
48
48
  BasicJob.results.clear
49
49
 
50
- Workhorse::Enqueuer.enqueue BasicJob.new(some_param: 5)
50
+ Workhorse.enqueue BasicJob.new(some_param: 5, sleep_time: 0)
51
51
  w = Workhorse::Worker.new polling_interval: 1
52
52
  w.start
53
- sleep 2
53
+ sleep 0.5
54
54
  w.shutdown
55
55
 
56
56
  assert_equal 'succeeded', Workhorse::DbJob.first.state
@@ -79,17 +79,27 @@ class Workhorse::WorkerTest < WorkhorseTest
79
79
 
80
80
  def test_no_queues
81
81
  enqueue_in_multiple_queues
82
- work 3, queues: [nil, :q1]
82
+ work 1
83
83
 
84
84
  jobs = Workhorse::DbJob.order(queue: :asc).to_a
85
85
  assert_equal 'succeeded', jobs[0].state
86
86
  assert_equal 'succeeded', jobs[1].state
87
+ assert_equal 'succeeded', jobs[2].state
88
+ end
89
+
90
+ def test_nil_queue
91
+ enqueue_in_multiple_queues
92
+ work 1, queues: [nil]
93
+
94
+ jobs = Workhorse::DbJob.order(queue: :asc).to_a
95
+ assert_equal 'succeeded', jobs[0].state
96
+ assert_equal 'waiting', jobs[1].state
87
97
  assert_equal 'waiting', jobs[2].state
88
98
  end
89
99
 
90
100
  def test_queues_with_nil
91
101
  enqueue_in_multiple_queues
92
- work 3, queues: [nil, :q1]
102
+ work 1, queues: [nil, :q1]
93
103
 
94
104
  jobs = Workhorse::DbJob.order(queue: :asc).to_a
95
105
  assert_equal 'succeeded', jobs[0].state
@@ -99,7 +109,7 @@ class Workhorse::WorkerTest < WorkhorseTest
99
109
 
100
110
  def test_queues_without_nil
101
111
  enqueue_in_multiple_queues
102
- work 3, queues: %i[q1 q2]
112
+ work 1, queues: %i[q1 q2]
103
113
 
104
114
  jobs = Workhorse::DbJob.order(queue: :asc).to_a
105
115
  assert_equal 'waiting', jobs[0].state
@@ -110,8 +120,8 @@ class Workhorse::WorkerTest < WorkhorseTest
110
120
  private
111
121
 
112
122
  def enqueue_in_multiple_queues
113
- Workhorse::Enqueuer.enqueue BasicJob.new(some_param: nil)
114
- Workhorse::Enqueuer.enqueue BasicJob.new(some_param: :q1), queue: :q1
115
- Workhorse::Enqueuer.enqueue BasicJob.new(some_param: :q2), queue: :q2
123
+ Workhorse.enqueue BasicJob.new(some_param: nil)
124
+ Workhorse.enqueue BasicJob.new(some_param: :q1), queue: :q1
125
+ Workhorse.enqueue BasicJob.new(some_param: :q2), queue: :q2
116
126
  end
117
127
  end
data/workhorse.gemspec CHANGED
@@ -1,55 +1,54 @@
1
1
  # -*- encoding: utf-8 -*-
2
- # stub: workhorse 0.0.2 ruby lib
3
2
 
4
3
  Gem::Specification.new do |s|
5
- s.name = "workhorse".freeze
6
- s.version = "0.0.2"
4
+ s.name = "workhorse"
5
+ s.version = "0.0.3"
7
6
 
8
- s.required_rubygems_version = Gem::Requirement.new(">= 0".freeze) if s.respond_to? :required_rubygems_version=
9
- s.require_paths = ["lib".freeze]
10
- s.authors = ["Sitrox".freeze]
11
- s.date = "2017-12-14"
12
- s.files = [".gitignore".freeze, ".releaser_config".freeze, ".rubocop.yml".freeze, ".travis.yml".freeze, "Gemfile".freeze, "LICENSE".freeze, "README.md".freeze, "RUBY_VERSION".freeze, "Rakefile".freeze, "VERSION".freeze, "bin/rubocop".freeze, "lib/generators/workhorse/install_generator.rb".freeze, "lib/generators/workhorse/templates/bin/workhorse.rb".freeze, "lib/generators/workhorse/templates/config/initializers/workhorse.rb".freeze, "lib/generators/workhorse/templates/create_table_jobs.rb".freeze, "lib/workhorse.rb".freeze, "lib/workhorse/daemon.rb".freeze, "lib/workhorse/daemon/shell_handler.rb".freeze, "lib/workhorse/db_job.rb".freeze, "lib/workhorse/enqueuer.rb".freeze, "lib/workhorse/jobs/run_rails_op.rb".freeze, "lib/workhorse/performer.rb".freeze, "lib/workhorse/poller.rb".freeze, "lib/workhorse/pool.rb".freeze, "lib/workhorse/worker.rb".freeze, "test/lib/db_schema.rb".freeze, "test/lib/jobs.rb".freeze, "test/lib/test_helper.rb".freeze, "test/workhorse/enqueuer_test.rb".freeze, "test/workhorse/performer_test.rb".freeze, "test/workhorse/poller_test.rb".freeze, "test/workhorse/pool_test.rb".freeze, "test/workhorse/worker_test.rb".freeze, "workhorse.gemspec".freeze]
13
- s.rubygems_version = "2.6.14".freeze
14
- s.summary = "Multi-threaded job backend with database queuing for ruby.".freeze
15
- s.test_files = ["test/lib/db_schema.rb".freeze, "test/lib/jobs.rb".freeze, "test/lib/test_helper.rb".freeze, "test/workhorse/enqueuer_test.rb".freeze, "test/workhorse/performer_test.rb".freeze, "test/workhorse/poller_test.rb".freeze, "test/workhorse/pool_test.rb".freeze, "test/workhorse/worker_test.rb".freeze]
7
+ s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
8
+ s.authors = ["Sitrox"]
9
+ s.date = "2017-12-18"
10
+ s.files = [".gitignore", ".releaser_config", ".rubocop.yml", ".travis.yml", "FAQ.md", "Gemfile", "LICENSE", "README.md", "RUBY_VERSION", "Rakefile", "VERSION", "bin/rubocop", "lib/generators/workhorse/install_generator.rb", "lib/generators/workhorse/templates/bin/workhorse.rb", "lib/generators/workhorse/templates/config/initializers/workhorse.rb", "lib/generators/workhorse/templates/create_table_jobs.rb", "lib/workhorse.rb", "lib/workhorse/daemon.rb", "lib/workhorse/daemon/shell_handler.rb", "lib/workhorse/db_job.rb", "lib/workhorse/enqueuer.rb", "lib/workhorse/jobs/run_rails_op.rb", "lib/workhorse/performer.rb", "lib/workhorse/poller.rb", "lib/workhorse/pool.rb", "lib/workhorse/worker.rb", "test/lib/db_schema.rb", "test/lib/jobs.rb", "test/lib/test_helper.rb", "test/workhorse/enqueuer_test.rb", "test/workhorse/performer_test.rb", "test/workhorse/poller_test.rb", "test/workhorse/pool_test.rb", "test/workhorse/worker_test.rb", "workhorse.gemspec"]
11
+ s.require_paths = ["lib"]
12
+ s.rubygems_version = "2.0.14.1"
13
+ s.summary = "Multi-threaded job backend with database queuing for ruby."
14
+ s.test_files = ["test/lib/db_schema.rb", "test/lib/jobs.rb", "test/lib/test_helper.rb", "test/workhorse/enqueuer_test.rb", "test/workhorse/performer_test.rb", "test/workhorse/poller_test.rb", "test/workhorse/pool_test.rb", "test/workhorse/worker_test.rb"]
16
15
 
17
16
  if s.respond_to? :specification_version then
18
17
  s.specification_version = 4
19
18
 
20
19
  if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then
21
- s.add_development_dependency(%q<bundler>.freeze, ["~> 1.3"])
22
- s.add_development_dependency(%q<rake>.freeze, [">= 0"])
23
- s.add_development_dependency(%q<rubocop>.freeze, ["= 0.51.0"])
24
- s.add_development_dependency(%q<minitest>.freeze, [">= 0"])
25
- s.add_development_dependency(%q<mysql2>.freeze, ["~> 0.3.13"])
26
- s.add_development_dependency(%q<benchmark-ips>.freeze, [">= 0"])
27
- s.add_runtime_dependency(%q<activesupport>.freeze, [">= 0"])
28
- s.add_runtime_dependency(%q<activerecord>.freeze, [">= 0"])
29
- s.add_runtime_dependency(%q<schemacop>.freeze, ["~> 2.0"])
30
- s.add_runtime_dependency(%q<concurrent-ruby>.freeze, [">= 0"])
20
+ s.add_development_dependency(%q<bundler>, ["~> 1.3"])
21
+ s.add_development_dependency(%q<rake>, [">= 0"])
22
+ s.add_development_dependency(%q<rubocop>, ["= 0.51.0"])
23
+ s.add_development_dependency(%q<minitest>, [">= 0"])
24
+ s.add_development_dependency(%q<mysql2>, ["~> 0.3.13"])
25
+ s.add_development_dependency(%q<benchmark-ips>, [">= 0"])
26
+ s.add_runtime_dependency(%q<activesupport>, [">= 0"])
27
+ s.add_runtime_dependency(%q<activerecord>, [">= 0"])
28
+ s.add_runtime_dependency(%q<schemacop>, ["~> 2.0"])
29
+ s.add_runtime_dependency(%q<concurrent-ruby>, [">= 0"])
31
30
  else
32
- s.add_dependency(%q<bundler>.freeze, ["~> 1.3"])
33
- s.add_dependency(%q<rake>.freeze, [">= 0"])
34
- s.add_dependency(%q<rubocop>.freeze, ["= 0.51.0"])
35
- s.add_dependency(%q<minitest>.freeze, [">= 0"])
36
- s.add_dependency(%q<mysql2>.freeze, ["~> 0.3.13"])
37
- s.add_dependency(%q<benchmark-ips>.freeze, [">= 0"])
38
- s.add_dependency(%q<activesupport>.freeze, [">= 0"])
39
- s.add_dependency(%q<activerecord>.freeze, [">= 0"])
40
- s.add_dependency(%q<schemacop>.freeze, ["~> 2.0"])
41
- s.add_dependency(%q<concurrent-ruby>.freeze, [">= 0"])
31
+ s.add_dependency(%q<bundler>, ["~> 1.3"])
32
+ s.add_dependency(%q<rake>, [">= 0"])
33
+ s.add_dependency(%q<rubocop>, ["= 0.51.0"])
34
+ s.add_dependency(%q<minitest>, [">= 0"])
35
+ s.add_dependency(%q<mysql2>, ["~> 0.3.13"])
36
+ s.add_dependency(%q<benchmark-ips>, [">= 0"])
37
+ s.add_dependency(%q<activesupport>, [">= 0"])
38
+ s.add_dependency(%q<activerecord>, [">= 0"])
39
+ s.add_dependency(%q<schemacop>, ["~> 2.0"])
40
+ s.add_dependency(%q<concurrent-ruby>, [">= 0"])
42
41
  end
43
42
  else
44
- s.add_dependency(%q<bundler>.freeze, ["~> 1.3"])
45
- s.add_dependency(%q<rake>.freeze, [">= 0"])
46
- s.add_dependency(%q<rubocop>.freeze, ["= 0.51.0"])
47
- s.add_dependency(%q<minitest>.freeze, [">= 0"])
48
- s.add_dependency(%q<mysql2>.freeze, ["~> 0.3.13"])
49
- s.add_dependency(%q<benchmark-ips>.freeze, [">= 0"])
50
- s.add_dependency(%q<activesupport>.freeze, [">= 0"])
51
- s.add_dependency(%q<activerecord>.freeze, [">= 0"])
52
- s.add_dependency(%q<schemacop>.freeze, ["~> 2.0"])
53
- s.add_dependency(%q<concurrent-ruby>.freeze, [">= 0"])
43
+ s.add_dependency(%q<bundler>, ["~> 1.3"])
44
+ s.add_dependency(%q<rake>, [">= 0"])
45
+ s.add_dependency(%q<rubocop>, ["= 0.51.0"])
46
+ s.add_dependency(%q<minitest>, [">= 0"])
47
+ s.add_dependency(%q<mysql2>, ["~> 0.3.13"])
48
+ s.add_dependency(%q<benchmark-ips>, [">= 0"])
49
+ s.add_dependency(%q<activesupport>, [">= 0"])
50
+ s.add_dependency(%q<activerecord>, [">= 0"])
51
+ s.add_dependency(%q<schemacop>, ["~> 2.0"])
52
+ s.add_dependency(%q<concurrent-ruby>, [">= 0"])
54
53
  end
55
54
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: workhorse
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.2
4
+ version: 0.0.3
5
5
  platform: ruby
6
6
  authors:
7
7
  - Sitrox
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2017-12-14 00:00:00.000000000 Z
11
+ date: 2017-12-18 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: bundler
@@ -160,6 +160,7 @@ files:
160
160
  - ".releaser_config"
161
161
  - ".rubocop.yml"
162
162
  - ".travis.yml"
163
+ - FAQ.md
163
164
  - Gemfile
164
165
  - LICENSE
165
166
  - README.md