chore-core 1.8.0 → 1.8.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +89 -2
- data/lib/chore/hooks.rb +7 -1
- data/lib/chore/unit_of_work.rb +2 -2
- data/lib/chore/version.rb +1 -1
- data/lib/chore/worker.rb +5 -3
- metadata +2 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: e8debf965223bd61c1b247c0c275c3a4f758085f
|
4
|
+
data.tar.gz: 51b646120ff568753112859186b6714a3846a819
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 6b2b1f4e194ea2704734b212255135f32c504937b4152e6bc490a25401861d34da64f38e0a8e2ca2f8c7b0be5a5f1a45dd8553fbcbcf7aa0a86028c569ac5b00
|
7
|
+
data.tar.gz: 264df2f96cc45d6a83a48688b602c96ee4bb2df19ad23b3d2a26c2476276ee8545798ece8e94bda0edf1a6d338e35d10fe9718d72e1ca03ff3ecfe970cb0b85a
|
data/README.md
CHANGED
@@ -52,11 +52,32 @@ Note that you can use one or the other but not both. Chore will quit and make fu
|
|
52
52
|
|
53
53
|
### Tips for configuring Chore
|
54
54
|
|
55
|
+
For Rails, it can be necessary to add the directory you store your jobs in to the eager loading path,
|
56
|
+
found in application.rb. You likely need a similar approach for most apps using jobs, unless you places
|
57
|
+
them into a directory that is already eager loaded by default. One example of this might be:
|
58
|
+
|
59
|
+
```ruby
|
60
|
+
config.eager_load_paths += File.join(config.root, "app", "jobs")
|
61
|
+
```
|
62
|
+
|
63
|
+
However, due to the way eager_load_paths works in Rails, this may only solve the issue
|
64
|
+
in your production environment. You might also find it useful for other environments
|
65
|
+
to throw soemthing like this in an config/initializers/chore.rb file, although you
|
66
|
+
can choose to load the job files in any way you like:
|
67
|
+
|
68
|
+
```ruby
|
69
|
+
Dir["#{Rails.root}/app/jobs/**/*"].each do |file|
|
70
|
+
require file unless File.directory?(file)
|
71
|
+
end unless Rails.env.production?
|
72
|
+
```
|
73
|
+
|
74
|
+
### Producing and Consuming Jobs
|
75
|
+
|
55
76
|
When it comes to configuring Chore, you have 2 main use cases - as a producer of messages, or as a consumer of messages (the consumer is also able to produce messages if need be, but is running as it's own isolated instance of your application).
|
56
77
|
|
57
78
|
For producers, you must do all of your Chore configuration in an intializer.
|
58
79
|
|
59
|
-
For consumers, you need to either Chorefile or Chorefile + an initializer.
|
80
|
+
For consumers, you need to either use a Chorefile or Chorefile + an initializer.
|
60
81
|
|
61
82
|
Because you are likely to use the same app as the basis for both producing and consuming messages, you'll already have a considerable amount of configuration in your Producer - it makes sense to use Chorefile to simply provide the `require` option, and stick to the initializer for the rest of the configuration to keep things DRY.
|
62
83
|
|
@@ -206,7 +227,7 @@ class TestJob
|
|
206
227
|
def perform(args={})
|
207
228
|
# Do something cool
|
208
229
|
end
|
209
|
-
|
230
|
+
|
210
231
|
def before_perform_log(message)
|
211
232
|
Chore.logger.debug "About to do something cool with: #{message.inspect}"
|
212
233
|
end
|
@@ -281,6 +302,72 @@ Chore has several plugin gems available, which extend it's core functionality
|
|
281
302
|
|
282
303
|
[Airbrake](https://github.com/Tapjoy/chore-airbrake) - Integrating Chore with Airbrake
|
283
304
|
|
305
|
+
## Managing Chore processes
|
306
|
+
|
307
|
+
### Sample Upstart
|
308
|
+
|
309
|
+
There are lots of ways to create upstart scripts, so it's difficult to give a prescriptive
|
310
|
+
example of the "right" way to do it. However, here are some ideas from how we run it in production
|
311
|
+
at Tapjoy:
|
312
|
+
|
313
|
+
You should have a specific user that the process runs under, for security reasons. Swap to
|
314
|
+
this user at the beginning of your exec line
|
315
|
+
|
316
|
+
```bash
|
317
|
+
su - $USER --command '...'
|
318
|
+
```
|
319
|
+
|
320
|
+
For the command to run Chore itself keeping all of the necessary environment variables in an env
|
321
|
+
file that your upstart can source on it's exec line, to prevent having to mix changing environment
|
322
|
+
variables with having to change the upstart script itself
|
323
|
+
|
324
|
+
```bash
|
325
|
+
source $PATHTOENVVARS ;
|
326
|
+
```
|
327
|
+
|
328
|
+
After that, you'll want to make sure you're running Chore under the right ruby version. Additionally,
|
329
|
+
we like to redirect STDOUT and STDERR to logger, with an app name. This makes it easy to find
|
330
|
+
information in syslog later on. Putting that all together looks like:
|
331
|
+
|
332
|
+
```bash
|
333
|
+
rvm use $RUBYVERSION do bundle exec chore -c Chorefile 2>&1 | logger -t $APPNAME
|
334
|
+
```
|
335
|
+
|
336
|
+
There are many other ways you could manage the Upstart file, but these are a few of the ways we
|
337
|
+
prefer to do it. Putting it all together, it looks something like:
|
338
|
+
|
339
|
+
```bash
|
340
|
+
exec su - special_user --command 'source /the/path/to/env ; rvm use ruby-1.9.3-p484 do bundle exec chore -c Chorefile 2>&1 | logger chore-app'
|
341
|
+
```
|
342
|
+
|
343
|
+
### Locating Processes
|
344
|
+
|
345
|
+
As Chore does not keep a PIDfile, and has both a master and a potential number of workers,
|
346
|
+
you may find it difficult to isolate the exact PID for the master process.
|
347
|
+
|
348
|
+
To find Chore master processes via ```ps```, you can run the following:
|
349
|
+
|
350
|
+
```bash
|
351
|
+
ps aux | grep bin/chore
|
352
|
+
```
|
353
|
+
|
354
|
+
or
|
355
|
+
|
356
|
+
```bash
|
357
|
+
pgrep -f bin/chore
|
358
|
+
```
|
359
|
+
|
360
|
+
To find a list of only Chore worker processes:
|
361
|
+
|
362
|
+
```bash
|
363
|
+
ps aux | grep chore-
|
364
|
+
```
|
365
|
+
|
366
|
+
or
|
367
|
+
|
368
|
+
```bash
|
369
|
+
pgrep -f chore-
|
370
|
+
```
|
284
371
|
## Copyright
|
285
372
|
|
286
373
|
Copyright (c) 2013 - 2015 Tapjoy. See LICENSE.txt for
|
data/lib/chore/hooks.rb
CHANGED
@@ -15,7 +15,13 @@ module Chore
|
|
15
15
|
|
16
16
|
private
|
17
17
|
def hooks_for(event)
|
18
|
-
|
18
|
+
candidate_methods.grep(/^#{event}/).sort
|
19
|
+
end
|
20
|
+
|
21
|
+
# NOTE: Any hook methods defined after this is first referenced (i.e.,
|
22
|
+
# after chore begins processing jobs) will not be called.
|
23
|
+
def candidate_methods
|
24
|
+
@_chore_hooks_candidate_methods ||= (self.methods - Object.methods)
|
19
25
|
end
|
20
26
|
|
21
27
|
def global_hooks_for(event)
|
data/lib/chore/unit_of_work.rb
CHANGED
@@ -8,10 +8,10 @@ module Chore
|
|
8
8
|
# * +:previous_attempts+ The number of times the work has been attempted previously.
|
9
9
|
# * +:consumer+ The consumer instance used to fetch this message. Most queue implementations won't need access to this, but some (RabbitMQ) will. So we
|
10
10
|
# make sure to pass it along with each message. This instance will be used by the Worker for things like <tt>complete</tt> and </tt>reject</tt>.
|
11
|
-
class UnitOfWork < Struct.new(:id,:queue_name,:queue_timeout,:message,:previous_attempts,:consumer)
|
11
|
+
class UnitOfWork < Struct.new(:id,:queue_name,:queue_timeout,:message,:previous_attempts,:consumer,:decoded_message, :klass)
|
12
12
|
# The current attempt number for the worker processing this message.
|
13
13
|
def current_attempt
|
14
14
|
previous_attempts + 1
|
15
15
|
end
|
16
16
|
end
|
17
|
-
end
|
17
|
+
end
|
data/lib/chore/version.rb
CHANGED
data/lib/chore/worker.rb
CHANGED
@@ -56,6 +56,8 @@ module Chore
|
|
56
56
|
@work.each do |item|
|
57
57
|
return if @stopping
|
58
58
|
begin
|
59
|
+
item.decoded_message = options[:payload_handler].decode(item.message)
|
60
|
+
item.klass = options[:payload_handler].payload_class(item.decoded_message)
|
59
61
|
start_item(item)
|
60
62
|
rescue => e
|
61
63
|
Chore.logger.error { "Failed to run job for #{item.message} with error: #{e.message} #{e.backtrace * "\n"}" }
|
@@ -76,9 +78,9 @@ module Chore
|
|
76
78
|
|
77
79
|
private
|
78
80
|
def start_item(item)
|
79
|
-
|
80
|
-
|
81
|
-
return unless klass.run_hooks_for(:before_perform,message)
|
81
|
+
klass = item.klass
|
82
|
+
message = item.decoded_message
|
83
|
+
return unless klass.run_hooks_for(:before_perform, message)
|
82
84
|
|
83
85
|
begin
|
84
86
|
Chore.logger.info { "Running job #{klass} with params #{message}"}
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: chore-core
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.8.
|
4
|
+
version: 1.8.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Tapjoy
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2016-01-22 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: json
|