stalker 0.2.0 → 0.2.1
Sign up to get free protection for your applications and to get access to all the features.
- data/README.md +89 -0
- data/VERSION +1 -1
- data/examples/enqueue.rb +1 -1
- data/examples/jobs.rb +1 -1
- data/lib/stalker.rb +18 -6
- metadata +6 -5
data/README.md
ADDED
@@ -0,0 +1,89 @@
|
|
1
|
+
Stalker - a job queueing DSL for Beanstalk
|
2
|
+
==========================================
|
3
|
+
|
4
|
+
[Beanstalkd](http://kr.github.com/beanstalkd/) is a fast, lightweight queueing backend inspired by mmemcached. The [Ruby Beanstalk client](http://beanstalk.rubyforge.org/) is a bit raw, however, so Stalker provides a thin wrapper to make job queueing from your Ruby app easy and fun.
|
5
|
+
|
6
|
+
Queueing jobs
|
7
|
+
-------------
|
8
|
+
|
9
|
+
From anywhere in your app:
|
10
|
+
|
11
|
+
require 'stalker'
|
12
|
+
|
13
|
+
Stalker.enqueue('email.send', :to => 'joe@example.com')
|
14
|
+
Stalker.enqueue('post.cleanup.all')
|
15
|
+
Stalker.enqueue('post.cleanup', :id => post.id)
|
16
|
+
|
17
|
+
Working jobs
|
18
|
+
------------
|
19
|
+
|
20
|
+
In a standalone file, typically jobs.rb or worker.rb:
|
21
|
+
|
22
|
+
require 'stalker'
|
23
|
+
include Stalker
|
24
|
+
|
25
|
+
job 'email.send' do |args|
|
26
|
+
Pony.send(:to => args['to'], :subject => "Hello there")
|
27
|
+
end
|
28
|
+
|
29
|
+
job 'post.cleanup.all' do |args|
|
30
|
+
Post.all.each do |post|
|
31
|
+
enqueue('post.cleanup', :id => post.all)
|
32
|
+
end
|
33
|
+
end
|
34
|
+
|
35
|
+
job 'post.cleanup' do |args|
|
36
|
+
Post.find(args['id']).cleanup
|
37
|
+
end
|
38
|
+
|
39
|
+
Running
|
40
|
+
-------
|
41
|
+
|
42
|
+
First, make sure you have Beanstalkd installed and running:
|
43
|
+
|
44
|
+
$ sudo port install beanstalkd
|
45
|
+
$ beanstalkd
|
46
|
+
|
47
|
+
Stalker:
|
48
|
+
|
49
|
+
$ sudo gem install stalker
|
50
|
+
|
51
|
+
Now run a worker using the stalk binary:
|
52
|
+
|
53
|
+
$ stalk jobs.rb
|
54
|
+
[Thu May 13 01:08:19 -0700 2010] Working 3 jobs: [ email.send post.cleanup.all post.cleanup ]
|
55
|
+
[Thu May 13 01:08:21 -0700 2010] -> send.email (email=hello@example.com)
|
56
|
+
[Thu May 13 01:08:21 -0700 2010] -> send.email finished in 31ms
|
57
|
+
|
58
|
+
Stalker will log to stdout as it starts working each job, and then again when the job finishes including the ellapsed time in milliseconds.
|
59
|
+
|
60
|
+
Filter to a list of jobs you wish to run with an argument:
|
61
|
+
|
62
|
+
$ stalk jobs.rb post.cleanup.all,post.cleanup
|
63
|
+
[Sat Apr 17 14:13:40 -0700 2010] Working 2 jobs: [ post.cleanup.all post.cleanup ]
|
64
|
+
|
65
|
+
In a production environment you may run one or more high-priority workers (limited to short/urgent jobs) and any number of regular workers (working all jobs). For example, two workers working just the email.send job, and four running all jobs:
|
66
|
+
|
67
|
+
$ for i in 1 2; do stalk jobs.rb email.send > log/urgent-worker.log 2>&1; end
|
68
|
+
$ for i in 1 2 3 4; do stalk jobs.rb > log/worker.log 2>&1; end
|
69
|
+
|
70
|
+
Tidbits
|
71
|
+
-------
|
72
|
+
|
73
|
+
* Jobs are serialized as JSON, so you should stick to strings, integers, arrays, and hashes as arguments to jobs. e.g. don't pass full Ruby objects - use something like an ActiveRecord/MongoMapper/CouchRest id instead.
|
74
|
+
* Because there are no class definitions associated with jobs, you can queue jobs from anywhere without needing to include your full app's environment.
|
75
|
+
* If you need to change the location of your Beanstalk from the default (localhost:11300), set BEANSTALK_URL in your environment, e.g. export BEANSTALK_URL=beanstalk://example.com:11300/
|
76
|
+
* The stalk binary is just for convenience, you can also run a worker with a straight Ruby command:
|
77
|
+
$ ruby -r jobs -e Stalker.work
|
78
|
+
|
79
|
+
Meta
|
80
|
+
----
|
81
|
+
|
82
|
+
Created by Adam Wiggins
|
83
|
+
|
84
|
+
Heavily inspired by [Minion](http://github.com/orionz/minion) by Orion Henry
|
85
|
+
|
86
|
+
Released under the MIT License: http://www.opensource.org/licenses/mit-license.php
|
87
|
+
|
88
|
+
http://github.com/adamwiggins/stalker
|
89
|
+
|
data/VERSION
CHANGED
@@ -1 +1 @@
|
|
1
|
-
0.2.
|
1
|
+
0.2.1
|
data/examples/enqueue.rb
CHANGED
data/examples/jobs.rb
CHANGED
data/lib/stalker.rb
CHANGED
@@ -27,7 +27,7 @@ module Stalker
|
|
27
27
|
raise(NoSuchJob, job) unless @@handlers[job]
|
28
28
|
end
|
29
29
|
|
30
|
-
log "Working #{jobs.size} jobs
|
30
|
+
log "Working #{jobs.size} jobs: [ #{jobs.join(' ')} ]"
|
31
31
|
|
32
32
|
beanstalk.list_tubes_watched.each { |tube| beanstalk.ignore(tube) }
|
33
33
|
jobs.each { |job| beanstalk.watch(job) }
|
@@ -40,22 +40,34 @@ module Stalker
|
|
40
40
|
def work_one_job
|
41
41
|
job = beanstalk.reserve
|
42
42
|
name, args = JSON.parse job.body
|
43
|
-
|
43
|
+
log_job_begin(name, args)
|
44
44
|
handler = @@handlers[name]
|
45
45
|
raise(NoSuchJob, name) unless handler
|
46
46
|
handler.call(args)
|
47
47
|
job.delete
|
48
|
+
log_job_end(name)
|
48
49
|
rescue => e
|
49
50
|
STDERR.puts exception_message(e)
|
50
51
|
job.bury
|
51
52
|
end
|
52
53
|
|
53
|
-
def
|
54
|
-
args_flat = args.
|
55
|
-
accum
|
54
|
+
def log_job_begin(name, args)
|
55
|
+
args_flat = unless args.empty?
|
56
|
+
'(' + args.inject([]) do |accum, (key,value)|
|
57
|
+
accum << "#{key}=#{value}"
|
58
|
+
end.join(' ') + ')'
|
59
|
+
else
|
60
|
+
''
|
56
61
|
end
|
57
62
|
|
58
|
-
log
|
63
|
+
log [ "->", name, args_flat ].join(' ')
|
64
|
+
@job_begun = Time.now
|
65
|
+
end
|
66
|
+
|
67
|
+
def log_job_end(name)
|
68
|
+
ellapsed = Time.now - @job_begun
|
69
|
+
ms = (ellapsed.to_f * 1000).to_i
|
70
|
+
log "-> #{name} finished in #{ms}ms"
|
59
71
|
end
|
60
72
|
|
61
73
|
def log(msg)
|
metadata
CHANGED
@@ -5,8 +5,8 @@ version: !ruby/object:Gem::Version
|
|
5
5
|
segments:
|
6
6
|
- 0
|
7
7
|
- 2
|
8
|
-
-
|
9
|
-
version: 0.2.
|
8
|
+
- 1
|
9
|
+
version: 0.2.1
|
10
10
|
platform: ruby
|
11
11
|
authors:
|
12
12
|
- Adam Wiggins
|
@@ -14,7 +14,7 @@ autorequire:
|
|
14
14
|
bindir: bin
|
15
15
|
cert_chain: []
|
16
16
|
|
17
|
-
date: 2010-
|
17
|
+
date: 2010-05-13 00:00:00 -07:00
|
18
18
|
default_executable: stalk
|
19
19
|
dependencies:
|
20
20
|
- !ruby/object:Gem::Dependency
|
@@ -35,9 +35,10 @@ executables:
|
|
35
35
|
- stalk
|
36
36
|
extensions: []
|
37
37
|
|
38
|
-
extra_rdoc_files:
|
39
|
-
|
38
|
+
extra_rdoc_files:
|
39
|
+
- README.md
|
40
40
|
files:
|
41
|
+
- README.md
|
41
42
|
- Rakefile
|
42
43
|
- VERSION
|
43
44
|
- bin/stalk
|