kthxbye 1.1.1 → 1.2.0
Sign up to get free protection for your applications and to get access to all the features.
- data/{README.textile → README.rdoc} +55 -42
- data/bin/kthxbye-monitor +25 -0
- data/config.ru +5 -4
- data/lib/kthxbye/config.rb +1 -1
- data/lib/kthxbye/version.rb +1 -1
- data/lib/kthxbye/worker.rb +0 -2
- data/test/test_kthxbye.rb +2 -2
- metadata +7 -9
@@ -1,4 +1,16 @@
|
|
1
|
-
|
1
|
+
== kthxbye
|
2
|
+
|
3
|
+
== Quick Links
|
4
|
+
|
5
|
+
code: http://github.com/plukevdh/kthxbye
|
6
|
+
|
7
|
+
docs: http://rdoc.info/github/plukevdh/kthxbye/
|
8
|
+
|
9
|
+
gem: https://rubygems.org/gems/kthxbye
|
10
|
+
|
11
|
+
issues: http://github.com/plukevdh/kthxbye/issues
|
12
|
+
|
13
|
+
== History
|
2
14
|
|
3
15
|
Kthxbye is the answer to a fairly unique-yet-common problem: Background job
|
4
16
|
processing when we care about the result.
|
@@ -16,7 +28,7 @@ support can take on average one minute or more with a standard deviation of
|
|
16
28
|
almost 2 minutes (with a forced timeout of 5 minutes). That's kinda a long time
|
17
29
|
to sit and wait on a web app.
|
18
30
|
|
19
|
-
|
31
|
+
http://img.skitch.com/20100901-gadna641fj4wdeswgj74y2pssq.png
|
20
32
|
|
21
33
|
We don't really want users sitting waiting for up to 5 minutes (when it forces
|
22
34
|
failure) unable to do anything or (even worse) hitting refresh or the action
|
@@ -29,17 +41,17 @@ require the result to be returned to the web app (think mass-mailers, queue
|
|
29
41
|
population, image resizing). They just run and the output goes to a database,
|
30
42
|
an inbox or a file server.
|
31
43
|
|
32
|
-
|
44
|
+
=== Enter Kthxbye
|
33
45
|
|
34
46
|
Kthxbye is an attempt to solve this problem. It is based heavily off of
|
35
47
|
"Resque":http://github.com/defunkt/resque and why not an addition to Resque?
|
36
48
|
I needed some hack time with Redis on my own as I've never used it before...
|
37
49
|
|
38
|
-
|
39
|
-
|
40
|
-
|
50
|
+
"I can learn any language or tool in a matter of days if you give me
|
51
|
+
1. a good manual
|
52
|
+
2. an even better project to work on."
|
41
53
|
|
42
|
-
|
54
|
+
-Prof. Shumacher
|
43
55
|
|
44
56
|
This project accomplishes both those goals. This is an attempt to learn
|
45
57
|
something, using Resque and Redis docs as a manual, while at the same time
|
@@ -47,54 +59,54 @@ creating a much needed solution to a problem.
|
|
47
59
|
|
48
60
|
The idea is to be able to do the following:
|
49
61
|
|
50
|
-
|
51
|
-
|
52
|
-
|
53
|
-
|
54
|
-
|
55
|
-
end
|
62
|
+
# dummy job class
|
63
|
+
class MyJob
|
64
|
+
def self.perform(data)
|
65
|
+
puts "Do something with #{data}"
|
66
|
+
data.gsub(/hello/i, "Goodbye")
|
56
67
|
end
|
68
|
+
end
|
57
69
|
|
58
|
-
|
59
|
-
|
60
|
-
|
61
|
-
|
62
|
-
|
70
|
+
# setup options, then connect
|
71
|
+
Kthxbye::Config.setup(:redis_server => 'localhost', :redis_port => 8080)
|
72
|
+
|
73
|
+
# each enqueued job returns a unique id to poll with
|
74
|
+
unique_id = Kthxbye.enqueue("jobs", MyJob, "Hello World")
|
63
75
|
|
64
|
-
|
76
|
+
# ... code code code ...
|
65
77
|
|
66
|
-
|
67
|
-
|
78
|
+
# polls queue every 5 seconds
|
79
|
+
computed_value = Kthxbye.poll("jobs", unique_id, 5)
|
68
80
|
|
69
81
|
and then in some other world, on some other machine, (that still has knowledge of MyJob)
|
70
82
|
|
71
|
-
|
72
|
-
|
83
|
+
# inits with queue
|
84
|
+
worker = Kthxbye::Worker.new("jobs")
|
85
|
+
|
86
|
+
# connects to queue and runs jobs found there
|
87
|
+
worker.run
|
73
88
|
|
74
|
-
|
75
|
-
worker.run
|
89
|
+
*Pretty... damn... simple.*
|
76
90
|
|
77
|
-
|
91
|
+
== Installation
|
78
92
|
|
79
|
-
h2. Installation
|
80
93
|
Installation isn't hard. Simply:
|
81
94
|
|
82
|
-
|
95
|
+
gem install kthxbye
|
83
96
|
|
84
97
|
or in Rails
|
85
98
|
|
86
|
-
|
99
|
+
gem "kthxbye"
|
87
100
|
|
88
101
|
and then in your project directory run
|
89
102
|
|
90
|
-
|
103
|
+
rails g kthxbye
|
91
104
|
|
92
105
|
which will install all of the necessary assets, of which there are three:
|
93
|
-
|
94
|
-
from the worker
|
95
|
-
|
96
|
-
|
97
|
-
# public/stylesheets/kthxbye.css - styles for the widget
|
106
|
+
|
107
|
+
1. public/javascripts/kthxbye.js - a responder that will catch job status message from the worker
|
108
|
+
2. public/images/kthxbye_widget.png - a notifier widget to show the status of a job
|
109
|
+
3. public/stylesheets/kthxbye.css - styles for the widget
|
98
110
|
|
99
111
|
These can be ignored if you really don't want to have an asynchronus widget to
|
100
112
|
display the job's status to the user in real-time. You can simply use the gem
|
@@ -104,8 +116,9 @@ If are using the widget however, we depend on "node.js":http://nodejs.org/ to
|
|
104
116
|
handle the Redis PUBSUB listening and updating and "Socket.io":http://socket.io/
|
105
117
|
to stream the results back to the clients.
|
106
118
|
|
107
|
-
In order to install Socket.io on your application side, run
|
108
|
-
|
119
|
+
In order to install Socket.io on your application side, run
|
120
|
+
|
121
|
+
git clone http://github.com/LearnBoost/Socket.IO.git socket-io --recursive
|
109
122
|
|
110
123
|
*Note:* due to the way Rails tries to serve these files and the seemingly
|
111
124
|
conflicting way that Socket.io has hardcoded some paths into the code, you may
|
@@ -120,8 +133,9 @@ via "git clone git://github.com/LearnBoost/Socket.IO-node.git socket.io-node --r
|
|
120
133
|
(from the socket.io install instructions http://github.com/LearnBoost/Socket.IO-node)
|
121
134
|
|
122
135
|
|
123
|
-
|
124
|
-
|
136
|
+
== Troubleshooting
|
137
|
+
|
138
|
+
=== The part of the show where I try to save you some trouble
|
125
139
|
|
126
140
|
There are a few things I found along the way that were necessary to make this
|
127
141
|
integration run super smoothly:
|
@@ -133,8 +147,7 @@ You may need to create a link in your public/ dir to socket.io to the install
|
|
133
147
|
in your public/javascripts/ dir.
|
134
148
|
|
135
149
|
|
136
|
-
|
137
|
-
h2. Note on Patches/Pull Requests
|
150
|
+
== Note on Patches/Pull Requests
|
138
151
|
|
139
152
|
* Fork the project.
|
140
153
|
* Make your feature addition or bug fix.
|
@@ -144,6 +157,6 @@ h2. Note on Patches/Pull Requests
|
|
144
157
|
(if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
|
145
158
|
* Send me a pull request. Bonus points for topic branches.
|
146
159
|
|
147
|
-
|
160
|
+
== Copyright
|
148
161
|
|
149
162
|
Copyright (c) 2010 Luke van der Hoeven. See LICENSE for details.
|
data/bin/kthxbye-monitor
ADDED
@@ -0,0 +1,25 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
$LOAD_PATH.unshift File.expand_path(File.dirname(__FILE__) + '/../lib')
|
3
|
+
|
4
|
+
begin
|
5
|
+
require 'vegas'
|
6
|
+
rescue
|
7
|
+
require 'rubygems'
|
8
|
+
require 'vegas'
|
9
|
+
end
|
10
|
+
|
11
|
+
require 'kthxbye/web_interface'
|
12
|
+
require 'kthxbye/config'
|
13
|
+
|
14
|
+
Vegas::Runner.new(Kthxbye::WebInterface, 'kthxbye-monitor') do |runner, args, app|
|
15
|
+
Kthxbye::Config.setup
|
16
|
+
|
17
|
+
args.on('-S SERVER', '--server SERVER', "set the redis server") {|server|
|
18
|
+
Kthxbye::Config.options[:redis_server] = server
|
19
|
+
}
|
20
|
+
args.on('-P PORT', '--port PORT', "set the redis port") {|port|
|
21
|
+
Kthxbye::Config.options[:redis_port] = port
|
22
|
+
}
|
23
|
+
|
24
|
+
end
|
25
|
+
|
data/config.ru
CHANGED
@@ -1,10 +1,11 @@
|
|
1
1
|
#!/usr/bin/env ruby
|
2
2
|
|
3
3
|
$LOAD_PATH.unshift ::File.expand_path(::File.dirname(__FILE__) + '/lib')
|
4
|
-
require 'kthxbye'
|
5
4
|
require 'kthxbye/web_interface'
|
6
5
|
|
7
|
-
|
8
|
-
|
6
|
+
if ENV['KTHXBYECONFIG'] && ::File.exists?(::File.expand_path(ENV['KTHXBYECONFIG']))
|
7
|
+
load ::File.expand_path(ENV['KTHXBYECONFIG'])
|
8
|
+
end
|
9
9
|
|
10
|
-
|
10
|
+
use Rack::ShowExceptions
|
11
|
+
Kthxbye::WebInterface.new
|
data/lib/kthxbye/config.rb
CHANGED
@@ -20,7 +20,7 @@ module Kthxbye
|
|
20
20
|
#
|
21
21
|
# NEEDS TO BE REWORKED FOR RAILS APPS
|
22
22
|
def self.setup( args=nil )
|
23
|
-
@options
|
23
|
+
@options ||= DEFAULT.dup
|
24
24
|
@options.merge!( YAML.load('./config/kthxbye.yaml') ) if File.exist?( './config/kthxbye.yaml' )
|
25
25
|
@options.merge!( args ) if args
|
26
26
|
end
|
data/lib/kthxbye/version.rb
CHANGED
data/lib/kthxbye/worker.rb
CHANGED
data/test/test_kthxbye.rb
CHANGED
@@ -4,7 +4,7 @@ class TestKthxbye < Test::Unit::TestCase
|
|
4
4
|
context "See Kthxbye Configuration" do
|
5
5
|
|
6
6
|
should "configure an app with given params" do
|
7
|
-
|
7
|
+
Kthxbye::Config.setup(:redis_server => "localhost", :redis_port => 8080, :verbose => true)
|
8
8
|
|
9
9
|
assert_equal 'localhost', Kthxbye::Config.options[:redis_server]
|
10
10
|
assert_equal 8080, Kthxbye::Config.options[:redis_port]
|
@@ -23,7 +23,7 @@ class TestKthxbye < Test::Unit::TestCase
|
|
23
23
|
|
24
24
|
context "See Kthxbye" do
|
25
25
|
setup do
|
26
|
-
Kthxbye::Config.setup
|
26
|
+
Kthxbye::Config.setup(:verbose => false)
|
27
27
|
Kthxbye.redis.flushall
|
28
28
|
end
|
29
29
|
|
metadata
CHANGED
@@ -1,13 +1,12 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: kthxbye
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
hash: 17
|
5
4
|
prerelease: false
|
6
5
|
segments:
|
7
6
|
- 1
|
8
|
-
-
|
9
|
-
-
|
10
|
-
version: 1.
|
7
|
+
- 2
|
8
|
+
- 0
|
9
|
+
version: 1.2.0
|
11
10
|
platform: ruby
|
12
11
|
authors:
|
13
12
|
- Luke van der Hoeven
|
@@ -15,7 +14,7 @@ autorequire:
|
|
15
14
|
bindir: bin
|
16
15
|
cert_chain: []
|
17
16
|
|
18
|
-
date: 2010-10-
|
17
|
+
date: 2010-10-08 00:00:00 -04:00
|
19
18
|
default_executable:
|
20
19
|
dependencies: []
|
21
20
|
|
@@ -27,8 +26,9 @@ extensions: []
|
|
27
26
|
|
28
27
|
extra_rdoc_files:
|
29
28
|
- LICENSE
|
30
|
-
- README.
|
29
|
+
- README.rdoc
|
31
30
|
files:
|
31
|
+
- bin/kthxbye-monitor
|
32
32
|
- lib/kthxbye/version.rb
|
33
33
|
- lib/kthxbye/job.rb
|
34
34
|
- lib/kthxbye/exceptions.rb
|
@@ -61,7 +61,7 @@ files:
|
|
61
61
|
- lib/generators/kthxbye/kthxbye_generator.rb
|
62
62
|
- lib/kthxbye.rb
|
63
63
|
- LICENSE
|
64
|
-
- README.
|
64
|
+
- README.rdoc
|
65
65
|
- DESIGN.textile
|
66
66
|
- config.ru
|
67
67
|
- Gemfile
|
@@ -85,7 +85,6 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
85
85
|
requirements:
|
86
86
|
- - ">="
|
87
87
|
- !ruby/object:Gem::Version
|
88
|
-
hash: 3
|
89
88
|
segments:
|
90
89
|
- 0
|
91
90
|
version: "0"
|
@@ -94,7 +93,6 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
94
93
|
requirements:
|
95
94
|
- - ">="
|
96
95
|
- !ruby/object:Gem::Version
|
97
|
-
hash: 23
|
98
96
|
segments:
|
99
97
|
- 1
|
100
98
|
- 3
|