kthxbye 1.1.1 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/{README.textile → README.rdoc} +55 -42
- data/bin/kthxbye-monitor +25 -0
- data/config.ru +5 -4
- data/lib/kthxbye/config.rb +1 -1
- data/lib/kthxbye/version.rb +1 -1
- data/lib/kthxbye/worker.rb +0 -2
- data/test/test_kthxbye.rb +2 -2
- metadata +7 -9
@@ -1,4 +1,16 @@
|
|
1
|
-
|
1
|
+
== kthxbye
|
2
|
+
|
3
|
+
== Quick Links
|
4
|
+
|
5
|
+
code: http://github.com/plukevdh/kthxbye
|
6
|
+
|
7
|
+
docs: http://rdoc.info/github/plukevdh/kthxbye/
|
8
|
+
|
9
|
+
gem: https://rubygems.org/gems/kthxbye
|
10
|
+
|
11
|
+
issues: http://github.com/plukevdh/kthxbye/issues
|
12
|
+
|
13
|
+
== History
|
2
14
|
|
3
15
|
Kthxbye is the answer to a fairly unique-yet-common problem: Background job
|
4
16
|
processing when we care about the result.
|
@@ -16,7 +28,7 @@ support can take on average one minute or more with a standard deviation of
|
|
16
28
|
almost 2 minutes (with a forced timeout of 5 minutes). That's kinda a long time
|
17
29
|
to sit and wait on a web app.
|
18
30
|
|
19
|
-
|
31
|
+
http://img.skitch.com/20100901-gadna641fj4wdeswgj74y2pssq.png
|
20
32
|
|
21
33
|
We don't really want users sitting waiting for up to 5 minutes (when it forces
|
22
34
|
failure) unable to do anything or (even worse) hitting refresh or the action
|
@@ -29,17 +41,17 @@ require the result to be returned to the web app (think mass-mailers, queue
|
|
29
41
|
population, image resizing). They just run and the output goes to a database,
|
30
42
|
an inbox or a file server.
|
31
43
|
|
32
|
-
|
44
|
+
=== Enter Kthxbye
|
33
45
|
|
34
46
|
Kthxbye is an attempt to solve this problem. It is based heavily off of
|
35
47
|
"Resque":http://github.com/defunkt/resque and why not an addition to Resque?
|
36
48
|
I needed some hack time with Redis on my own as I've never used it before...
|
37
49
|
|
38
|
-
|
39
|
-
|
40
|
-
|
50
|
+
"I can learn any language or tool in a matter of days if you give me
|
51
|
+
1. a good manual
|
52
|
+
2. an even better project to work on."
|
41
53
|
|
42
|
-
|
54
|
+
-Prof. Shumacher
|
43
55
|
|
44
56
|
This project accomplishes both those goals. This is an attempt to learn
|
45
57
|
something, using Resque and Redis docs as a manual, while at the same time
|
@@ -47,54 +59,54 @@ creating a much needed solution to a problem.
|
|
47
59
|
|
48
60
|
The idea is to be able to do the following:
|
49
61
|
|
50
|
-
|
51
|
-
|
52
|
-
|
53
|
-
|
54
|
-
|
55
|
-
end
|
62
|
+
# dummy job class
|
63
|
+
class MyJob
|
64
|
+
def self.perform(data)
|
65
|
+
puts "Do something with #{data}"
|
66
|
+
data.gsub(/hello/i, "Goodbye")
|
56
67
|
end
|
68
|
+
end
|
57
69
|
|
58
|
-
|
59
|
-
|
60
|
-
|
61
|
-
|
62
|
-
|
70
|
+
# setup options, then connect
|
71
|
+
Kthxbye::Config.setup(:redis_server => 'localhost', :redis_port => 8080)
|
72
|
+
|
73
|
+
# each enqueued job returns a unique id to poll with
|
74
|
+
unique_id = Kthxbye.enqueue("jobs", MyJob, "Hello World")
|
63
75
|
|
64
|
-
|
76
|
+
# ... code code code ...
|
65
77
|
|
66
|
-
|
67
|
-
|
78
|
+
# polls queue every 5 seconds
|
79
|
+
computed_value = Kthxbye.poll("jobs", unique_id, 5)
|
68
80
|
|
69
81
|
and then in some other world, on some other machine, (that still has knowledge of MyJob)
|
70
82
|
|
71
|
-
|
72
|
-
|
83
|
+
# inits with queue
|
84
|
+
worker = Kthxbye::Worker.new("jobs")
|
85
|
+
|
86
|
+
# connects to queue and runs jobs found there
|
87
|
+
worker.run
|
73
88
|
|
74
|
-
|
75
|
-
worker.run
|
89
|
+
*Pretty... damn... simple.*
|
76
90
|
|
77
|
-
|
91
|
+
== Installation
|
78
92
|
|
79
|
-
h2. Installation
|
80
93
|
Installation isn't hard. Simply:
|
81
94
|
|
82
|
-
|
95
|
+
gem install kthxbye
|
83
96
|
|
84
97
|
or in Rails
|
85
98
|
|
86
|
-
|
99
|
+
gem "kthxbye"
|
87
100
|
|
88
101
|
and then in your project directory run
|
89
102
|
|
90
|
-
|
103
|
+
rails g kthxbye
|
91
104
|
|
92
105
|
which will install all of the necessary assets, of which there are three:
|
93
|
-
|
94
|
-
from the worker
|
95
|
-
|
96
|
-
|
97
|
-
# public/stylesheets/kthxbye.css - styles for the widget
|
106
|
+
|
107
|
+
1. public/javascripts/kthxbye.js - a responder that will catch job status message from the worker
|
108
|
+
2. public/images/kthxbye_widget.png - a notifier widget to show the status of a job
|
109
|
+
3. public/stylesheets/kthxbye.css - styles for the widget
|
98
110
|
|
99
111
|
These can be ignored if you really don't want to have an asynchronus widget to
|
100
112
|
display the job's status to the user in real-time. You can simply use the gem
|
@@ -104,8 +116,9 @@ If are using the widget however, we depend on "node.js":http://nodejs.org/ to
|
|
104
116
|
handle the Redis PUBSUB listening and updating and "Socket.io":http://socket.io/
|
105
117
|
to stream the results back to the clients.
|
106
118
|
|
107
|
-
In order to install Socket.io on your application side, run
|
108
|
-
|
119
|
+
In order to install Socket.io on your application side, run
|
120
|
+
|
121
|
+
git clone http://github.com/LearnBoost/Socket.IO.git socket-io --recursive
|
109
122
|
|
110
123
|
*Note:* due to the way Rails tries to serve these files and the seemingly
|
111
124
|
conflicting way that Socket.io has hardcoded some paths into the code, you may
|
@@ -120,8 +133,9 @@ via "git clone git://github.com/LearnBoost/Socket.IO-node.git socket.io-node --r
|
|
120
133
|
(from the socket.io install instructions http://github.com/LearnBoost/Socket.IO-node)
|
121
134
|
|
122
135
|
|
123
|
-
|
124
|
-
|
136
|
+
== Troubleshooting
|
137
|
+
|
138
|
+
=== The part of the show where I try to save you some trouble
|
125
139
|
|
126
140
|
There are a few things I found along the way that were necessary to make this
|
127
141
|
integration run super smoothly:
|
@@ -133,8 +147,7 @@ You may need to create a link in your public/ dir to socket.io to the install
|
|
133
147
|
in your public/javascripts/ dir.
|
134
148
|
|
135
149
|
|
136
|
-
|
137
|
-
h2. Note on Patches/Pull Requests
|
150
|
+
== Note on Patches/Pull Requests
|
138
151
|
|
139
152
|
* Fork the project.
|
140
153
|
* Make your feature addition or bug fix.
|
@@ -144,6 +157,6 @@ h2. Note on Patches/Pull Requests
|
|
144
157
|
(if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
|
145
158
|
* Send me a pull request. Bonus points for topic branches.
|
146
159
|
|
147
|
-
|
160
|
+
== Copyright
|
148
161
|
|
149
162
|
Copyright (c) 2010 Luke van der Hoeven. See LICENSE for details.
|
data/bin/kthxbye-monitor
ADDED
@@ -0,0 +1,25 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
$LOAD_PATH.unshift File.expand_path(File.dirname(__FILE__) + '/../lib')
|
3
|
+
|
4
|
+
begin
|
5
|
+
require 'vegas'
|
6
|
+
rescue
|
7
|
+
require 'rubygems'
|
8
|
+
require 'vegas'
|
9
|
+
end
|
10
|
+
|
11
|
+
require 'kthxbye/web_interface'
|
12
|
+
require 'kthxbye/config'
|
13
|
+
|
14
|
+
Vegas::Runner.new(Kthxbye::WebInterface, 'kthxbye-monitor') do |runner, args, app|
|
15
|
+
Kthxbye::Config.setup
|
16
|
+
|
17
|
+
args.on('-S SERVER', '--server SERVER', "set the redis server") {|server|
|
18
|
+
Kthxbye::Config.options[:redis_server] = server
|
19
|
+
}
|
20
|
+
args.on('-P PORT', '--port PORT', "set the redis port") {|port|
|
21
|
+
Kthxbye::Config.options[:redis_port] = port
|
22
|
+
}
|
23
|
+
|
24
|
+
end
|
25
|
+
|
data/config.ru
CHANGED
@@ -1,10 +1,11 @@
|
|
1
1
|
#!/usr/bin/env ruby
|
2
2
|
|
3
3
|
$LOAD_PATH.unshift ::File.expand_path(::File.dirname(__FILE__) + '/lib')
|
4
|
-
require 'kthxbye'
|
5
4
|
require 'kthxbye/web_interface'
|
6
5
|
|
7
|
-
|
8
|
-
|
6
|
+
if ENV['KTHXBYECONFIG'] && ::File.exists?(::File.expand_path(ENV['KTHXBYECONFIG']))
|
7
|
+
load ::File.expand_path(ENV['KTHXBYECONFIG'])
|
8
|
+
end
|
9
9
|
|
10
|
-
|
10
|
+
use Rack::ShowExceptions
|
11
|
+
Kthxbye::WebInterface.new
|
data/lib/kthxbye/config.rb
CHANGED
@@ -20,7 +20,7 @@ module Kthxbye
|
|
20
20
|
#
|
21
21
|
# NEEDS TO BE REWORKED FOR RAILS APPS
|
22
22
|
def self.setup( args=nil )
|
23
|
-
@options
|
23
|
+
@options ||= DEFAULT.dup
|
24
24
|
@options.merge!( YAML.load('./config/kthxbye.yaml') ) if File.exist?( './config/kthxbye.yaml' )
|
25
25
|
@options.merge!( args ) if args
|
26
26
|
end
|
data/lib/kthxbye/version.rb
CHANGED
data/lib/kthxbye/worker.rb
CHANGED
data/test/test_kthxbye.rb
CHANGED
@@ -4,7 +4,7 @@ class TestKthxbye < Test::Unit::TestCase
|
|
4
4
|
context "See Kthxbye Configuration" do
|
5
5
|
|
6
6
|
should "configure an app with given params" do
|
7
|
-
|
7
|
+
Kthxbye::Config.setup(:redis_server => "localhost", :redis_port => 8080, :verbose => true)
|
8
8
|
|
9
9
|
assert_equal 'localhost', Kthxbye::Config.options[:redis_server]
|
10
10
|
assert_equal 8080, Kthxbye::Config.options[:redis_port]
|
@@ -23,7 +23,7 @@ class TestKthxbye < Test::Unit::TestCase
|
|
23
23
|
|
24
24
|
context "See Kthxbye" do
|
25
25
|
setup do
|
26
|
-
Kthxbye::Config.setup
|
26
|
+
Kthxbye::Config.setup(:verbose => false)
|
27
27
|
Kthxbye.redis.flushall
|
28
28
|
end
|
29
29
|
|
metadata
CHANGED
@@ -1,13 +1,12 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: kthxbye
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
hash: 17
|
5
4
|
prerelease: false
|
6
5
|
segments:
|
7
6
|
- 1
|
8
|
-
-
|
9
|
-
-
|
10
|
-
version: 1.
|
7
|
+
- 2
|
8
|
+
- 0
|
9
|
+
version: 1.2.0
|
11
10
|
platform: ruby
|
12
11
|
authors:
|
13
12
|
- Luke van der Hoeven
|
@@ -15,7 +14,7 @@ autorequire:
|
|
15
14
|
bindir: bin
|
16
15
|
cert_chain: []
|
17
16
|
|
18
|
-
date: 2010-10-
|
17
|
+
date: 2010-10-08 00:00:00 -04:00
|
19
18
|
default_executable:
|
20
19
|
dependencies: []
|
21
20
|
|
@@ -27,8 +26,9 @@ extensions: []
|
|
27
26
|
|
28
27
|
extra_rdoc_files:
|
29
28
|
- LICENSE
|
30
|
-
- README.
|
29
|
+
- README.rdoc
|
31
30
|
files:
|
31
|
+
- bin/kthxbye-monitor
|
32
32
|
- lib/kthxbye/version.rb
|
33
33
|
- lib/kthxbye/job.rb
|
34
34
|
- lib/kthxbye/exceptions.rb
|
@@ -61,7 +61,7 @@ files:
|
|
61
61
|
- lib/generators/kthxbye/kthxbye_generator.rb
|
62
62
|
- lib/kthxbye.rb
|
63
63
|
- LICENSE
|
64
|
-
- README.
|
64
|
+
- README.rdoc
|
65
65
|
- DESIGN.textile
|
66
66
|
- config.ru
|
67
67
|
- Gemfile
|
@@ -85,7 +85,6 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
85
85
|
requirements:
|
86
86
|
- - ">="
|
87
87
|
- !ruby/object:Gem::Version
|
88
|
-
hash: 3
|
89
88
|
segments:
|
90
89
|
- 0
|
91
90
|
version: "0"
|
@@ -94,7 +93,6 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
94
93
|
requirements:
|
95
94
|
- - ">="
|
96
95
|
- !ruby/object:Gem::Version
|
97
|
-
hash: 23
|
98
96
|
segments:
|
99
97
|
- 1
|
100
98
|
- 3
|