nfo-resque-mongo 1.15.1 → 1.17.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/HISTORY.md +32 -0
- data/README.markdown +135 -62
- data/Rakefile +6 -2
- data/lib/nfo-resque-mongo.rb +2 -0
- data/lib/resque.rb +20 -6
- data/lib/resque/failure/hoptoad.rb +22 -113
- data/lib/resque/failure/mongo.rb +6 -1
- data/lib/resque/failure/multiple.rb +5 -1
- data/lib/resque/failure/redis.rb +51 -0
- data/lib/resque/helpers.rb +13 -17
- data/lib/resque/plugin.rb +5 -0
- data/lib/resque/server.rb +18 -8
- data/lib/resque/server/public/favicon.ico +0 -0
- data/lib/resque/server/public/style.css +2 -2
- data/lib/resque/server/views/failed.erb +3 -2
- data/lib/resque/server/views/queues.erb +1 -1
- data/lib/resque/server/views/stats.erb +15 -1
- data/lib/resque/server/views/working.erb +10 -6
- data/lib/resque/version.rb +1 -1
- data/lib/resque/worker.rb +9 -4
- data/{tasks → lib/tasks}/redis.rake +0 -0
- data/{tasks → lib/tasks}/resque.rake +0 -0
- data/test/hoptoad_test.rb +25 -0
- data/test/job_hooks_test.rb +40 -0
- data/test/queue_stats_test.rb +1 -1
- data/test/resque_test.rb +1 -1
- data/test/test_helper.rb +5 -2
- metadata +66 -61
data/HISTORY.md
CHANGED
@@ -1,3 +1,35 @@
|
|
1
|
+
## 1.17.1 (2011-05-27)
|
2
|
+
|
3
|
+
* Reverted `exit` change. Back to `exit!`.
|
4
|
+
|
5
|
+
## 1.17.0 (2011-05-26)
|
6
|
+
|
7
|
+
* Workers exit with `exit` instead of `exit!`. This means you
|
8
|
+
can now use `at_exit` hooks inside workers.
|
9
|
+
* More monit typo fixes.
|
10
|
+
* Fixed bug in Hoptoad backend.
|
11
|
+
* Web UI: Wrap preformatted arguments.
|
12
|
+
|
13
|
+
## 1.16.1 (2011-05-17)
|
14
|
+
|
15
|
+
* Bugfix: Resque::Failure::Hoptoad.configure works again
|
16
|
+
* Bugfix: Loading rake tasks
|
17
|
+
|
18
|
+
## 1.16.0 (2011-05-16)
|
19
|
+
|
20
|
+
* Optional Hoptoad backend extracted into hoptoad_notifier. Install the gem to use it.
|
21
|
+
* Added `Worker#paused?` method
|
22
|
+
* Bugfix: Properly reseed random number generator after forking.
|
23
|
+
* Bugfix: Resque.redis=(<a Redis::Namespace>)
|
24
|
+
* Bugfix: Monit example stdout/stderr redirection
|
25
|
+
* Bugfix: Removing single failure now works with multiple failure backends
|
26
|
+
* Web: 'Remove Queue' now requires confirmation
|
27
|
+
* Web: Favicon!
|
28
|
+
* Web Bugfix: Dates display in Safari
|
29
|
+
* Web Bugfix: Dates display timezone
|
30
|
+
* Web Bugfix: Race condition querying working workers
|
31
|
+
* Web Bugfix: Fix polling /workers/all in resque-web
|
32
|
+
|
1
33
|
## 1.15.0 (2011-03-18)
|
2
34
|
|
3
35
|
* Fallback to Redis.connect. Makes ENV variables and whatnot work.
|
data/README.markdown
CHANGED
@@ -6,8 +6,7 @@ library functionality has been ported (all tests pass) and the
|
|
6
6
|
monitoring sinatra app works except the "stats" panel, although there are
|
7
7
|
a lot of details that need to been taken care of.
|
8
8
|
|
9
|
-
Resque-mongo uses a fairly
|
10
|
-
findAndModify that is only supported by 0.20+ version ofthe ruby mongo driver.
|
9
|
+
Resque-mongo uses a fairly nice feature of mongo, [findAndModify()][fnr] that is only supported by 0.20+ version of the Ruby mongo driver.
|
11
10
|
|
12
11
|
Also, check your mongo version: 1.3.0 or higher
|
13
12
|
|
@@ -17,6 +16,7 @@ Also, check your mongo version: 1.3.0 or higher
|
|
17
16
|
# Rails integration, config/environment.rb
|
18
17
|
config.gem 'nfo-resque-mongo', :lib => 'resque'
|
19
18
|
|
19
|
+
To make the regular merge with the [official Resque repository](https://github.com/defunkt/resque/) easier, some redis-related files were kept.
|
20
20
|
|
21
21
|
Resque
|
22
22
|
======
|
@@ -67,14 +67,17 @@ later, pull those jobs off the queue and process them.
|
|
67
67
|
Resque jobs are Ruby classes (or modules) which respond to the
|
68
68
|
`perform` method. Here's an example:
|
69
69
|
|
70
|
-
class Archive
|
71
|
-
@queue = :file_serve
|
72
70
|
|
73
|
-
|
74
|
-
|
75
|
-
|
76
|
-
|
77
|
-
|
71
|
+
``` ruby
|
72
|
+
class Archive
|
73
|
+
@queue = :file_serve
|
74
|
+
|
75
|
+
def self.perform(repo_id, branch = 'master')
|
76
|
+
repo = Repository.find(repo_id)
|
77
|
+
repo.create_archive(branch)
|
78
|
+
end
|
79
|
+
end
|
80
|
+
```
|
78
81
|
|
79
82
|
The `@queue` class instance variable determines which queue `Archive`
|
80
83
|
jobs will be placed in. Queues are arbitrary and created on the fly -
|
@@ -83,11 +86,13 @@ you can name them whatever you want and have as many as you want.
|
|
83
86
|
To place an `Archive` job on the `file_serve` queue, we might add this
|
84
87
|
to our application's pre-existing `Repository` class:
|
85
88
|
|
86
|
-
|
87
|
-
|
88
|
-
|
89
|
-
|
90
|
-
|
89
|
+
``` ruby
|
90
|
+
class Repository
|
91
|
+
def async_create_archive(branch)
|
92
|
+
Resque.enqueue(Archive, self.id, branch)
|
93
|
+
end
|
94
|
+
end
|
95
|
+
```
|
91
96
|
|
92
97
|
Now when we call `repo.async_create_archive('masterbrew')` in our
|
93
98
|
application, a job will be created and placed on the `file_serve`
|
@@ -95,12 +100,16 @@ queue.
|
|
95
100
|
|
96
101
|
Later, a worker will run something like this code to process the job:
|
97
102
|
|
98
|
-
|
99
|
-
|
103
|
+
``` ruby
|
104
|
+
klass, args = Resque.reserve(:file_serve)
|
105
|
+
klass.perform(*args) if klass.respond_to? :perform
|
106
|
+
```
|
100
107
|
|
101
108
|
Which translates to:
|
102
109
|
|
103
|
-
|
110
|
+
``` ruby
|
111
|
+
Archive.perform(44, 'masterbrew')
|
112
|
+
```
|
104
113
|
|
105
114
|
Let's start a worker to run `file_serve` jobs:
|
106
115
|
|
@@ -149,25 +158,33 @@ needs to be crunched later into a queue.
|
|
149
158
|
Jobs are persisted to queues as JSON objects. Let's take our `Archive`
|
150
159
|
example from above. We'll run the following code to create a job:
|
151
160
|
|
152
|
-
|
153
|
-
|
161
|
+
``` ruby
|
162
|
+
repo = Repository.find(44)
|
163
|
+
repo.async_create_archive('masterbrew')
|
164
|
+
```
|
154
165
|
|
155
166
|
The following JSON will be stored in the `file_serve` queue:
|
156
167
|
|
157
|
-
|
158
|
-
|
159
|
-
|
160
|
-
|
168
|
+
``` javascript
|
169
|
+
{
|
170
|
+
'class': 'Archive',
|
171
|
+
'args': [ 44, 'masterbrew' ]
|
172
|
+
}
|
173
|
+
```
|
161
174
|
|
162
175
|
Because of this your jobs must only accept arguments that can be JSON encoded.
|
163
176
|
|
164
177
|
So instead of doing this:
|
165
178
|
|
166
|
-
|
179
|
+
``` ruby
|
180
|
+
Resque.enqueue(Archive, self, branch)
|
181
|
+
```
|
167
182
|
|
168
183
|
do this:
|
169
184
|
|
170
|
-
|
185
|
+
``` ruby
|
186
|
+
Resque.enqueue(Archive, self.id, branch)
|
187
|
+
```
|
171
188
|
|
172
189
|
This is why our above example (and all the examples in `examples/`)
|
173
190
|
uses object IDs instead of passing around the objects.
|
@@ -207,15 +224,17 @@ Workers
|
|
207
224
|
|
208
225
|
Resque workers are rake tasks that run forever. They basically do this:
|
209
226
|
|
210
|
-
|
211
|
-
|
212
|
-
|
213
|
-
|
214
|
-
|
215
|
-
|
216
|
-
|
217
|
-
|
218
|
-
|
227
|
+
``` ruby
|
228
|
+
start
|
229
|
+
loop do
|
230
|
+
if job = reserve
|
231
|
+
job.process
|
232
|
+
else
|
233
|
+
sleep 5
|
234
|
+
end
|
235
|
+
end
|
236
|
+
shutdown
|
237
|
+
```
|
219
238
|
|
220
239
|
Starting a worker is simple. Here's our example from earlier:
|
221
240
|
|
@@ -234,13 +253,17 @@ This will load the environment before starting a worker. Alternately
|
|
234
253
|
we can define a `resque:setup` task with a dependency on the
|
235
254
|
`environment` rake task:
|
236
255
|
|
237
|
-
|
256
|
+
``` ruby
|
257
|
+
task "resque:setup" => :environment
|
258
|
+
```
|
238
259
|
|
239
260
|
GitHub's setup task looks like this:
|
240
261
|
|
241
|
-
|
242
|
-
|
243
|
-
|
262
|
+
``` ruby
|
263
|
+
task "resque:setup" => :environment do
|
264
|
+
Grit::Git.git_timeout = 10.minutes
|
265
|
+
end
|
266
|
+
```
|
244
267
|
|
245
268
|
We don't want the `git_timeout` as high as 10 minutes in our web app,
|
246
269
|
but in the Resque workers it's fine.
|
@@ -421,7 +444,7 @@ The Front End
|
|
421
444
|
Resque comes with a Sinatra-based front end for seeing what's up with
|
422
445
|
your queue.
|
423
446
|
|
424
|
-

|
425
448
|
|
426
449
|
### Standalone
|
427
450
|
|
@@ -455,11 +478,13 @@ Nginx: <http://www.modrails.com/documentation/Users%20guide%20Nginx.html#deployi
|
|
455
478
|
If you want to load Resque on a subpath, possibly alongside other
|
456
479
|
apps, it's easy to do with Rack's `URLMap`:
|
457
480
|
|
458
|
-
|
481
|
+
``` ruby
|
482
|
+
require 'resque/server'
|
459
483
|
|
460
|
-
|
461
|
-
|
462
|
-
|
484
|
+
run Rack::URLMap.new \
|
485
|
+
"/" => Your::App.new,
|
486
|
+
"/resque" => Resque::Server.new
|
487
|
+
```
|
463
488
|
|
464
489
|
Check `examples/demo/config.ru` for a functional example (including
|
465
490
|
HTTP basic auth).
|
@@ -555,13 +580,8 @@ together. But, it's not that hard.
|
|
555
580
|
Resque Dependencies
|
556
581
|
-------------------
|
557
582
|
|
558
|
-
gem install
|
559
|
-
|
560
|
-
If you cannot install `yajl-ruby` (JRuby?), you can install the `json`
|
561
|
-
gem and Resque will use it instead.
|
562
|
-
|
563
|
-
When problems arise, make sure you have the newest versions of the
|
564
|
-
`redis` and `redis-namespace` gems.
|
583
|
+
$ gem install bundler
|
584
|
+
$ bundle install
|
565
585
|
|
566
586
|
|
567
587
|
Installing Resque
|
@@ -575,7 +595,9 @@ First install the gem.
|
|
575
595
|
|
576
596
|
Next include it in your application.
|
577
597
|
|
578
|
-
|
598
|
+
``` ruby
|
599
|
+
require 'resque'
|
600
|
+
```
|
579
601
|
|
580
602
|
Now start your application:
|
581
603
|
|
@@ -586,8 +608,10 @@ That's it! You can now create Resque jobs from within your app.
|
|
586
608
|
To start a worker, create a Rakefile in your app's root (or add this
|
587
609
|
to an existing Rakefile):
|
588
610
|
|
589
|
-
|
590
|
-
|
611
|
+
``` ruby
|
612
|
+
require 'your/app'
|
613
|
+
require 'resque/tasks'
|
614
|
+
```
|
591
615
|
|
592
616
|
Now:
|
593
617
|
|
@@ -597,7 +621,7 @@ Alternately you can define a `resque:setup` hook in your Rakefile if you
|
|
597
621
|
don't want to load your app every time rake runs.
|
598
622
|
|
599
623
|
|
600
|
-
### In a Rails app, as a gem
|
624
|
+
### In a Rails 2.x app, as a gem
|
601
625
|
|
602
626
|
First install the gem.
|
603
627
|
|
@@ -616,7 +640,9 @@ That's it! You can now create Resque jobs from within your app.
|
|
616
640
|
|
617
641
|
To start a worker, add this to your Rakefile in `RAILS_ROOT`:
|
618
642
|
|
619
|
-
|
643
|
+
``` ruby
|
644
|
+
require 'resque/tasks'
|
645
|
+
```
|
620
646
|
|
621
647
|
Now:
|
622
648
|
|
@@ -626,7 +652,7 @@ Don't forget you can define a `resque:setup` hook in
|
|
626
652
|
`lib/tasks/whatever.rake` that loads the `environment` task every time.
|
627
653
|
|
628
654
|
|
629
|
-
### In a Rails app, as a plugin
|
655
|
+
### In a Rails 2.x app, as a plugin
|
630
656
|
|
631
657
|
$ ./script/plugin install git://github.com/defunkt/resque
|
632
658
|
|
@@ -641,6 +667,40 @@ Don't forget you can define a `resque:setup` hook in
|
|
641
667
|
`lib/tasks/whatever.rake` that loads the `environment` task every time.
|
642
668
|
|
643
669
|
|
670
|
+
### In a Rails 3 app, as a gem
|
671
|
+
|
672
|
+
First include it in your Gemfile.
|
673
|
+
|
674
|
+
$ cat Gemfile
|
675
|
+
...
|
676
|
+
gem 'resque'
|
677
|
+
...
|
678
|
+
|
679
|
+
Next install it with Bundler.
|
680
|
+
|
681
|
+
$ bundle install
|
682
|
+
|
683
|
+
Now start your application:
|
684
|
+
|
685
|
+
$ rails server
|
686
|
+
|
687
|
+
That's it! You can now create Resque jobs from within your app.
|
688
|
+
|
689
|
+
To start a worker, add this to a file in `lib/tasks` (ex:
|
690
|
+
`lib/tasks/resque.rake`):
|
691
|
+
|
692
|
+
``` ruby
|
693
|
+
require 'resque/tasks'
|
694
|
+
```
|
695
|
+
|
696
|
+
Now:
|
697
|
+
|
698
|
+
$ QUEUE=* rake environment resque:work
|
699
|
+
|
700
|
+
Don't forget you can define a `resque:setup` hook in
|
701
|
+
`lib/tasks/whatever.rake` that loads the `environment` task every time.
|
702
|
+
|
703
|
+
|
644
704
|
Configuration
|
645
705
|
-------------
|
646
706
|
|
@@ -669,11 +729,13 @@ Here's our `config/resque.yml`:
|
|
669
729
|
|
670
730
|
And our initializer:
|
671
731
|
|
672
|
-
|
673
|
-
|
732
|
+
``` ruby
|
733
|
+
rails_root = ENV['RAILS_ROOT'] || File.dirname(__FILE__) + '/../..'
|
734
|
+
rails_env = ENV['RAILS_ENV'] || 'development'
|
674
735
|
|
675
|
-
|
676
|
-
|
736
|
+
resque_config = YAML.load_file(rails_root + '/config/resque.yml')
|
737
|
+
Resque.redis = resque_config[rails_env]
|
738
|
+
```
|
677
739
|
|
678
740
|
Easy peasy! Why not just use `RAILS_ROOT` and `RAILS_ENV`? Because
|
679
741
|
this way we can tell our Sinatra app about the config file:
|
@@ -685,7 +747,9 @@ Now everyone is on the same page.
|
|
685
747
|
Also, you could disable jobs queueing by setting 'inline' attribute.
|
686
748
|
For example, if you want to run all jobs in the same process for cucumber, try:
|
687
749
|
|
688
|
-
|
750
|
+
``` ruby
|
751
|
+
Resque.inline = ENV['RAILS_ENV'] == "cucumber"
|
752
|
+
```
|
689
753
|
|
690
754
|
|
691
755
|
Plugins and Hooks
|
@@ -712,7 +776,9 @@ in your Redis server.
|
|
712
776
|
|
713
777
|
Simply use the `Resque.redis.namespace` accessor:
|
714
778
|
|
715
|
-
|
779
|
+
``` ruby
|
780
|
+
Resque.redis.namespace = "resque:GitHub"
|
781
|
+
```
|
716
782
|
|
717
783
|
We recommend sticking this in your initializer somewhere after Redis
|
718
784
|
is configured.
|
@@ -743,6 +809,13 @@ of charge. This is **not** used by GitHub in production, so please
|
|
743
809
|
send patches for any tweaks or improvements you can make to it.
|
744
810
|
|
745
811
|
|
812
|
+
Questions
|
813
|
+
---------
|
814
|
+
|
815
|
+
Please add them to the [FAQ](https://github.com/defunkt/resque/wiki/FAQ) or
|
816
|
+
ask on the Mailing List. The Mailing List is explained further below
|
817
|
+
|
818
|
+
|
746
819
|
Development
|
747
820
|
-----------
|
748
821
|
|
data/Rakefile
CHANGED
@@ -1,4 +1,8 @@
|
|
1
|
-
|
1
|
+
#
|
2
|
+
# Setup
|
3
|
+
#
|
4
|
+
|
5
|
+
load 'lib/tasks/redis.rake'
|
2
6
|
|
3
7
|
$LOAD_PATH.unshift 'lib'
|
4
8
|
require 'resque/tasks'
|
@@ -27,7 +31,7 @@ else
|
|
27
31
|
Rake::TestTask.new do |test|
|
28
32
|
test.libs << "test"
|
29
33
|
test.test_files = FileList['test/**/*_test.rb']
|
30
|
-
end
|
34
|
+
end
|
31
35
|
end
|
32
36
|
|
33
37
|
if command? :kicker
|
data/lib/resque.rb
CHANGED
@@ -1,11 +1,5 @@
|
|
1
1
|
require 'mongo'
|
2
2
|
|
3
|
-
begin
|
4
|
-
require 'yajl'
|
5
|
-
rescue LoadError
|
6
|
-
require 'json'
|
7
|
-
end
|
8
|
-
|
9
3
|
require 'resque/version'
|
10
4
|
|
11
5
|
require 'resque/errors'
|
@@ -173,6 +167,19 @@ module Resque
|
|
173
167
|
|
174
168
|
# Pushes a job onto a queue. Queue name should be a string and the
|
175
169
|
# item should be any JSON-able Ruby object.
|
170
|
+
#
|
171
|
+
# Resque works generally expect the `item` to be a hash with the following
|
172
|
+
# keys:
|
173
|
+
#
|
174
|
+
# class - The String name of the job to run.
|
175
|
+
# args - An Array of arguments to pass the job. Usually passed
|
176
|
+
# via `class.to_class.perform(*args)`.
|
177
|
+
#
|
178
|
+
# Example
|
179
|
+
#
|
180
|
+
# Resque.push('archive', :class => 'Archive', :args => [ 35, 'tar' ])
|
181
|
+
#
|
182
|
+
# Returns nothing
|
176
183
|
def push(queue, item)
|
177
184
|
watch_queue(queue)
|
178
185
|
mongo << { :queue => queue.to_s, :item => item , :date => Time.now }
|
@@ -259,6 +266,12 @@ module Resque
|
|
259
266
|
#
|
260
267
|
# This method is considered part of the `stable` API.
|
261
268
|
def enqueue(klass, *args)
|
269
|
+
# Perform before_enqueue hooks. Don't perform enqueue if any hook returns false
|
270
|
+
before_hooks = Plugin.before_enqueue_hooks(klass).collect do |hook|
|
271
|
+
klass.send(hook, *args)
|
272
|
+
end
|
273
|
+
return if before_hooks.any? { |result| result == false }
|
274
|
+
|
262
275
|
Job.create(queue_from_class(klass), klass, *args)
|
263
276
|
|
264
277
|
Plugin.after_enqueue_hooks(klass).each do |hook|
|
@@ -388,3 +401,4 @@ module Resque
|
|
388
401
|
end
|
389
402
|
|
390
403
|
end
|
404
|
+
|