resque-pool 0.3.0.beta.2 → 0.3.0

data/Changelog.md

@@ -1,3 +1,27 @@
+## 0.3.0 (2012-05-22)
+
+This is mostly just a long overdue maintenance release.  Many pull requests were
+merged.  A few non-pull-request branches were merged too.  This version supports
+ruby 1.9.3, 1.8.7, and even ancient 1.8.6, and all are checked by
+[travis-ci](http://travis-ci.org/nevans/resque-pool).  It also explicitly
+supports resque ~> 1.20.  And (if you have `gem-man` installed), it now has man
+pages for bin and yml config.
+
+Many thanks to the contributers!
+
+ * [@agnellvj](https://github.com/agnellvj): ruby 1.9 compatibility
+ * [@geoffgarside](https://github.com/geoffgarside): man pages!
+ * [@imajes](https://github.com/imajes) - bugfix: Handle when a pid no longer
+   exists by the time you try and kill it.
+ * [@jeremy](https://github.com/jeremy) & [@jamis](https://github.com/jamis) -
+   tasks require `resque/pool` lazily
+ * [@jhsu](https://github.com/jhsu) - bugfix: undefined variable 'e' for errors
+ * [@gaffneyc](https://github.com/gaffneyc) - compatibility fix:
+   Resque::Pool::PooledWorker as a module rather than class
+ * [@kcrayon](https://github.com/kcrayon) - bugfix: fix worker shutdown
+ * [@alexkwolfe](https://github.com/alexkwolfe) - added `app_name` for logging
+   (and maybe more in the future?)
+
 ## 0.2.0 (2011-03-15)
 
 * new feature: sending `HUP` to pool manager will reload the logfiles and

data/README.md

@@ -1,6 +1,9 @@
 Resque Pool
 ===========
 
+[![Build Status](https://secure.travis-ci.org/nevans/resque-pool.png)](http://travis-ci.org/nevans/resque-pool)
+[![Dependency Status](https://gemnasium.com/nevans/resque-pool.png)](https://gemnasium.com/nevans/resque-pool)
+
 Resque pool is a simple library for managing a pool of
 [resque](http://github.com/defunkt/resque) workers.  Given a a config file, it
 manages your workers for you, starting up the appropriate number of workers for

data/examples/Gemfile

@@ -1 +1,2 @@
+source :rubygems
 gemspec :path => ".."

data/examples/Gemfile.lock

@@ -1,58 +1,70 @@
 PATH
   remote: /home/nick/src/active/resque-pool
   specs:
-    resque-pool (0.1.0.dev)
+    resque-pool (0.3.0.dev)
       rake
-      resque (~> 1.13)
+      resque (~> 1.20)
       trollop (~> 1.16)
 
 GEM
+  remote: http://rubygems.org/
   specs:
-    SystemTimer (1.2.2)
-    aruba (0.3.2)
-      childprocess (~> 0.1.6)
-      cucumber (~> 0.10.0)
-      rspec (~> 2.3.0)
+    SystemTimer (1.2.3)
+    aruba (0.4.11)
+      childprocess (>= 0.2.3)
+      cucumber (>= 1.1.1)
+      ffi (>= 1.0.11)
+      rspec (>= 2.7.0)
     builder (3.0.0)
-    childprocess (0.1.7)
-      ffi (~> 0.6.3)
-    cucumber (0.10.0)
+    childprocess (0.3.2)
+      ffi (~> 1.0.6)
+    cucumber (1.1.9)
       builder (>= 2.1.2)
-      diff-lcs (~> 1.1.2)
-      gherkin (~> 2.3.2)
-      json (~> 1.4.6)
-      term-ansicolor (~> 1.0.5)
-    diff-lcs (1.1.2)
-    ffi (0.6.3)
-      rake (>= 0.8.7)
-    gherkin (2.3.3)
-      json (~> 1.4.6)
-    json (1.4.6)
-    rack (1.2.1)
-    rake (0.8.7)
-    redis (2.1.1)
-    redis-namespace (0.10.0)
+      diff-lcs (>= 1.1.2)
+      gherkin (~> 2.9.0)
+      json (>= 1.4.6)
+      term-ansicolor (>= 1.0.6)
+    diff-lcs (1.1.3)
+    ffi (1.0.11)
+    gherkin (2.9.3)
+      json (>= 1.4.6)
+    hpricot (0.8.6)
+    json (1.6.6)
+    multi_json (1.3.2)
+    mustache (0.99.4)
+    rack (1.4.1)
+    rack-protection (1.2.0)
+      rack
+    rake (0.9.2.2)
+    rdiscount (1.6.8)
+    redis (2.2.2)
+    redis-namespace (1.0.3)
       redis (< 3.0.0)
-    resque (1.13.0)
-      json (~> 1.4.6)
-      redis-namespace (>= 0.10.0)
+    resque (1.20.0)
+      multi_json (~> 1.0)
+      redis-namespace (~> 1.0.2)
       sinatra (>= 0.9.2)
       vegas (~> 0.1.2)
-    rspec (2.3.0)
-      rspec-core (~> 2.3.0)
-      rspec-expectations (~> 2.3.0)
-      rspec-mocks (~> 2.3.0)
-    rspec-core (2.3.1)
-    rspec-expectations (2.3.0)
-      diff-lcs (~> 1.1.2)
-    rspec-mocks (2.3.0)
-    sinatra (1.1.3)
-      rack (~> 1.1)
-      tilt (>= 1.2.2, < 2.0)
-    term-ansicolor (1.0.5)
-    tilt (1.2.2)
+    ronn (0.7.3)
+      hpricot (>= 0.8.2)
+      mustache (>= 0.7.0)
+      rdiscount (>= 1.5.8)
+    rspec (2.9.0)
+      rspec-core (~> 2.9.0)
+      rspec-expectations (~> 2.9.0)
+      rspec-mocks (~> 2.9.0)
+    rspec-core (2.9.0)
+    rspec-expectations (2.9.1)
+      diff-lcs (~> 1.1.3)
+    rspec-mocks (2.9.0)
+    sinatra (1.3.2)
+      rack (~> 1.3, >= 1.3.6)
+      rack-protection (~> 1.2)
+      tilt (~> 1.3, >= 1.3.3)
+    term-ansicolor (1.0.7)
+    tilt (1.3.3)
     trollop (1.16.2)
-    vegas (0.1.8)
+    vegas (0.1.11)
       rack (>= 1.0.0)
 
 PLATFORMS
@@ -60,8 +72,9 @@ PLATFORMS
 
 DEPENDENCIES
   SystemTimer
-  aruba (~> 0.3.2)
+  aruba (~> 0.4.11)
   bundler (~> 1.0)
-  cucumber (~> 0.10.0)
+  cucumber (~> 1.1.9)
   resque-pool!
-  rspec (~> 2.3.0)
+  ronn
+  rspec (~> 2.9.0)

data/examples/chef_cookbook/templates/default/initd.erb

@@ -41,7 +41,7 @@ case "$1" in
     start-stop-daemon --start --pidfile ${pidfile} \
       --chuid ${run_as_user} --chdir ${app_dir} \
       --startas ${bundler} -- exec \
-      resque-pool -d -p ${pidfile} -E ${environment} -o ${stdout_log} -e ${stderr_log}
+      resque-pool -d -a ${app_name} -p ${pidfile} -E ${environment} -o ${stdout_log} -e ${stderr_log}
     ;;
   reload)
     start-stop-daemon --stop --pidfile ${pidfile} --signal HUP

data/features/step_definitions/resque-pool_steps.rb

@@ -9,7 +9,7 @@ end
 def grab_worker_pids(count, str)
   puts "TODO: check output_or_log for #{count} worker started messages"
   pid_regex = (1..count).map { '(\d+)' }.join ', '
-  full_regex = /resque-pool-manager\[\d+\]: Pool contains worker PIDs: \[#{pid_regex}\]/m
+  full_regex = /resque-pool-manager\[aruba\]\[\d+\]: Pool contains worker PIDs: \[#{pid_regex}\]/m
   str.should =~ full_regex
   @worker_pids = full_regex.match(str).captures.map {|pid| pid.to_i }
 end
@@ -38,6 +38,16 @@ def output_or_log(report_log)
   end
 end
 
+class NotFinishedStarting < StandardError; end
+def worker_processes_for(queues)
+  children_of(background_pid).select do |pid, cmd|
+    raise NotFinishedStarting if cmd =~ /Starting$/
+    cmd =~ /^resque-\d+.\d+.\d+: Waiting for #{queues}$/
+  end
+rescue NotFinishedStarting
+  retry
+end
+
 def children_of(ppid)
   ps = `ps -eo ppid,pid,cmd | grep '^ *#{ppid} '`
   ps.split(/\s*\n/).map do |line|
@@ -57,7 +67,7 @@ When /^I send the pool manager the "([^"]*)" signal$/ do |signal|
   when "QUIT"
     keep_trying do
       step "the #{output_logfiles} should contain the following lines (with interpolated $PID):", <<-EOF
-resque-pool-manager[$PID]: QUIT: graceful shutdown, waiting for children
+resque-pool-manager[aruba][$PID]: QUIT: graceful shutdown, waiting for children
       EOF
     end
   else
@@ -92,15 +102,15 @@ end
 Then /^the pool manager should (report|log) that it has started up$/ do |report_log|
   keep_trying do
     step "the #{output_or_logfiles_string(report_log)} should contain the following lines (with interpolated $PID):", <<-EOF
-resque-pool-manager[$PID]: Resque Pool running in development environment
-resque-pool-manager[$PID]: started manager
+resque-pool-manager[aruba][$PID]: Resque Pool running in test environment
+resque-pool-manager[aruba][$PID]: started manager
     EOF
   end
 end
 
 Then /^the pool manager should (report|log) that the pool is empty$/ do |report_log|
   step "the #{output_or_logfiles_string(report_log)} should contain the following lines (with interpolated $PID):", <<-EOF
-resque-pool-manager[$PID]: Pool is empty
+resque-pool-manager[aruba][$PID]: Pool is empty
   EOF
 end
 
@@ -121,9 +131,7 @@ Then "the pool manager should have no child processes" do
 end
 
 Then /^the pool manager should have (\d+) "([^"]*)" worker child processes$/ do |count, queues|
-  children_of(background_pid).select do |pid, cmd|
-    cmd =~ /^resque-\d+.\d+.\d+: Waiting for #{queues}$/
-  end.should have(Integer(count)).members
+  worker_processes_for(queues).should have(Integer(count)).members
 end
 
 Then "the pool manager should finish" do
@@ -133,7 +141,7 @@ end
 
 Then /^the pool manager should (report|log) that it is finished$/ do |report_log|
   step "the #{output_or_logfiles_string(report_log)} should contain the following lines (with interpolated $PID):", <<-EOF
-resque-pool-manager[$PID]: manager finished
+resque-pool-manager[aruba][$PID]: manager finished
   EOF
 end
 

data/features/support/aruba_daemon_support.rb

@@ -1,3 +1,4 @@
+require 'aruba/cucumber'
 require 'aruba/api'
 require 'aruba/process'
 

data/features/support/env.rb

@@ -1 +1 @@
-require 'aruba/cucumber'
+ENV["RAILS_ENV"] = "test"

data/lib/resque/pool/cli.rb

@@ -5,7 +5,6 @@ require 'fileutils'
 module Resque
   class Pool
     module CLI
-      extend Logging
       extend self
 
       def run
@@ -31,6 +30,7 @@ Usage:
 where [options] are:
           EOS
           opt :config, "Alternate path to config file", :type => String, :short => "-c"
+          opt :appname, "Alternate appname",         :type => String,    :short => "-a"
           opt :daemon, "Run as a background daemon", :default => false,  :short => "-d"
           opt :stdout, "Redirect stdout to logfile", :type => String,    :short => '-o'
           opt :stderr, "Redirect stderr to logfile", :type => String,    :short => '-e'
@@ -100,8 +100,9 @@ where [options] are:
       end
 
       def setup_environment(opts)
+        Resque::Pool.app_name = opts[:appname]    if opts[:appname]
         ENV["RACK_ENV"] = ENV["RAILS_ENV"] = ENV["RESQUE_ENV"] = opts[:environment] if opts[:environment]
-        log "Resque Pool running in #{ENV["RAILS_ENV"] || "development"} environment"
+        Resque::Pool.log "Resque Pool running in #{ENV["RAILS_ENV"] || "development"} environment"
         ENV["RESQUE_POOL_CONFIG"] = opts[:config] if opts[:config]
       end
 

data/lib/resque/pool/logging.rb

@@ -38,21 +38,28 @@ module Resque
       # Procline is always in the format of:
       #   resque-pool-master: STRING
       def procline(string)
-        $0 = "resque-pool-master: #{string}"
+        $0 = "resque-pool-master#{app}: #{string}"
       end
 
       # TODO: make this use an actual logger
       def log(message)
-        puts "resque-pool-manager[#{Process.pid}]: #{message}"
+        puts "resque-pool-manager#{app}[#{Process.pid}]: #{message}"
         #$stdout.fsync
       end
 
       # TODO: make this use an actual logger
       def log_worker(message)
-        puts "resque-pool-worker[#{Process.pid}]: #{message}"
+        puts "resque-pool-worker#{app}[#{Process.pid}]: #{message}"
         #$stdout.fsync
       end
 
+      # Include optional app name in procline
+      def app
+        app_name   = self.respond_to?(:app_name)       && self.app_name
+        app_name ||= self.class.respond_to?(:app_name) && self.class.app_name
+        app_name ? "[#{app_name}]" : ""
+      end
+
     end
   end
 end

data/lib/resque/pool/pooled_worker.rb

@@ -2,19 +2,15 @@ require 'resque/worker'
 
 class Resque::Pool
   module PooledWorker
-
-    def initialize(*args)
-      @pool_master_pid = Process.pid
-      super
-    end
-
-    def pool_master_has_gone_away?
-      @pool_master_pid && @pool_master_pid != Process.ppid
+    def shutdown_with_pool
+      shutdown_without_pool || Process.ppid == 1
     end
 
-    # override +shutdown?+ method
-    def shutdown?
-      super || pool_master_has_gone_away?
+    def self.included(base)
+      base.instance_eval do
+        alias_method :shutdown_without_pool, :shutdown?
+        alias_method :shutdown?, :shutdown_with_pool
+      end
     end
 
   end

data/lib/resque/pool/version.rb

@@ -1,5 +1,5 @@
 module Resque
   class Pool
-    VERSION = "0.3.0.beta.2"
+    VERSION = "0.3.0"
   end
 end

data/lib/resque/pool.rb

@@ -15,6 +15,7 @@ module Resque
     CHUNK_SIZE = (16 * 1024)
 
     include Logging
+    extend  Logging
     attr_reader :config
     attr_reader :workers
 
@@ -50,7 +51,12 @@ module Resque
     # Config: class methods to start up the pool using the default config {{{
 
     @config_files = ["resque-pool.yml", "config/resque-pool.yml"]
-    class << self; attr_accessor :config_files; end
+    class << self; attr_accessor :config_files, :app_name; end
+
+    def self.app_name
+      @app_name ||= File.basename(Dir.pwd)
+    end
+
     def self.choose_config_file
       if ENV["RESQUE_POOL_CONFIG"]
         ENV["RESQUE_POOL_CONFIG"]

data/man/resque-pool.1

@@ -1,7 +1,7 @@
 .\" generated with Ronn/v0.7.3
 .\" http://github.com/rtomayko/ronn/tree/0.7.3
 .
-.TH "RESQUE\-POOL" "1" "November 2011" "RESQUE-POOL 0.3.0.DEV" "RESQUE-POOL"
+.TH "RESQUE\-POOL" "1" "May 2012" "RESQUE-POOL 0.3.0" "RESQUE-POOL"
 .
 .SH "NAME"
 \fBresque\-pool\fR \- resque worker pool management
@@ -38,11 +38,24 @@ When resque\-pool(1) is daemonized the \fBstdout\fR and \fBstderr\fR output stre
 .IP "\(bu" 4
 \fB\-E, \-\-environment\fR \fIname\fR: Specifies the environment \fIname\fR to be set for \fBRAILS_ENV\fR, \fBRACK_ENV\fR and \fBRESQUE_ENV\fR which will be passed on to the pooled resque workers\.
 .
+.IP "\(bu" 4
+\fB\-a, \-\-appname\fR \fIname\fR: Specifies the app name to be used for logging and procline\. If not specified, this defaults to the current working directory\.
+.
 .IP "" 0
 .
 .SH "HISTORY"
 .
 .TP
+\fBv0\.3\.0\fR
+Support for ruby 1\.9, resque 1\.20\.
+.
+.br
+Added appname for logging\.
+.
+.br
+Minor bugfixes\.
+.
+.TP
 \fBv0\.2\.0\fR
 Support for reloading logs and workers with \fBHUP\fR signal
 .

data/man/resque-pool.1.ronn

@@ -45,8 +45,17 @@ directory.
     Specifies the environment <name> to be set for `RAILS_ENV`, `RACK_ENV`
     and `RESQUE_ENV` which will be passed on to the pooled resque workers.
 
+  * `-a, --appname` <name>:
+    Specifies the app name to be used for logging and procline.  If not
+    specified, this defaults to the current working directory.
+
 ## HISTORY
 
+  * `v0.3.0`:
+    Support for ruby 1.9, resque 1.20.<br>
+    Added appname for logging.<br>
+    Minor bugfixes.
+
   * `v0.2.0`:
     Support for reloading logs and workers with `HUP` signal<br>
     Cleans up PID file on startup<br>

data/man/resque-pool.yml.5

@@ -1,7 +1,7 @@
 .\" generated with Ronn/v0.7.3
 .\" http://github.com/rtomayko/ronn/tree/0.7.3
 .
-.TH "RESQUE\-POOL\.YML" "5" "November 2011" "RESQUE-POOL 0.3.0.DEV" "RESQUE-POOL"
+.TH "RESQUE\-POOL\.YML" "5" "May 2012" "RESQUE-POOL 0.3.0" "RESQUE-POOL"
 .
 .SH "NAME"
 \fBresque\-pool\.yml\fR \- resque\-pool pool configuration

data/resque-pool.gemspec

@@ -18,7 +18,7 @@ Gem::Specification.new do |s|
   s.add_dependency "resque",  "~> 1.20"
   s.add_dependency "trollop", "~> 1.16"
   s.add_dependency "rake"
-  s.add_development_dependency "rspec",    "~> 2.8.0"
+  s.add_development_dependency "rspec",    "~> 2.9.0"
   s.add_development_dependency "cucumber", "~> 1.1.9"
   s.add_development_dependency "aruba",    "~> 0.4.11"
   s.add_development_dependency "bundler", "~> 1.0"