resque-pool 0.0.5 → 0.0.7

data/README.md

@@ -2,37 +2,41 @@ Resque Pool
 ===========
 
 Resque pool is a simple library for managing a pool of resque workers.  Given a
-configuration hash or a config file (resque-pool.yml or
-config/resque-pool.yml), it will manage your workers for you, starting up the
-appropriate number of workers for each.
+a config file (`resque-pool.yml` or `config/resque-pool.yml`) or a config hash,
+it will manage your workers for you, starting up the appropriate number of
+workers for each.
 
 Benefits
 ---------
 
 * Less memory consumption - If you are using Ruby Enterprise Edition, or any
-  ruby with copy-on-write safe garbage collection, this will save you a lot of
-  memory when you managing many workers.
-* Simpler (less) config - If you are using monit or god or an init script to
-  start up your workers, you can simply start up one pool, and it will manage
-  your workers for you.
+  ruby with copy-on-write safe garbage collection, this could save you a lot of
+  memory when you are managing many workers.
+* Simpler (less) config - If you are using monit or an init script to start up
+  your workers, you can start up one pool, and it will manage your workers for
+  you.
 * Faster startup - if you are starting many workers at once, you would normally
   have them competing for CPU as they load their environments.  Resque-pool can
-  load the environment once, and almost immediately fork all of your workers.
+  load the environment once and almost instantaneously fork all of the workers.
 
 How to use
 -----------
 
-To configure resque-pool, you can either set `Resque::Pool.config` to a hash in
-your `resque:pool:setup` or you can set the same config in either
-`resque-pool.yml` or `config/resque-pool.yml`.  To use resque-pool, require its
-rake tasks in your rake file, and call the resque:pool task.
+To configure resque-pool, you can use either `resque-pool.yml` or
+`config/resque-pool.yml`.  To use resque-pool, require its rake tasks in your
+rake file, and call the resque:pool task.
 
-For example, to use resque-pool with rails, in `config/resque-pool.yml`:
+The YAML file supports both using root level defaults as well as environment
+specific overrides.  For example, to use resque-pool with rails, in
+`config/resque-pool.yml`:
 
     foo: 1
     bar: 2
     "foo,bar,baz": 4
 
+    production:
+      "foo,bar,baz": 10
+
 and in `lib/tasks/resque.rake`:
 
     require 'resque/pool/tasks'
@@ -45,9 +49,6 @@ and in `lib/tasks/resque.rake`:
 
     # preload the rails environment in the pool master
     task "resque:pool:setup" do
-      # it's better to use a config file, but you can also config here:
-      # Resque::Pool.config = {"foo" => 1, "bar" => 1}
-
       # close any sockets or files in pool master
       ActiveRecord::Base.connection.disconnect!
 
@@ -127,10 +128,21 @@ TODO
 
 * do appropriate logging (e.g. all to one logfile, each queue to its own
   logfile, or each worker to its own logfile).  Logfile location must be
-  configurable.
-* (optionally) daemonize, setting a PID file somewhere
+  configurable, but default to `log/resque-pool.log`.  Of course, since resque
+  "logs" by writing to $stdout, this is really no more than redirecting stdout
+  to the appropriate logfile.
+* (optionally) daemonize, setting a PID file somewhere.  configurable, of
+  course, but default to `tmp/pids/resque-pool.pid`.
 * recover gracefully from a malformed config file (on startup and HUP)
-* figure out a good way to test this (preferably via cucumber or rspec)
+* procline for malformed config file, graceful shutdown... and other states?
+* figure out a good automated way to test this (cucumber or rspec?)
 * clean up the code (I stole most of it from unicorn, and it's still a bit
-  bastardized)
-* web interface for adding and removing workers (etc)
+  bastardized); excessive use of vim foldmarkers are a code smell.
+* rdoc
+* incorporate resque-batchworker features? (v2.0)
+* web interface for adding and removing workers (etc) (v2.0)
+
+Contributors
+-------------
+
+* John Schult (config file can be split by environment)

data/lib/resque/pool.rb

@@ -50,13 +50,9 @@ module Resque
     # Config: class methods to start up the pool using the default config {{{
 
     @config_files = ["resque-pool.yml", "config/resque-pool.yml"]
-    class << self; attr_accessor :config, :config_files; end
+    class << self; attr_accessor :config_files; end
     def self.load_default_config
-      if @config
-        @config
-      else
-        @config_files.detect { |f| File.exist?(f) }
-      end
+      @config_files.detect { |f| File.exist?(f) }
     end
 
     def self.run
@@ -69,24 +65,35 @@ module Resque
     # }}}
     # Config: load config and config file {{{
 
-    def load_config_from_file
-      return unless @pool_config_file
-      log "**** loading config from #{@pool_config_file}"
-      @config = YAML.load_file(@pool_config_file)
-    end
-
     def init_config(config)
       unless config
         raise ArgumentError,
           "No configuration found. Please setup config/resque-pool.yml"
       end
       if config.kind_of? String
-        @pool_config_file = config.to_s
-        load_config_from_file
+        @config_file = config.to_s
       else
         @config = config.dup
       end
-      log "**** config: #{@config.inspect}"
+      load_config
+    end
+
+    def load_config
+      if @config_file
+        @config = YAML.load_file(@config_file)
+      end
+      if config_environment
+        config.merge!(@config[config_environment])
+      end
+      config.delete_if {|key, value| value.is_a? Hash }
+    end
+
+    def config_environment
+      if defined? RAILS_ENV
+        RAILS_ENV
+      else
+        ENV['RACK_ENV'] || ENV['RAILS_ENV'] || ENV['RESQUE_ENV']
+      end
     end
 
     # }}}
@@ -139,7 +146,7 @@ module Resque
         signal_all_workers(signal)
       when :HUP
         log "HUP: reload config file"
-        load_config_from_file
+        load_config
         maintain_worker_count
       when :WINCH
         log "WINCH: gracefully stopping all workers"

data/lib/resque/pool/pooled_worker.rb

@@ -17,7 +17,7 @@ class Resque::Pool
     end
 
     # this entire method (except for one line) is copied and pasted from
-    # resque-1.9.9.  If shutdown were used as a method (attr_reader) rather
+    # resque-1.9.10.  If shutdown were used as a method (attr_reader) rather
     # than an instance variable, I wouldn't need to reduplicate this. :-(
     #
     # hopefully I can get defunkt to accept my patch for this.
@@ -34,7 +34,7 @@ class Resque::Pool
 
         if not @paused and job = reserve
           log "got: #{job.inspect}"
-          run_hook :before_fork
+          run_hook :before_fork, job
           working_on job
 
           if @child = fork

metadata

@@ -1,22 +1,21 @@
 --- !ruby/object:Gem::Specification 
 name: resque-pool
 version: !ruby/object:Gem::Version 
-  hash: 21
+  hash: 17
   prerelease: false
   segments: 
   - 0
   - 0
-  - 5
-  version: 0.0.5
+  - 7
+  version: 0.0.7
 platform: ruby
 authors: 
 - nicholas a. evans
-- Unicorn hackers
 autorequire: 
 bindir: bin
 cert_chain: []
 
-date: 2010-07-29 00:00:00 -04:00
+date: 2010-08-16 00:00:00 -04:00
 default_executable: 
 dependencies: 
 - !ruby/object:Gem::Dependency 
@@ -41,12 +40,12 @@ dependencies:
     requirements: 
     - - "="
       - !ruby/object:Gem::Version 
-        hash: 33
+        hash: 39
         segments: 
         - 1
         - 9
-        - 9
-        version: 1.9.9
+        - 10
+        version: 1.9.10
   type: :runtime
   version_requirements: *id002
 description: "    quickly and easily fork a pool of resque workers,\n    saving memory (w/REE) and monitoring their uptime\n"