scgi 0.6.0 → 0.7.0

data/README

@@ -38,6 +38,7 @@ Subversion access is available at svn://code.jeremyevans.net/ruby-scgi/.
   help keep your site alive under heavy load.
 * Simple to configure with your web server. Even if you use clustering you'll
   be able to manage your webserver and Rails application independently.
+* Supports supervisor mode for rock solid upgrades without losing connections
 * Reasonable defaults for almost everything based on user feedback.
 * Completely free code licensed under Rails's own license.
 * No external dependencies other than Ruby
@@ -60,13 +61,14 @@ have to request open ports.  Sorry, no UNIX Domain sockets in SCGI.
 
 == Comparison With WEBrick
 
-In theory WEBrick should be able to run just as fast as SRR.  They are both
-written in pure Ruby.  They both do similar processing (although WEBrick's are
-a little more complicated).  They both return about the same amount of data.
+In theory WEBrick should be able to run just as fast as ruby-scgi.  They are 
+both written in pure Ruby.  They both do similar processing (although WEBrick's
+are a little more complicated).  They both return about the same amount of
+data.
 
-In practice WEBrick in production mode runs much slower than SRR in production
-mode.  The (dis)advantage (depending on your point of view) is that you have to
-manage your webserver differently than you manage your application.
+In practice WEBrick in production mode runs much slower than ruby-scgi in
+production mode.  The (dis)advantage (depending on your point of view) is that
+you have to manage your webserver differently than you manage your application.
 
 == Comparison With CGI
 
@@ -76,10 +78,10 @@ framework is loaded.  This is very slow, but it's easy to install.
 An alternative is to use the cgi2scgi program distributed with the SCGI source
 available from http://www.mems-exchange.org/software/scgi/ along with the
 Apache modules.  This program basically is a small little C program that runs
-quickly as a CGI, but passes it's requests to your SRR backend.  It's not all
-that fast, but if you're stuck with cgi-bin only access then this might be just
-the way to go.  Since SCGI runs over TCP/IP you can even host your SRR on a
-totally different machine with this.
+quickly as a CGI, but passes it's requests to your ruby-scgi backend.  It's not
+all that fast, but if you're stuck with cgi-bin only access then this might be
+just the way to go.  Since SCGI runs over TCP/IP you can even host your 
+ruby-scgi on a totally different machine with this.
 
 == Running and Configuration
 
@@ -93,7 +95,7 @@ To stop the application:
   
 To restart the application:
 
-  scgi_ctrl -d /path/to/rails/app start # start actually restarts
+  scgi_ctrl -d /path/to/rails/app restart
   
 Note that restarting/stopping is controlled by a pid file (the location is
 configurable).  If the pidfile exists, it is read and the pids in it are
@@ -110,12 +112,14 @@ scgi_ctrl [option=value, ...] (start|stop)
   -d, --directory     Working directory [.]
   -e, --environment   Environment (for Rails) [production]
   -f, --fork          Number of listners on each port [1]
+  -k, --killtime      Number of seconds to wait when killing children [2]
   -l, --logfile       Location of log file [log/scgi.log]
   -m, --maxconns      Maximum number of concurrent users [2**30-1]
   -n, --number        Number of ports to bind to [1]
   -p, --port          Starting port to bind to [9999]
   -P, --pidfile       Location of pid file [log/scgi.pid]
   -r, --processor     Type of processor to use [Rails]
+  -s, --supervise     Whether to use a supervisor [No]
 
 Note that the -d (--directory) option changes the working directory of the
 process, so the -c, -l, and -P options are relative to that.
@@ -160,6 +164,12 @@ Here's a longer explanation of the options:
   it is not stable, switch to multiple port clustering.  It is possible to use
   both as once.
   
+  -k, --killtime      Number of seconds to wait when killing children [2]
+
+  This sets the time that ruby-scgi will wait when stopping or restarting
+  child processes.  The time can actually be twice as long as this, if the
+  child processes are not shutting down cleanly.
+
   -l, --logfile       Location of log file [log/scgi.log]
   
   This is the location of the log file, relative to the working directory.
@@ -201,14 +211,29 @@ Here's a longer explanation of the options:
   path (the RUBYLIB environment variable).  See RailsSCGIProcessor.rb for an
   example.
 
+  -s, --supervise     Whether to use a supervisor [No]
+
+  This starts a supervisor process in addition to the worker processes.  The
+  supervisor process can then add and remove children using the same SCGI
+  listening socket, which means that there will be no downtime when you have
+  to upgrade your application.  Using a supervisor restricts some of the
+  variables that will be updated when you use restart.  You can change the
+  environment, fork, killtime, logfile, maxconns, and processor variables.
+  Note that if you use a supervisor, you should specify those variables in
+  the config file and not on the command line, otherwise it won't be 
+  possible to update them.  This is the only option that does not take an 
+  argument on the command line.  To set it in the config file, see below.
+
 Each of the options can also be specified in the config file as a symbol.  An
 example config file would be:
 
 ---
 :port: 4000
 :fork: 3
+:supervise: true
 
-This sets up a single-port cluster on port 4000 with 3 listening processes.
+This sets up a supervised single-port cluster on port 4000 with 3 listening 
+processes.
 
 == Example configurations
 
@@ -301,9 +326,9 @@ main attack vectors you should be aware of when running this:
 * scgi_ctrl is fully configurable on the command line
 * Clustering and processing are now built into scgi_ctrl
 * DRb, Win32, and throttling are no longer supported
-* Soft reconfiguration is no longer supported (no SIGUSR1)
-* Restarting via SIGHUP is no longer supported (SIGHUP is the same as SIGINT)
-* The only commands available to scgi_ctrl are start and stop
+* Soft reconfiguration has changed (no SIGUSR1)
+* Restarting via SIGHUP is only supported in supervise mode
+* The only commands available to scgi_ctrl are start, stop, and restart
 
 == FAQ
 
@@ -323,9 +348,5 @@ A: I haven't tried.  If you have luck, let me know.
 Q: Is there an easy way to reload?  I don't want to take the whole thing down
 just to deploy new code.
 
-A: scgi_ctrl always uses SIGINT to stop processes, which allows existing
-clients to finish.  Restarting may have problems if a lot of clients are
-connected and the processes can't shutdown before the new listening socket is
-bound.  This hasn't been a problem for me yet, but it is a possible race
-condition.  In case it affects you, you may have to run scgi_ctrl start again
-if you get an error binding to a port.
+A: Use supervise mode, which ensures that no connections will be lost when
+updating your apps.

data/bin/scgi_ctrl

@@ -1,132 +1,236 @@
 #!/usr/local/bin/ruby
 require 'getoptlong'
 require 'yaml'
+require 'socket'
 SCGI_DEFAULT_CONFIG = {:pidfile=>'log/scgi.pid', :number=>1, :port=>9999,
   :processor=>'Rails', :fork=>1, :logfile=>'log/scgi.log', :maxconns=>2**30-1,
-  :bind=>'127.0.0.1', :command=>''}
+  :bind=>'127.0.0.1', :cliconfig=>{}, :killtime=>2, :config=>'config/scgi.yaml'}
 SCGI_CONFIG = {}
 
-def config_file_options(filename)
-  begin
-    config = YAML.load(File.read(filename))
-    config.is_a?(Hash) ? config : Hash.new
-  rescue
-    Hash.new
-  end
-end
+module SCGI
+  class Starter
+    def initialize
+      super
+      parse_options
+    end
+    
+    def config_file_options(filename)
+      begin
+        config = YAML.load(File.read(filename))
+        config.is_a?(Hash) ? config : Hash.new
+      rescue
+        Hash.new
+      end
+    end
 
-def get_processor(string)
-  raise NameError, "#{string} is not a valid processor!" unless string =~ /\A[A-Z][A-ZA-z]*\z/
-  processor = "#{string}SCGIProcessor"
-  require processor
-  eval(processor)
-end
+    def get_processor(string)
+      raise NameError, "#{string} is not a valid processor!" unless string =~ /\A[A-Z][A-ZA-z]*\z/
+      processor = "#{string}SCGIProcessor"
+      require processor
+      eval(processor)
+    end
+    
+    def kill_children_forcefully(pids)
+      kill_pids('TERM', *pids)
+      sleep(SCGI_CONFIG[:killtime])
+      kill_pids('KILL', *pids)
+    end
+    
+    def kill_children_gently(pids)
+      kill_pids('INT', *pids)
+      sleep(SCGI_CONFIG[:killtime])
+      sleep(SCGI_CONFIG[:killtime]) if kill_pids('TERM', *pids)
+      kill_pids('KILL', *pids)
+    end
 
-def parse_options
-  config = {:config => 'config/scgi.yaml'}
-  opts = GetoptLong.new(
-    [ '--bind', '-b', GetoptLong::REQUIRED_ARGUMENT ],
-    [ '--config', '-c', GetoptLong::REQUIRED_ARGUMENT ],
-    [ '--directory', '-d', GetoptLong::REQUIRED_ARGUMENT ],
-    [ '--environment', '-e', GetoptLong::REQUIRED_ARGUMENT ],
-    [ '--fork', '-f', GetoptLong::REQUIRED_ARGUMENT ],
-    [ '--logfile', '-l', GetoptLong::REQUIRED_ARGUMENT ],
-    [ '--maxconns', '-m', GetoptLong::REQUIRED_ARGUMENT ],
-    [ '--number', '-n', GetoptLong::REQUIRED_ARGUMENT ],
-    [ '--port', '-p', GetoptLong::REQUIRED_ARGUMENT ],
-    [ '--pidfile', '-P', GetoptLong::REQUIRED_ARGUMENT ],
-    [ '--processor', '-r', GetoptLong::REQUIRED_ARGUMENT ]
-  )
-  opts.each do |opt, arg|
-    case opt
-      when '--bind'
-        config[:bind] = arg
-      when '--config'
-        config[:config] = arg
-      when '--directory'
-        Dir.chdir(arg)
-      when '--environment'
-        config[:environment] = arg
-      when '--fork'
-        config[:fork] = arg.to_i
-      when '--logfile'
-        config[:logile] = arg
-      when '--maxconns'
-        config[:maxconns] = arg.to_i
-      when '--number'
-        config[:number] = arg.to_i
-      when '--port'
-        config[:port] = arg.to_i
-      when '--pidfile'
-        config[:pidfile] = arg
-      when '--processor'
-        config[:processor] = arg
+    def kill_pids(signal, *pids)
+      Process.kill(signal, *pids) rescue nil
     end
-  end
-  
-  # Configuration Precedence: Command Line > Config File > Default
-  SCGI_CONFIG.merge!(SCGI_DEFAULT_CONFIG)
-  SCGI_CONFIG.merge!(config_file_options(config[:config]))
-  SCGI_CONFIG.merge!(config)
-end
 
-def process(command)
-  case command
-    when 'start'
-      stop
-      start
-    when 'stop'
-      stop
-    else
-      puts usage
-  end
-end
+    def parse_options
+      cliconfig = {}
+      opts = GetoptLong.new(
+        [ '--bind', '-b', GetoptLong::REQUIRED_ARGUMENT ],
+        [ '--config', '-c', GetoptLong::REQUIRED_ARGUMENT ],
+        [ '--directory', '-d', GetoptLong::REQUIRED_ARGUMENT ],
+        [ '--environment', '-e', GetoptLong::REQUIRED_ARGUMENT ],
+        [ '--fork', '-f', GetoptLong::REQUIRED_ARGUMENT ],
+        [ '--killtime', '-k', GetoptLong::REQUIRED_ARGUMENT ],
+        [ '--logfile', '-l', GetoptLong::REQUIRED_ARGUMENT ],
+        [ '--maxconns', '-m', GetoptLong::REQUIRED_ARGUMENT ],
+        [ '--number', '-n', GetoptLong::REQUIRED_ARGUMENT ],
+        [ '--port', '-p', GetoptLong::REQUIRED_ARGUMENT ],
+        [ '--pidfile', '-P', GetoptLong::REQUIRED_ARGUMENT ],
+        [ '--processor', '-r', GetoptLong::REQUIRED_ARGUMENT ],
+        [ '--supervise', '-s', GetoptLong::NO_ARGUMENT]
+      )
+      opts.each do |opt, arg|
+        case opt
+          when '--bind'
+            cliconfig[:bind] = arg
+          when '--config'
+            cliconfig[:config] = arg
+          when '--directory'
+            Dir.chdir(arg)
+          when '--environment'
+            cliconfig[:environment] = arg
+          when '--fork'
+            cliconfig[:fork] = arg.to_i
+          when '--killtime'
+            cliconfig[:killtime] = arg.to_i
+          when '--logfile'
+            cliconfig[:logile] = arg
+          when '--maxconns'
+            cliconfig[:maxconns] = arg.to_i
+          when '--number'
+            cliconfig[:number] = arg.to_i
+          when '--port'
+            cliconfig[:port] = arg.to_i
+          when '--pidfile'
+            cliconfig[:pidfile] = arg
+          when '--processor'
+            cliconfig[:processor] = arg
+          when '--supervise'
+            cliconfig[:supervise] = true
+        end
+      end
+
+      # Configuration Precedence: Command Line > Config File > Default
+      SCGI_CONFIG.merge!(SCGI_DEFAULT_CONFIG)
+      SCGI_CONFIG.merge!(config_file_options(cliconfig[:config] || SCGI_DEFAULT_CONFIG[:config]))
+      SCGI_CONFIG.merge!(cliconfig)
+      SCGI_CONFIG[:cliconfig] = cliconfig
+    end
+
+    def process(command)
+      return process_supervisor(command) if SCGI_CONFIG[:supervise]
+      case command
+        when 'restart'
+          stop
+          start
+        when 'start'
+          start
+        when 'stop'
+          stop
+        else
+          puts usage
+      end
+    end
+
+    def process_supervisor(command)
+      case command
+        when 'restart'
+          supervisor_restart
+        when 'start'
+          supervisor_start
+        when 'stop'
+          supervisor_stop
+        else
+          puts usage
+      end
+    end
+
+    def reload_config
+      SCGI_CONFIG.merge!(config_file_options(SCGI_CONFIG[:config]))
+      SCGI_CONFIG.merge!(SCGI_CONFIG[:cliconfig])
+    end
 
-def start
-  processor = get_processor(SCGI_CONFIG[:processor]).new(SCGI_CONFIG)
-  pids = []
-  SCGI_CONFIG[:number].times do |i|
-    socket = TCPServer.new(SCGI_CONFIG[:bind], port = SCGI_CONFIG[:port]+i)
-    SCGI_CONFIG[:fork].times do
-      if pid = fork
-        pids << pid
-      else
+    def start_child(socket, port)
+      fork do
         $0 = "scgi-#{SCGI_CONFIG[:processor]} dir:#{Dir.pwd} port:#{port}"
-        return processor.listen(socket)
+        get_processor(SCGI_CONFIG[:processor]).new(SCGI_CONFIG).listen(socket)
+      end
+    end
+
+    def start
+      pids = []
+      SCGI_CONFIG[:number].times do |i|
+        socket = TCPServer.new(SCGI_CONFIG[:bind], port = SCGI_CONFIG[:port]+i)
+        SCGI_CONFIG[:fork].times do
+          pids << start_child(socket, port)
+        end
+      end
+      File.open(SCGI_CONFIG[:pidfile], 'wb'){|file| file.print("#{pids.join(' ')}")}
+    end
+
+    def stop
+      if File.file?(SCGI_CONFIG[:pidfile])
+        pids = nil
+        File.open(SCGI_CONFIG[:pidfile], 'rb'){|f| pids = f.read.split.collect{|x| x.to_i if x.to_i > 0}.compact}
+        if pids.length > 0
+          kill_children_gently(pids)
+          File.delete(SCGI_CONFIG[:pidfile])
+        end
       end
     end
-  end
-  File.open(SCGI_CONFIG[:pidfile], 'wb'){|file| file.print("#{pids.join(' ')}")}
-end
 
-def stop
-  if File.file?(SCGI_CONFIG[:pidfile])
-    pids = nil
-    File.open(SCGI_CONFIG[:pidfile], 'rb'){|f| pids = f.read.split.collect{|x| x.to_i if x.to_i > 0}.compact}
-    if pids.length > 0
-      Process.kill("INT", *pids) rescue return nil
+    def supervisor_exit
       File.delete(SCGI_CONFIG[:pidfile])
+      exit
+    end
+
+    def supervisor_loop
+      $0 = "scgi-#{SCGI_CONFIG[:processor]} dir:#{Dir.pwd} supervisor"
+      trap("HUP") { reload_config; supervisor_restart_children }
+      trap("TERM") { kill_children_forcefully(SCGI_CONFIG[:pids].values.flatten); supervisor_exit }
+      trap("INT") { kill_children_gently(SCGI_CONFIG[:pids].values.flatten); supervisor_exit }
+      loop{sleep(60)}
+    end
+
+    def supervisor_restart
+      kill_pids("HUP", File.read(SCGI_CONFIG[:pidfile]).to_i)
+    end
+
+    def supervisor_restart_children
+      new_pids = {}
+      SCGI_CONFIG[:pids].each do |port, pids|
+        SCGI_CONFIG[:fork].times do
+          (new_pids[port] ||= []) << start_child(SCGI_CONFIG[:sockets][port], port)
+        end
+        kill_children_gently(pids)
+        pids.each{|pid| Process.detach(pid)}
+      end
+      SCGI_CONFIG[:pids] = new_pids
+    end
+
+    def supervisor_start
+      SCGI_CONFIG[:pids] = {}
+      SCGI_CONFIG[:sockets] = {}
+      SCGI_CONFIG[:number].times do |i| 
+        port = SCGI_CONFIG[:port]+i
+        SCGI_CONFIG[:sockets][port] = TCPServer.new(SCGI_CONFIG[:bind], port)
+        SCGI_CONFIG[:fork].times do
+          (SCGI_CONFIG[:pids][port] ||= []) << start_child(SCGI_CONFIG[:sockets][port], port)
+        end
+      end
+      File.open(SCGI_CONFIG[:pidfile], 'wb'){|file| file.print(fork{supervisor_loop})}
+    end
+
+    def supervisor_stop
+      kill_pids("INT", File.read(SCGI_CONFIG[:pidfile]).to_i)
+    end
+
+    def usage
+      <<-END
+    scgi_ctrl [option value, ...] (restart|start|stop)
+     Options:
+      -b, --bind          IP address to bind to [127.0.0.1]
+      -c, --config        Location of config file [config/scgi.yaml]
+      -d, --directory     Working directory [.]
+      -e, --environment   Environment (for Rails) [production]
+      -f, --fork          Number of listners on each port [1]
+      -k, --killtime      Number of seconds to wait when killing children [2]
+      -l, --logfile       Location of log file [log/scgi.log]
+      -m, --maxconns      Maximum number of concurrent users [2**30-1]
+      -n, --number        Number of ports to bind to [1]
+      -p, --port          Starting port to bind to [9999]
+      -P, --pidfile       Location of pid file [log/scgi.pid]
+      -r, --processor     Type of processor to use [Rails]
+      -s, --supervise     Whether to use a supervisor [No]
+      END
     end
   end
 end
 
-def usage
-  <<-END
-scgi_ctrl [option=value, ...] (start|stop)
- Options:
-  -b, --bind          IP address to bind to [127.0.0.1]
-  -c, --config        Location of config file [config/scgi.yaml]
-  -d, --directory     Working directory [.]
-  -e, --environment   Environment (for Rails) [production]
-  -f, --fork          Number of listners on each port [1]
-  -l, --logfile       Location of log file [log/scgi.log]
-  -m, --maxconns      Maximum number of concurrent users [2**30-1]
-  -n, --number        Number of ports to bind to [1]
-  -p, --port          Starting port to bind to [9999]
-  -P, --pidfile       Location of pid file [log/scgi.pid]
-  -r, --processor     Type of processor to use [Rails]
-END
-end
+SCGI::Starter.new.process(ARGV[0])
 
-parse_options
-process(ARGV[0])

metadata

@@ -3,8 +3,8 @@ rubygems_version: 0.9.0
 specification_version: 1
 name: scgi
 version: !ruby/object:Gem::Version 
-  version: 0.6.0
-date: 2007-08-13 00:00:00 -07:00
+  version: 0.7.0
+date: 2007-08-15 00:00:00 -07:00
 summary: Simple support for using SCGI in ruby apps, such as Rails
 require_paths: 
 - lib
@@ -35,8 +35,9 @@ files:
 - lib/RailsSCGIProcessor.rb
 test_files: []
 
-rdoc_options: []
-
+rdoc_options: 
+- --inline-source
+- --line-numbers
 extra_rdoc_files: []
 
 executables: