unicorn 0.2.1 → 0.2.2

data/.document

@@ -5,7 +5,8 @@ CONTRIBUTORS
 LICENSE
 SIGNALS
 TODO
-bin
+bin/unicorn
+bin/unicorn_rails
 lib
 ext/**/*.c
 ext/**/*.rl

data/CHANGELOG

@@ -1,25 +1,4 @@
+v0.2.2 - small bug fixes, fix Rack multi-value headers (Set-Cookie:)
 v0.2.1 - Fix broken Manifest that cause unicorn_rails to not be bundled
-
 v0.2.0 - unicorn_rails launcher script.
-
 v0.1.0 - Unicorn - UNIX-only fork of Mongrel free of threading
-
-  -- old Mongrel changelog --
-
-  v2.0. (WIP) Rack support.
-
-  v1.1.4. Fix camping handler. Correct treatment of @throttle parameter.
-
-  v1.1.3. Fix security flaw of DirHandler; reported on mailing list.
-
-  v1.1.2. Fix worker termination bug; fix JRuby 1.0.3 load order issue; fix require issue on systems without Rubygems.
-
-  v1.1.1. Fix mongrel_rails restart bug; fix bug with Rack status codes.
-
-  v1.1. Pure Ruby URIClassifier. More modular architecture. JRuby support. Move C URIClassifier into mongrel_experimental project.
-
-  v1.0.4. Backport fixes for versioning inconsistency, mongrel_rails bug, and DirHandler bug.
-
-  v1.0.3. Fix user-switching bug; make people upgrade to the latest from the RC.
-
-  v1.0.2. Signed gem; many minor bugfixes and patches.

data/Manifest

@@ -20,10 +20,12 @@ ext/unicorn/http11/http11_parser.h
 ext/unicorn/http11/http11_parser.rl
 ext/unicorn/http11/http11_parser_common.rl
 lib/unicorn.rb
+lib/unicorn/app/exec_cgi.rb
 lib/unicorn/configurator.rb
 lib/unicorn/const.rb
 lib/unicorn/http_request.rb
 lib/unicorn/http_response.rb
+lib/unicorn/launcher.rb
 lib/unicorn/socket.rb
 lib/unicorn/util.rb
 setup.rb
@@ -37,6 +39,7 @@ test/test_helper.rb
 test/tools/trickletest.rb
 test/unit/test_configurator.rb
 test/unit/test_http_parser.rb
+test/unit/test_request.rb
 test/unit/test_response.rb
 test/unit/test_server.rb
 test/unit/test_upload.rb

data/README

@@ -49,7 +49,7 @@ least a friend who can build it for you.
 You may download the tarball from the Mongrel project page on Rubyforge
 and run setup.rb after unpacking it:
 
-  http://rubyforge.org/frs/?group_id=1306
+http://rubyforge.org/frs/?group_id=1306
 
 You may also install it via Rubygems on Rubyforge:
 
@@ -67,7 +67,7 @@ If you have web browser software for the World Wide Web
 (on the Information Superhighway), you may browse the code from
 your web browser and download the latest snapshot tarballs here:
 
-* http://git.bogomips.org/cgit/unicorn.git
+* http://git.bogomips.org/cgit/unicorn.git (this server runs Unicorn!)
 * http://repo.or.cz/w/unicorn.git (gitweb mirror)
 
 == Usage
@@ -103,12 +103,16 @@ functionality of the `unicorn' launcher.
 
 == Disclaimer
 
-There are no known production instances of unicorn deployed
-anywhere in the world.  The original author of unicorn only has
-one, internal, low-traffic Sinatra application deployed with it.
-Maybe you'll be the first guinea pig to test it in production.
-Of course there is NO WARRANTY whatsoever if anything goes wrong,
-but let us know and we'll try our best to fix it.
+There are only a few instances of Unicorn deployed anywhere in the
+world.  The only public site known to run Unicorn at this time is
+http://git.bogomips.org/cgit which runs Unicorn::App::ExecCgi to
+fork()+exec() cgit.
+
+Be one of the first brave guinea pigs to run it on your production site!
+Of course there is NO WARRANTY whatsoever if anything goes wrong, but
+let us know and we'll try our best to fix it.  Unicorn is still in the
+early stages and testing + feedback would be *greatly* appreciated;
+maybe you'll get Rainbows as a reward!
 
 == Known Issues
 

data/SIGNALS

@@ -22,6 +22,9 @@ processes are documented here as well.
    should be sent to the original process once the child is verified to
    be up and running.
 
+ * WINCH - gracefully stops workers but keep the master running.
+   This will only work for daemonized processes.
+
 === Worker Processes
 
 Sending signals directly to the worker processes should not normally be
@@ -34,3 +37,51 @@ automatically respawned.
 
  * USR1 - reopen all logs owned by the worker process
    See Unicorn::Util.reopen_logs for what is considered a log.
+
+=== Procedure to replace a running unicorn executable
+
+You may replace a running instance of unicorn with a new one without
+losing any incoming connections.  Doing so will reload all of your
+application code, Unicorn config, Ruby executable, and all libraries.
+The only things that will not change (due to OS limitations) are:
+
+1. The listener backlog size of already-bound sockets
+
+2. The path to the unicorn executable script.  If you want to change to
+   a different installation of Ruby, you can modify the shebang
+   line to point to your alternative interpreter.
+
+The procedure is exactly like that of nginx:
+
+1. Send USR2 to the master process
+
+2. Check your process manager or pid files to see if a new master spawned
+   successfully.  If you're using a pid file, the old process will have
+   ".oldbin" appended to its path.  You should have two master instances
+   of unicorn running now, both of which will have workers servicing
+   requests.  Your process tree should look something like this:
+
+   unicorn master (old)
+   \_ unicorn worker[0]
+   \_ unicorn worker[1]
+   \_ unicorn worker[2]
+   \_ unicorn worker[3]
+   \_ unicorn master
+      \_ unicorn worker[0]
+      \_ unicorn worker[1]
+      \_ unicorn worker[2]
+      \_ unicorn worker[3]
+
+4. You can now send WINCH to the old master process so only the new workers
+   serve requests.  If your unicorn process is bound to an interactive
+   terminal, you can skip this step.  Step 5 will be more difficult but
+   you can also skip it if your process is not daemonized.
+
+5. You should now ensure that everything is running correctly with the
+   new workers as the old workers die off.
+
+6a. If everything seems ok, then send QUIT to the old master.  You're done!
+
+6b. If something is broken, then send HUP to the old master to reload
+    the config and restart its workers.  Then send QUIT to the new master
+    process.

data/bin/unicorn

@@ -1,6 +1,5 @@
 #!/home/ew/bin/ruby
-$stdin.sync = $stdout.sync = $stderr.sync = true
-require 'unicorn' # require this first to populate Unicorn::DEFAULT_START_CTX
+require 'unicorn/launcher'
 require 'optparse'
 
 env = "development"
@@ -163,27 +162,5 @@ if $DEBUG
   })
 end
 
-# only daemonize if we're not inheriting file descriptors from our parent
-if daemonize
-
-  $stdin.reopen("/dev/null")
-  unless ENV['UNICORN_FD']
-    exit if fork
-    Process.setsid
-    exit if fork
-  end
-
-  # We don't do a lot of standard daemonization stuff:
-  #   * $stderr/$stderr can/will be redirected separately
-  #   * umask is whatever was set by the parent process at startup
-  #     and can be set in config.ru and config_file, so making it
-  #     0000 and potentially exposing sensitive log data can be bad
-  #     policy.
-  #   * Don't bother to chdir here since Unicorn is designed to
-  #     run inside APP_ROOT.  Unicorn will also re-chdir() to
-  #     the directory it was started in when being re-executed
-  #     to pickup code changes if the original deployment directory
-  #     is a symlink or otherwise got replaced.
-end
-
+Unicorn::Launcher.daemonize! if daemonize
 Unicorn.run(app, options)

data/bin/unicorn_rails

@@ -1,6 +1,5 @@
 #!/home/ew/bin/ruby
-$stdin.sync = $stdout.sync = $stderr.sync = true
-require 'unicorn' # require this first to populate Unicorn::DEFAULT_START_CTX
+require 'unicorn/launcher'
 require 'optparse'
 require 'fileutils'
 
@@ -118,49 +117,12 @@ rails_loader = lambda do ||
     abort "#$0 must be run inside RAILS_ROOT (#{::RAILS_ROOT})"
   end
 
-  if ENV['UNICORN_RAILS_USE_SYSTEM_RACK'].to_i == 0
-    rails_ver = Rails::VERSION::STRING
-
-    # maps Rails versions to the vendorized Rack version they bundle
-    version_map = {
-      '2.3.2' => '1.0',
-      # 3.0.0 => false, # assuming 3.0.0 doesn't need vendorized Rack anymore
-    }
-    rack_ver = version_map[rails_ver] or
-      warn "Possibly unsupported Rails version: v#{rails_ver}"
-
-    rack_path = nil
-    case rack_ver
-    when String, NilClass
-      version_map.values.find_all { |v| String === v }.sort.each do |v|
-        $LOAD_PATH.grep(%r{/actionpack-[\d\.]+/lib/?\z}).each do |path|
-          rack_path = File.join(path, "action_controller/vendor/rack-#{v}")
-          File.directory?(rack_path) and break
-          rack_path = nil
-        end
-        break if rack_path
-      end
-      rack_path or abort(
-        "Unable to find Rails-vendorized Rack library.\n" \
-        "Perhaps this script is no longer with your" \
-        "Rails version (#{rails_ver}).\n")
-      puts "vendorized Rack load path #{rack_path}"
-      $LOAD_PATH.unshift(rack_path)
-    when FalseClass
-      # using non-vendorized rack library (most likely via gems)
-    end
-  end # Vendorized Rack LOAD_PATH finder
-
-  # require Rack as late as possible in case $LOAD_PATH is modified
-  # in config.ru or command-line
-  require 'rack'
-
   # return the lambda
   config = ::ARGV[0] || (File.exist?('config.ru') ? 'config.ru' : nil)
   case config
   when nil
     lambda do ||
-      require "#{RAILS_ROOT}/config/environment"
+      require 'config/environment'
       ActionController::Dispatcher.new
     end
   when /\.ru$/
@@ -208,31 +170,13 @@ if $DEBUG
   })
 end
 
-# only daemonize if we're not inheriting file descriptors from our parent
-if daemonize
-  options[:pid] = rails_pid
-  $stdin.reopen("/dev/null")
-  unless ENV['UNICORN_FD']
-    exit if fork
-    Process.setsid
-    exit if fork
-  end
-
-  # We don't do a lot of standard daemonization stuff:
-  #   * $stderr/$stderr can/will be redirected separately
-  #   * umask is whatever was set by the parent process at startup
-  #     and can be set in config.ru and config_file, so making it
-  #     0000 and potentially exposing sensitive log data can be bad
-  #     policy.
-  #   * Don't bother to chdir here since Unicorn is designed to
-  #     run inside APP_ROOT.  Unicorn will also re-chdir() to
-  #     the directory it was started in when being re-executed
-  #     to pickup code changes if the original deployment directory
-  #     is a symlink or otherwise got replaced.
-end
-
 # ensure Rails standard tmp paths exist
 %w(cache pids sessions sockets).each do |dir|
   FileUtils.mkdir_p("tmp/#{dir}")
 end
+
+if daemonize
+  options[:pid] = rails_pid
+  Unicorn::Launcher.daemonize!
+end
 Unicorn.run(app, options)

data/ext/unicorn/http11/http11.c

@@ -28,13 +28,9 @@ static VALUE global_fragment;
 static VALUE global_query_string;
 static VALUE global_http_version;
 static VALUE global_content_length;
-static VALUE global_http_content_length;
 static VALUE global_request_path;
 static VALUE global_content_type;
-static VALUE global_http_content_type;
 static VALUE global_http_body;
-static VALUE global_gateway_interface;
-static VALUE global_gateway_interface_value;
 static VALUE global_server_name;
 static VALUE global_server_port;
 static VALUE global_server_protocol;
@@ -129,6 +125,7 @@ static int common_field_cmp(const void *a, const void *b)
 }
 #endif /* HAVE_QSORT_BSEARCH */
 
+/* this function is not performance-critical */
 static void init_common_fields(void)
 {
   int i;
@@ -137,8 +134,15 @@ static void init_common_fields(void)
   memcpy(tmp, HTTP_PREFIX, HTTP_PREFIX_LEN);
 
   for(i = 0; i < ARRAY_SIZE(common_http_fields); cf++, i++) {
-    memcpy(tmp + HTTP_PREFIX_LEN, cf->name, cf->len + 1);
-    cf->value = rb_obj_freeze(rb_str_new(tmp, HTTP_PREFIX_LEN + cf->len));
+    /* Rack doesn't like certain headers prefixed with "HTTP_" */
+    if (!strcmp("CONTENT_LENGTH", cf->name) ||
+        !strcmp("CONTENT_TYPE", cf->name)) {
+      cf->value = rb_str_new(cf->name, cf->len);
+    } else {
+      memcpy(tmp + HTTP_PREFIX_LEN, cf->name, cf->len + 1);
+      cf->value = rb_str_new(tmp, HTTP_PREFIX_LEN + cf->len);
+    }
+    cf->value = rb_obj_freeze(cf->value);
     rb_global_variable(&cf->value);
   }
 
@@ -275,21 +279,8 @@ static void header_done(void *data, const char *at, size_t length)
 {
   VALUE req = (VALUE)data;
   VALUE temp = Qnil;
-  VALUE ctype = Qnil;
-  VALUE clen = Qnil;
   char *colon = NULL;
 
-  clen = rb_hash_aref(req, global_http_content_length);
-  if(clen != Qnil) {
-    rb_hash_aset(req, global_content_length, clen);
-  }
-
-  ctype = rb_hash_aref(req, global_http_content_type);
-  if(ctype != Qnil) {
-    rb_hash_aset(req, global_content_type, ctype);
-  }
-
-  rb_hash_aset(req, global_gateway_interface, global_gateway_interface_value);
   if((temp = rb_hash_aref(req, global_http_host)) != Qnil) {
     colon = memchr(RSTRING_PTR(temp), ':', RSTRING_LEN(temp));
     if(colon != NULL) {
@@ -497,12 +488,8 @@ void Init_http11()
   DEF_GLOBAL(http_version, "HTTP_VERSION");
   DEF_GLOBAL(request_path, "REQUEST_PATH");
   DEF_GLOBAL(content_length, "CONTENT_LENGTH");
-  DEF_GLOBAL(http_content_length, "HTTP_CONTENT_LENGTH");
   DEF_GLOBAL(http_body, "HTTP_BODY");
   DEF_GLOBAL(content_type, "CONTENT_TYPE");
-  DEF_GLOBAL(http_content_type, "HTTP_CONTENT_TYPE");
-  DEF_GLOBAL(gateway_interface, "GATEWAY_INTERFACE");
-  DEF_GLOBAL(gateway_interface_value, "CGI/1.2");
   DEF_GLOBAL(server_name, "SERVER_NAME");
   DEF_GLOBAL(server_port, "SERVER_PORT");
   DEF_GLOBAL(server_protocol, "SERVER_PROTOCOL");

data/lib/unicorn.rb

@@ -23,7 +23,6 @@ module Unicorn
   # forked worker children.
   class HttpServer
     attr_reader :logger
-    include Process
     include ::Unicorn::SocketHelper
 
     DEFAULT_START_CTX = {
@@ -160,7 +159,8 @@ module Unicorn
       # are trapped.  See trap_deferred
       @rd_sig, @wr_sig = IO.pipe unless (@rd_sig && @wr_sig)
       @rd_sig.nonblock = @wr_sig.nonblock = true
-      ready = mode = nil
+      mode = nil
+      respawn = true
 
       QUEUE_SIGS.each { |sig| trap_deferred(sig) }
       trap('CHLD') { |sig_nr| awaken_master }
@@ -172,18 +172,30 @@ module Unicorn
           case (mode = @sig_queue.shift)
           when nil
             murder_lazy_workers
-            spawn_missing_workers
+            spawn_missing_workers if respawn
+            master_sleep
           when 'QUIT' # graceful shutdown
             break
           when 'TERM', 'INT' # immediate shutdown
             stop(false)
             break
           when 'USR1' # rotate logs
-            kill_each_worker('USR1')
+            logger.info "master rotating logs..."
             Unicorn::Util.reopen_logs
+            logger.info "master done rotating logs"
+            kill_each_worker('USR1')
           when 'USR2' # exec binary, stay alive in case something went wrong
             reexec
+          when 'WINCH'
+            if Process.ppid == 1 || Process.getpgrp != $$
+              respawn = false
+              logger.info "gracefully stopping all workers"
+              kill_each_worker('QUIT')
+            else
+              logger.info "SIGWINCH ignored because we're not daemonized"
+            end
           when 'HUP'
+            respawn = true
             if @config.config_file
               load_config!
               redo # immediate reaping since we may have QUIT workers
@@ -195,18 +207,6 @@ module Unicorn
           else
             logger.error "master process in unknown mode: #{mode}"
           end
-          reap_all_workers
-
-          ready = begin
-            IO.select([@rd_sig], nil, nil, 1) or next
-          rescue Errno::EINTR # next
-          end
-          ready[0] && ready[0][0] or next
-          begin
-            @rd_sig.sysread(1)
-          rescue Errno::EAGAIN, Errno::EINTR
-            # spurious wakeup? ignore it
-          end
         end
       rescue Errno::EINTR
         retry
@@ -239,7 +239,8 @@ module Unicorn
     private
 
     # list of signals we care about and trap in master.
-    QUEUE_SIGS = %w(QUIT INT TERM USR1 USR2 HUP).map { |x| x.freeze }.freeze
+    QUEUE_SIGS =
+      %w(WINCH QUIT INT TERM USR1 USR2 HUP).map { |x| x.freeze }.freeze
 
     # defer a signal for later processing in #join (master process)
     def trap_deferred(signal)
@@ -253,6 +254,17 @@ module Unicorn
       end
     end
 
+    # wait for a signal hander to wake us up and then consume the pipe
+    # Wake up every second anyways to run murder_lazy_workers
+    def master_sleep
+      begin
+        ready = IO.select([@rd_sig], nil, nil, 1)
+        ready && ready[0] && ready[0][0] or return
+        loop { @rd_sig.sysread(Const::CHUNK_SIZE) }
+      rescue Errno::EAGAIN, Errno::EINTR
+      end
+    end
+
     def awaken_master
       begin
         @wr_sig.syswrite('.') # wakeup master process from IO.select
@@ -266,17 +278,18 @@ module Unicorn
     def reap_all_workers
       begin
         loop do
-          pid = waitpid(-1, WNOHANG) or break
+          pid, status = Process.waitpid2(-1, Process::WNOHANG)
+          pid or break
           if @reexec_pid == pid
-            logger.error "reaped exec()-ed PID:#{pid} status=#{$?.exitstatus}"
+            logger.error "reaped #{status.inspect} exec()-ed"
             @reexec_pid = 0
             self.pid = @pid.chomp('.oldbin') if @pid
+            $0 = "unicorn master"
           else
             worker = @workers.delete(pid)
             worker.tempfile.close rescue nil
-            logger.info "reaped PID:#{pid} " \
-                        "worker=#{worker.nr rescue 'unknown'} " \
-                        "status=#{$?.exitstatus}"
+            logger.info "reaped #{status.inspect} " \
+                        "worker=#{worker.nr rescue 'unknown'}"
           end
         end
       rescue Errno::ECHILD
@@ -324,6 +337,7 @@ module Unicorn
         @before_exec.call(self) if @before_exec
         exec(*cmd)
       end
+      $0 = "unicorn master (old)"
     end
 
     # forcibly terminate all workers that haven't checked in in @timeout
@@ -428,7 +442,7 @@ module Unicorn
         @listeners.each { |sock| sock.close rescue nil } # break IO.select
       end
 
-      while alive && @master_pid == ppid
+      while alive && @master_pid == Process.ppid
         # we're a goner in @timeout seconds anyways if tempfile.chmod
         # breaks, so don't trap the exception.  Using fchmod() since
         # futimes() is not available in base Ruby and I very strongly
@@ -494,7 +508,7 @@ module Unicorn
     # is no longer running.
     def kill_worker(signal, pid)
       begin
-        kill(signal, pid)
+        Process.kill(signal, pid)
       rescue Errno::ESRCH
         worker = @workers.delete(pid) and worker.tempfile.close rescue nil
       end
@@ -516,7 +530,7 @@ module Unicorn
     def valid_pid?(path)
       if File.exist?(path) && (pid = File.read(path).to_i) > 1
         begin
-          kill(0, pid)
+          Process.kill(0, pid)
           return pid
         rescue Errno::ESRCH
         end

data/lib/unicorn/app/exec_cgi.rb

@@ -0,0 +1,150 @@
+require 'unicorn'
+require 'rack'
+
+module Unicorn::App
+
+  # This class is highly experimental (even more so than the rest of Unicorn)
+  # and has never run anything other than cgit.
+  class ExecCgi
+
+    CHUNK_SIZE = 16384
+    PASS_VARS = %w(
+      CONTENT_LENGTH
+      CONTENT_TYPE
+      GATEWAY_INTERFACE
+      AUTH_TYPE
+      PATH_INFO
+      PATH_TRANSLATED
+      QUERY_STRING
+      REMOTE_ADDR
+      REMOTE_HOST
+      REMOTE_IDENT
+      REMOTE_USER
+      REQUEST_METHOD
+      SERVER_NAME
+      SERVER_PORT
+      SERVER_PROTOCOL
+      SERVER_SOFTWARE
+    ).map { |x| x.freeze }.freeze # frozen strings are faster for Hash lookups
+
+    # Intializes the app, example of usage in a config.ru
+    #   map "/cgit" do
+    #     run Unicorn::App::ExecCgi.new("/path/to/cgit.cgi")
+    #   end
+    def initialize(*args)
+      @args = args.dup
+      first = @args[0] or
+        raise ArgumentError, "need path to executable"
+      first[0..0] == "/" or @args[0] = ::File.expand_path(first)
+      File.executable?(@args[0]) or
+        raise ArgumentError, "#{@args[0]} is not executable"
+    end
+
+    # Calls the app
+    def call(env)
+      out, err = Tempfile.new(''), Tempfile.new('')
+      out.unlink
+      err.unlink
+      inp = force_file_input(env)
+      inp.sync = out.sync = err.sync = true
+      pid = fork { run_child(inp, out, err, env) }
+      inp.close
+      pid, status = Process.waitpid2(pid)
+      write_errors(env, err, status) if err.stat.size > 0
+      err.close
+
+      return parse_output!(out) if status.success?
+      out.close
+      [ 500, { 'Content-Length' => '0', 'Content-Type' => 'text/plain' }, [] ]
+    end
+
+    private
+
+    def run_child(inp, out, err, env)
+      PASS_VARS.each do |key|
+        val = env[key] or next
+        ENV[key] = val
+      end
+      ENV['SCRIPT_NAME'] = @args[0]
+      ENV['GATEWAY_INTERFACE'] = 'CGI/1.1'
+      env.keys.grep(/^HTTP_/) { |key| ENV[key] = env[key] }
+
+      IO.new(0).reopen(inp)
+      IO.new(1).reopen(out)
+      IO.new(2).reopen(err)
+      exec(*@args)
+    end
+
+    # Extracts headers from CGI out, will change the offset of out.
+    # This returns a standard Rack-compatible return value:
+    #   [ 200, HeadersHash, body ]
+    def parse_output!(out)
+      size = out.stat.size
+      out.sysseek(0)
+      head = out.sysread(CHUNK_SIZE)
+      offset = 2
+      head, body = head.split(/\n\n/, 2)
+      if body.nil?
+        head, body = head.split(/\r\n\r\n/, 2)
+        offset = 4
+      end
+      offset += head.length
+      out.instance_variable_set('@unicorn_app_exec_cgi_offset', offset)
+      size -= offset
+
+      # Allows +out+ to be used as a Rack body.
+      def out.each
+        sysseek(@unicorn_app_exec_cgi_offset)
+        begin
+          loop { yield(sysread(CHUNK_SIZE)) }
+        rescue EOFError
+        end
+      end
+
+      prev = nil
+      headers = Rack::Utils::HeaderHash.new
+      head.split(/\r?\n/).each do |line|
+        case line
+        when /^([A-Za-z0-9-]+):\s*(.*)$/ then headers[prev = $1] = $2
+        when /^[ \t]/ then headers[prev] << "\n#{line}" if prev
+        end
+      end
+      headers['Content-Length'] = size.to_s
+      [ 200, headers, out ]
+    end
+
+    # ensures rack.input is a file handle that we can redirect stdin to
+    def force_file_input(env)
+      inp = env['rack.input']
+      if inp.respond_to?(:fileno) && Integer === inp.fileno
+        inp
+      elsif inp.size == 0 # inp could be a StringIO or StringIO-like object
+        ::File.open('/dev/null')
+      else
+        tmp = Tempfile.new('')
+        tmp.unlink
+        tmp.binmode
+
+        # Rack::Lint::InputWrapper doesn't allow sysread :(
+        while buf = inp.read(CHUNK_SIZE)
+          tmp.syswrite(buf)
+        end
+        tmp.sysseek(0)
+        tmp
+      end
+    end
+
+    # rack.errors this may not be an IO object, so we couldn't
+    # just redirect the CGI executable to that earlier.
+    def write_errors(env, err, status)
+      err.seek(0)
+      dst = env['rack.errors']
+      pid = status.pid
+      dst.write("#{pid}: #{@args.inspect} status=#{status} stderr:\n")
+      err.each_line { |line| dst.write("#{pid}: #{line}") }
+      dst.flush
+    end
+
+  end
+
+end

data/lib/unicorn/configurator.rb

@@ -173,13 +173,14 @@ module Unicorn
     # worker processes.  For per-worker listeners, see the after_fork example
     def listeners(addresses)
       Array === addresses or addresses = Array(addresses)
+      addresses.map! { |addr| expand_addr(addr) }
       @set[:listeners] = addresses
     end
 
     # adds an +address+ to the existing listener set
     def listen(address)
       @set[:listeners] = [] unless Array === @set[:listeners]
-      @set[:listeners] << address
+      @set[:listeners] << expand_addr(address)
     end
 
     # sets the +path+ for the PID file of the unicorn master process
@@ -253,5 +254,17 @@ module Unicorn
       @set[var] = my_proc
     end
 
+    # expands pathnames of sockets if relative to "~" or "~username"
+    # expands "*:port and ":port" to "0.0.0.0:port"
+    def expand_addr(address) #:nodoc
+      return address unless String === address
+      if address[0..0] == '~'
+        return File.expand_path(address)
+      elsif address =~ %r{\A\*?:(\d+)\z}
+        return "0.0.0.0:#$1"
+      end
+      address
+    end
+
   end
 end

data/lib/unicorn/const.rb

@@ -68,7 +68,7 @@ module Unicorn
     REQUEST_URI='REQUEST_URI'.freeze
     REQUEST_PATH='REQUEST_PATH'.freeze
     
-    UNICORN_VERSION="0.2.1".freeze
+    UNICORN_VERSION="0.2.2".freeze
 
     UNICORN_TMP_BASE="unicorn".freeze
 

data/lib/unicorn/http_request.rb

@@ -130,8 +130,6 @@ module Unicorn
       raise "No REQUEST PATH" unless @params[Const::REQUEST_PATH]
 
       @params["QUERY_STRING"] ||= ''
-      @params.delete "HTTP_CONTENT_TYPE"
-      @params.delete "HTTP_CONTENT_LENGTH"
       @params.update({ "rack.version" => [0,1],
                       "rack.input" => @body,
                       "rack.errors" => $stderr,
@@ -155,7 +153,7 @@ module Unicorn
       end
       true # success!
     rescue Object => e
-      logger.error "Error reading HTTP body: #{e.inspect}"
+      @logger.error "Error reading HTTP body: #{e.inspect}"
       socket.closed? or socket.close rescue nil
 
       # Any errors means we should delete the file, including if the file

data/lib/unicorn/http_response.rb

@@ -21,35 +21,32 @@ module Unicorn
 
   class HttpResponse
 
-    # headers we allow duplicates for
-    ALLOWED_DUPLICATES = {
-      'Set-Cookie' => true,
-      'Set-Cookie2' => true,
-      'Warning' => true,
-      'WWW-Authenticate' => true,
-    }.freeze
+    # Rack does not set/require a Date: header.  We always override the
+    # Connection: and Date: headers no matter what (if anything) our
+    # Rack application sent us.
+    SKIP = { 'connection' => true, 'date' => true }.freeze
 
     # writes the rack_response to socket as an HTTP response
     def self.write(socket, rack_response)
       status, headers, body = rack_response
+      out = [ "Date: #{Time.now.httpdate}" ]
 
-      # Rack does not set/require Date, but don't worry about Content-Length
-      # since Rack applications that conform to Rack::Lint enforce that
-      out = [ "#{Const::DATE}: #{Time.now.httpdate}" ]
-      sent = { Const::CONNECTION => true, Const::DATE => true }
-
+      # Don't bother enforcing duplicate supression, it's a Hash most of
+      # the time anyways so just hope our app knows what it's doing
       headers.each do |key, value|
-        if ! sent[key] || ALLOWED_DUPLICATES[key]
-          sent[key] = true
-          out << "#{key}: #{value}"
-        end
+        next if SKIP.include?(key.downcase)
+        value.split(/\n/).each { |v| out << "#{key}: #{v}" }
       end
 
+      # Rack should enforce Content-Length or chunked transfer encoding,
+      # so don't worry or care about them.
       socket_write(socket,
                    "HTTP/1.1 #{status} #{HTTP_STATUS_CODES[status]}\r\n" \
                    "Connection: close\r\n" \
                    "#{out.join("\r\n")}\r\n\r\n")
       body.each { |chunk| socket_write(socket, chunk) }
+      ensure
+        body.respond_to?(:close) and body.close rescue nil
     end
 
     private

data/lib/unicorn/launcher.rb

@@ -0,0 +1,33 @@
+$stdin.sync = $stdout.sync = $stderr.sync = true
+require 'unicorn'
+
+class Unicorn::Launcher
+
+  # We don't do a lot of standard daemonization stuff:
+  #   * umask is whatever was set by the parent process at startup
+  #     and can be set in config.ru and config_file, so making it
+  #     0000 and potentially exposing sensitive log data can be bad
+  #     policy.
+  #   * don't bother to chdir("/") here since unicorn is designed to
+  #     run inside APP_ROOT.  Unicorn will also re-chdir() to
+  #     the directory it was started in when being re-executed
+  #     to pickup code changes if the original deployment directory
+  #     is a symlink or otherwise got replaced.
+  def self.daemonize!
+    $stdin.reopen("/dev/null")
+
+    # We only start a new process group if we're not being reexecuted
+    # and inheriting file descriptors from our parent
+    unless ENV['UNICORN_FD']
+      exit if fork
+      Process.setsid
+      exit if fork
+
+      # $stderr/$stderr can/will be redirected separately in the Unicorn config
+      $stdout.reopen("/dev/null", "a")
+      $stderr.reopen("/dev/null", "a")
+    end
+    $stdin.sync = $stdout.sync = $stderr.sync = true
+  end
+
+end

data/lib/unicorn/socket.rb

@@ -75,7 +75,6 @@ module Unicorn
     def bind_listen(address = '0.0.0.0:8080', backlog = 1024)
       return address unless String === address
 
-      address = File.expand_path(address) if address[0..0] == "~"
       domain, bind_addr = if address[0..0] == "/"
         if File.exist?(address)
           if File.socket?(address)

data/test/exec/test_exec.rb

@@ -300,7 +300,7 @@ end
       sleep DEFAULT_RES
       log = File.readlines(rotate.path)
     end
-    assert_equal 4, log.grep(/rotating logs\.\.\./).size
+    assert_equal 4, log.grep(/worker=\d+ rotating logs\.\.\./).size
     assert_equal 0, log.grep(/done rotating logs/).size
 
     tries = DEFAULT_TRIES
@@ -309,7 +309,7 @@ end
       sleep DEFAULT_RES
       log = File.readlines(COMMON_TMP.path)
     end
-    assert_equal 4, log.grep(/done rotating logs/).size
+    assert_equal 4, log.grep(/worker=\d+ done rotating logs/).size
     assert_equal 0, log.grep(/rotating logs\.\.\./).size
     assert_nothing_raised { Process.kill('QUIT', pid) }
     status = nil

data/test/unit/test_http_parser.rb

@@ -25,11 +25,10 @@ class HttpParserTest < Test::Unit::TestCase
     assert_equal '/', req['REQUEST_PATH']
     assert_equal 'HTTP/1.1', req['HTTP_VERSION']
     assert_equal '/', req['REQUEST_URI']
-    assert_equal 'CGI/1.2', req['GATEWAY_INTERFACE']
     assert_equal 'GET', req['REQUEST_METHOD']    
     assert_nil req['FRAGMENT']
     assert_nil req['QUERY_STRING']
-    
+
     parser.reset
     assert parser.nread == 0, "Number read after reset should be 0"
   end

data/test/unit/test_request.rb

@@ -0,0 +1,82 @@
+# Copyright (c) 2009 Eric Wong
+# You can redistribute it and/or modify it under the same terms as Ruby.
+
+if RUBY_VERSION =~ /1\.9/
+  warn "#$0 current broken under Ruby 1.9 with Rack"
+  exit 0
+end
+
+require 'test/test_helper'
+begin
+  require 'rack'
+  require 'rack/lint'
+rescue LoadError
+  warn "Unable to load rack, skipping test"
+  exit 0
+end
+
+include Unicorn
+
+class RequestTest < Test::Unit::TestCase
+
+  class MockRequest < StringIO
+    def unicorn_peeraddr
+      '666.666.666.666'
+    end
+  end
+
+  def setup
+    @request = HttpRequest.new(Logger.new($stderr))
+    @app = lambda do |env|
+      [ 200, { 'Content-Length' => '0', 'Content-Type' => 'text/plain' }, [] ]
+    end
+    @lint = Rack::Lint.new(@app)
+  end
+
+  def test_rack_lint_get
+    client = MockRequest.new("GET / HTTP/1.1\r\nHost: foo\r\n\r\n")
+    res = env = nil
+    assert_nothing_raised { env = @request.read(client) }
+    assert_equal '666.666.666.666', env['REMOTE_ADDR']
+    assert_nothing_raised { res = @lint.call(env) }
+  end
+
+  def test_rack_lint_put
+    client = MockRequest.new(
+      "PUT / HTTP/1.1\r\n" \
+      "Host: foo\r\n" \
+      "Content-Length: 5\r\n" \
+      "\r\n" \
+      "abcde")
+    res = env = nil
+    assert_nothing_raised { env = @request.read(client) }
+    assert_nothing_raised { res = @lint.call(env) }
+  end
+
+  def test_rack_lint_big_put
+    count = 100
+    bs = 0x10000
+    buf = (' ' * bs).freeze
+    length = bs * count
+    client = Tempfile.new('big_put')
+    def client.unicorn_peeraddr
+      '1.1.1.1'
+    end
+    client.syswrite(
+      "PUT / HTTP/1.1\r\n" \
+      "Host: foo\r\n" \
+      "Content-Length: #{length}\r\n" \
+      "\r\n")
+    count.times { assert_equal bs, client.syswrite(buf) }
+    assert_equal 0, client.sysseek(0)
+    res = env = nil
+    assert_nothing_raised { env = @request.read(client) }
+    assert_equal length, env['rack.input'].size
+    count.times { assert_equal buf, env['rack.input'].read(bs) }
+    assert_nil env['rack.input'].read(bs)
+    assert_nothing_raised { env['rack.input'].rewind }
+    assert_nothing_raised { res = @lint.call(env) }
+  end
+
+end
+

data/test/unit/test_response.rb

@@ -41,5 +41,12 @@ class ResponseTest < Test::Unit::TestCase
     io.rewind
     assert_match(/.* #{HTTP_STATUS_CODES[code]}$/, io.readline.chomp, "wrong default reason phrase")
   end
+
+  def test_rack_multivalue_headers
+    out = StringIO.new
+    HttpResponse.write(out,[200, {"X-Whatever" => "stuff\nbleh"}, []])
+    assert_match(/^X-Whatever: stuff\r\nX-Whatever: bleh\r\n/, out.string)
+  end
+
 end
 

data/unicorn.gemspec

@@ -2,17 +2,17 @@
 
 Gem::Specification.new do |s|
   s.name = %q{unicorn}
-  s.version = "0.2.1"
+  s.version = "0.2.2"
 
   s.required_rubygems_version = Gem::Requirement.new(">= 1.2") if s.respond_to? :required_rubygems_version=
   s.authors = ["Eric Wong"]
-  s.date = %q{2009-03-18}
+  s.date = %q{2009-03-22}
   s.description = %q{A small fast HTTP library and server for Rack applications.}
   s.email = %q{normalperson@yhbt.net}
   s.executables = ["unicorn", "unicorn_rails"]
   s.extensions = ["ext/unicorn/http11/extconf.rb"]
-  s.extra_rdoc_files = ["CHANGELOG", "LICENSE", "README", "TODO", "bin/unicorn", "bin/unicorn_rails", "ext/unicorn/http11/ext_help.h", "ext/unicorn/http11/extconf.rb", "ext/unicorn/http11/http11.c", "ext/unicorn/http11/http11_parser.c", "ext/unicorn/http11/http11_parser.h", "ext/unicorn/http11/http11_parser.rl", "ext/unicorn/http11/http11_parser_common.rl", "lib/unicorn.rb", "lib/unicorn/configurator.rb", "lib/unicorn/const.rb", "lib/unicorn/http_request.rb", "lib/unicorn/http_response.rb", "lib/unicorn/socket.rb", "lib/unicorn/util.rb"]
-  s.files = [".document", ".gitignore", "CHANGELOG", "CONTRIBUTORS", "DESIGN", "GNUmakefile", "LICENSE", "Manifest", "README", "Rakefile", "SIGNALS", "TODO", "bin/unicorn", "bin/unicorn_rails", "ext/unicorn/http11/ext_help.h", "ext/unicorn/http11/extconf.rb", "ext/unicorn/http11/http11.c", "ext/unicorn/http11/http11_parser.c", "ext/unicorn/http11/http11_parser.h", "ext/unicorn/http11/http11_parser.rl", "ext/unicorn/http11/http11_parser_common.rl", "lib/unicorn.rb", "lib/unicorn/configurator.rb", "lib/unicorn/const.rb", "lib/unicorn/http_request.rb", "lib/unicorn/http_response.rb", "lib/unicorn/socket.rb", "lib/unicorn/util.rb", "setup.rb", "test/aggregate.rb", "test/benchmark/previous.rb", "test/benchmark/simple.rb", "test/benchmark/utils.rb", "test/exec/README", "test/exec/test_exec.rb", "test/test_helper.rb", "test/tools/trickletest.rb", "test/unit/test_configurator.rb", "test/unit/test_http_parser.rb", "test/unit/test_response.rb", "test/unit/test_server.rb", "test/unit/test_upload.rb", "unicorn.gemspec"]
+  s.extra_rdoc_files = ["CHANGELOG", "LICENSE", "README", "TODO", "bin/unicorn", "bin/unicorn_rails", "ext/unicorn/http11/ext_help.h", "ext/unicorn/http11/extconf.rb", "ext/unicorn/http11/http11.c", "ext/unicorn/http11/http11_parser.c", "ext/unicorn/http11/http11_parser.h", "ext/unicorn/http11/http11_parser.rl", "ext/unicorn/http11/http11_parser_common.rl", "lib/unicorn.rb", "lib/unicorn/app/exec_cgi.rb", "lib/unicorn/configurator.rb", "lib/unicorn/const.rb", "lib/unicorn/http_request.rb", "lib/unicorn/http_response.rb", "lib/unicorn/launcher.rb", "lib/unicorn/socket.rb", "lib/unicorn/util.rb"]
+  s.files = [".document", ".gitignore", "CHANGELOG", "CONTRIBUTORS", "DESIGN", "GNUmakefile", "LICENSE", "Manifest", "README", "Rakefile", "SIGNALS", "TODO", "bin/unicorn", "bin/unicorn_rails", "ext/unicorn/http11/ext_help.h", "ext/unicorn/http11/extconf.rb", "ext/unicorn/http11/http11.c", "ext/unicorn/http11/http11_parser.c", "ext/unicorn/http11/http11_parser.h", "ext/unicorn/http11/http11_parser.rl", "ext/unicorn/http11/http11_parser_common.rl", "lib/unicorn.rb", "lib/unicorn/app/exec_cgi.rb", "lib/unicorn/configurator.rb", "lib/unicorn/const.rb", "lib/unicorn/http_request.rb", "lib/unicorn/http_response.rb", "lib/unicorn/launcher.rb", "lib/unicorn/socket.rb", "lib/unicorn/util.rb", "setup.rb", "test/aggregate.rb", "test/benchmark/previous.rb", "test/benchmark/simple.rb", "test/benchmark/utils.rb", "test/exec/README", "test/exec/test_exec.rb", "test/test_helper.rb", "test/tools/trickletest.rb", "test/unit/test_configurator.rb", "test/unit/test_http_parser.rb", "test/unit/test_request.rb", "test/unit/test_response.rb", "test/unit/test_server.rb", "test/unit/test_upload.rb", "unicorn.gemspec"]
   s.has_rdoc = true
   s.homepage = %q{http://unicorn.bogomips.org}
   s.rdoc_options = ["--line-numbers", "--inline-source", "--title", "Unicorn", "--main", "README"]
@@ -20,7 +20,7 @@ Gem::Specification.new do |s|
   s.rubyforge_project = %q{unicorn}
   s.rubygems_version = %q{1.3.1}
   s.summary = %q{A small fast HTTP library and server for Rack applications.}
-  s.test_files = ["test/unit/test_configurator.rb", "test/unit/test_response.rb", "test/unit/test_upload.rb", "test/unit/test_http_parser.rb", "test/unit/test_server.rb"]
+  s.test_files = ["test/unit/test_configurator.rb", "test/unit/test_server.rb", "test/unit/test_upload.rb", "test/unit/test_response.rb", "test/unit/test_http_parser.rb", "test/unit/test_request.rb"]
 
   if s.respond_to? :specification_version then
     current_version = Gem::Specification::CURRENT_SPECIFICATION_VERSION

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification 
 name: unicorn
 version: !ruby/object:Gem::Version 
-  version: 0.2.1
+  version: 0.2.2
 platform: ruby
 authors: 
 - Eric Wong
@@ -9,7 +9,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2009-03-18 00:00:00 -07:00
+date: 2009-03-22 00:00:00 -07:00
 default_executable: 
 dependencies: []
 
@@ -35,10 +35,12 @@ extra_rdoc_files:
 - ext/unicorn/http11/http11_parser.rl
 - ext/unicorn/http11/http11_parser_common.rl
 - lib/unicorn.rb
+- lib/unicorn/app/exec_cgi.rb
 - lib/unicorn/configurator.rb
 - lib/unicorn/const.rb
 - lib/unicorn/http_request.rb
 - lib/unicorn/http_response.rb
+- lib/unicorn/launcher.rb
 - lib/unicorn/socket.rb
 - lib/unicorn/util.rb
 files: 
@@ -64,10 +66,12 @@ files:
 - ext/unicorn/http11/http11_parser.rl
 - ext/unicorn/http11/http11_parser_common.rl
 - lib/unicorn.rb
+- lib/unicorn/app/exec_cgi.rb
 - lib/unicorn/configurator.rb
 - lib/unicorn/const.rb
 - lib/unicorn/http_request.rb
 - lib/unicorn/http_response.rb
+- lib/unicorn/launcher.rb
 - lib/unicorn/socket.rb
 - lib/unicorn/util.rb
 - setup.rb
@@ -81,6 +85,7 @@ files:
 - test/tools/trickletest.rb
 - test/unit/test_configurator.rb
 - test/unit/test_http_parser.rb
+- test/unit/test_request.rb
 - test/unit/test_response.rb
 - test/unit/test_server.rb
 - test/unit/test_upload.rb
@@ -119,7 +124,8 @@ specification_version: 2
 summary: A small fast HTTP library and server for Rack applications.
 test_files: 
 - test/unit/test_configurator.rb
-- test/unit/test_response.rb
+- test/unit/test_server.rb
 - test/unit/test_upload.rb
+- test/unit/test_response.rb
 - test/unit/test_http_parser.rb
-- test/unit/test_server.rb
+- test/unit/test_request.rb