puma 2.0.0.b4-java → 2.0.0.b7-java

data/History.txt

@@ -1,3 +1,56 @@
+=== 2.0.0.b7 / 2013-03-18
+
+* 5 minor enhancements:
+
+  * Add -q option for :start
+  * Add -V, --version
+  * Add default Rack handler helper
+  * Upstart support
+  * Set worker directory from configuration file
+
+* 12 bug fixes:
+
+  * Close the binder in the right place. Fixes #192
+  * Handle early term in workers. Fixes #206
+  * Make sure that the default port is 80 when the request doesn't include HTTP_X_FORWARDED_PROTO.
+  * Prevent Errno::EBADF errors on restart when running ruby 2.0 
+  * Record the proper @master_pid
+  * Respect the header HTTP_X_FORWARDED_PROTO when the host doesn't include a port number.
+  * Retry EAGAIN/EWOULDBLOCK during syswrite
+  * Run exec properly to restart. Fixes #154
+  * Set Rack run_once to false
+  * Syncronize all access to @timeouts. Fixes #208
+  * Write out the state post-daemonize. Fixes #189
+  * Prevent crash when all workers are gone
+
+=== 2.0.0.b6 / 2013-02-06
+
+* 2 minor enhancements:
+
+  * Add hook for running when a worker boots
+  * Advertise the Configuration object for apps to use.
+
+* 1 bug fix:
+
+  * Change directory in working during upgrade. Fixes #185
+
+=== 2.0.0.b5 / 2013-02-05
+
+* 2 major features:
+  * Add phased worker upgrade
+  * Add support for the rack hijack protocol
+
+* 2 minor features:
+  * Add -R to specify the restart command
+  * Add config file option to specify the restart command
+
+* 5 bug fixes:
+  * Cleanup pipes properly. Fixes #182
+  * Daemonize earlier so that we don't lose app threads. Fixes #183
+  * Drain the notification pipe. Fixes #176, thanks @cryo28
+  * Move write_pid to after we daemonize. Fixes #180
+  * Redirect IO properly and emit message for checkpointing
+
 === 2.0.0.b4 / 2012-12-12
 
 * 4 bug fixes:

data/Manifest.txt

@@ -44,12 +44,17 @@ lib/puma/java_io_buffer.rb
 lib/puma/jruby_restart.rb
 lib/puma/minissl.rb
 lib/puma/null_io.rb
+lib/puma/rack_default.rb
 lib/puma/rack_patch.rb
 lib/puma/reactor.rb
 lib/puma/server.rb
 lib/puma/thread_pool.rb
+lib/puma/util.rb
 lib/rack/handler/puma.rb
 puma.gemspec
-tools/jungle/README.md
-tools/jungle/puma
-tools/jungle/run-puma
+tools/jungle/init.d/README.md
+tools/jungle/init.d/puma
+tools/jungle/init.d/run-puma
+tools/jungle/upstart/README.md
+tools/jungle/upstart/puma-manager.conf
+tools/jungle/upstart/puma.conf

data/README.md

@@ -66,7 +66,7 @@ Puma provides numerous options for controlling the operation of the server. Cons
 Puma utilizes a dynamic thread pool which you can modify. You can set the minimum and maximum number of threads that are available in the pool with the `-t` (or `--threads`) flag:
 
     $ puma -t 8:32
-    
+
 Puma will automatically scale the number of threads based on how much traffic is present. The current default is `0:16`. Feel free to experiment, but be careful not to set the number of maximum threads to a very large number, as you may exhaust resources on the system (or hit resource limits).
 
 ### Binding TCP / Sockets
@@ -95,9 +95,21 @@ Puma comes with a builtin status/control app that can be used query and control
 
 This directs puma to start the control server on localhost port 9293. Additionally, all requests to the control server will need to include `token=foo` as a query parameter. This allows for simple authentication. Check out https://github.com/puma/puma/blob/master/lib/puma/app/status.rb to see what the app has available.
 
+### Configuration file
+
+You can also provide a configuration file which puma will use:
+
+    $ puma --config /path/to/config
+
+or
+
+    $ puma -C /path/to/config
+
+Take the following [sample configuration](https://github.com/puma/puma/blob/master/examples/config.rb) as inspiration or check out [configuration.rb](https://github.com/puma/puma/blob/master/lib/puma/configuration.rb#L138) to see all available options.
+
 ## Restart
 
-Puma includes the ability to restart itself, allowing for new versions to be easily upgraded to. When available (currently anywhere but JRuby), puma performs a "hot restart". This is the same functionality available in *unicorn* and *nginx* which keep the server sockets open between restarts. This makes sure that no pending requests are dropped while the restart is taking place.
+Puma includes the ability to restart itself, allowing for new versions to be easily upgraded to. When available (MRI, Rubinius, JRuby), puma performs a "hot restart". This is the same functionality available in *unicorn* and *nginx* which keep the server sockets open between restarts. This makes sure that no pending requests are dropped while the restart is taking place.
 
 To perform a restart, there are 2 builtin mechanism:
 
@@ -120,7 +132,7 @@ If you start puma with `-S some/path` then you can pass that same path to the `p
 
     $ pumactl -S some/path command
 
-or 
+or
 
     $ pumactl -C url -T token command
 
@@ -128,9 +140,28 @@ will cause the server to perform a restart. `pumactl` is a simple CLI frontend t
 
 Allowed commands: status, restart, halt, stop
 
-## Managing multiple Pumas / init.d script 
+## Managing multiple Pumas / init.d / upstart scripts
+
+If you want an easy way to manage multiple scripts at once check [tools/jungle](https://github.com/puma/puma/tree/master/tools/jungle) for init.d and upstart scripts.
+
+## Capistrano deployment
+
+Puma has included Capistrano [deploy script](https://github.com/plentz/puma/blob/master/lib/puma/capistrano.rb), you just need require that:
+
+config/deploy.rb
+
+```ruby
+require 'puma/capistrano'
+```
+
+and then
+
+```bash
+$ bunde exec cap puma:start
+$ bunde exec cap puma:restart 
+$ bunde exec cap puma:stop
+```
 
-If you want an easy way to manage multiple scripts at once check [tools/jungle](https://github.com/puma/puma/tree/master/tools/jungle) for an init.d script.
 
 ## License
 

data/Rakefile

@@ -19,7 +19,7 @@ HOE = Hoe.spec "puma" do
 
   require_ruby_version ">= 1.8.7"
 
-  dependency "rack", "~> 1.2"
+  dependency "rack", [">= 1.1", "< 2.0"]
 
   extra_dev_deps << ["rake-compiler", "~> 0.8.0"]
 end

data/lib/puma/binder.rb

@@ -15,7 +15,7 @@ module Puma
         "rack.errors".freeze => events.stderr,
         "rack.multithread".freeze => true,
         "rack.multiprocess".freeze => false,
-        "rack.run_once".freeze => true,
+        "rack.run_once".freeze => false,
         "SCRIPT_NAME".freeze => ENV['SCRIPT_NAME'] || "",
 
         # Rack blows up if this is an empty string, and Rack::Lint

data/lib/puma/capistrano.rb

@@ -1,26 +1,28 @@
 Capistrano::Configuration.instance.load do
-  after "deploy:stop", "puma:stop"
-  after "deploy:start", "puma:start"
-  after "deploy:restart", "puma:restart"
+  after 'deploy:stop', 'puma:stop'
+  after 'deploy:start', 'puma:start'
+  after 'deploy:restart', 'puma:restart'
 
-  _cset(:puma_role)      { :app }
+  _cset(:puma_cmd) { "#{fetch(:bundle_cmd, 'bundle')} exec puma" }
+  _cset(:pumactl_cmd) { "#{fetch(:bundle_cmd, 'bundle')} exec pumactl" }
+  _cset(:puma_state) { "#{shared_path}/sockets/puma.state" }
+  _cset(:puma_role) { :app }
 
   namespace :puma do
-    desc "Start puma"
+    desc 'Start puma'
     task :start, :roles => lambda { fetch(:puma_role) }, :on_no_matching_servers => :continue do
-      puma_env = fetch(:rack_env, fetch(:rails_env, "production"))
-      run "cd #{current_path} && #{fetch(:bundle_cmd, "bundle")} exec puma -d -e #{puma_env} -b 'unix://#{shared_path}/sockets/puma.sock' -S #{shared_path}/sockets/puma.state --control 'unix://#{shared_path}/sockets/pumactl.sock'", :pty => false
+      puma_env = fetch(:rack_env, fetch(:rails_env, 'production'))
+      run "cd #{current_path} && #{fetch(:puma_cmd)} -q -d -e #{puma_env} -b 'unix://#{shared_path}/sockets/puma.sock' -S #{fetch(:puma_state)} --control 'unix://#{shared_path}/sockets/pumactl.sock'", :pty => false
     end
 
-    desc "Stop puma"
+    desc 'Stop puma'
     task :stop, :roles => lambda { fetch(:puma_role) }, :on_no_matching_servers => :continue do
-      run "cd #{current_path} && #{fetch(:bundle_cmd, "bundle")} exec pumactl -S #{shared_path}/sockets/puma.state stop"
+      run "cd #{current_path} && #{fetch(:pumactl_cmd)} -S fetch(:puma_state) stop"
     end
 
-    desc "Restart puma"
+    desc 'Restart puma'
     task :restart, :roles => lambda { fetch(:puma_role) }, :on_no_matching_servers => :continue do
-      run "cd #{current_path} && #{fetch(:bundle_cmd, "bundle")} exec pumactl -S #{shared_path}/sockets/puma.state restart"
+      run "cd #{current_path} && #{fetch(:pumactl_cmd)} -S fetch(:puma_state) restart"
     end
-
   end
 end

data/lib/puma/cli.rb

@@ -7,6 +7,7 @@ require 'puma/configuration'
 require 'puma/binder'
 require 'puma/detect'
 require 'puma/daemon_ext'
+require 'puma/util'
 
 require 'rack/commonlogger'
 require 'rack/utils'
@@ -27,6 +28,7 @@ module Puma
       @stdout = stdout
       @stderr = stderr
 
+      @phase = 0
       @workers = []
 
       @events = Events.new @stdout, @stderr
@@ -35,6 +37,10 @@ module Puma
       @status = nil
 
       @restart = false
+      @phased_state = :idle
+
+      @io_redirected = false
+
 
       ENV['NEWRELIC_DISPATCHER'] ||= "puma"
 
@@ -60,6 +66,7 @@ module Puma
 
         if s_env.ino == s_pwd.ino and s_env.dev == s_pwd.dev
           @restart_dir = dir
+          @options[:worker_directory] = dir
         end
       end
 
@@ -82,6 +89,10 @@ module Puma
           arg0 = [Gem.ruby, "-S", $0]
         end
 
+        # Detect and reinject -Ilib from the command line
+        lib = File.expand_path "lib"
+        arg0[1,0] = ["-I", lib] if $:[0] == lib
+
         @restart_argv = arg0 + ARGV
       end
     end
@@ -104,10 +115,12 @@ module Puma
         end
 
         require 'puma/jruby_restart'
-        JRubyRestart.chdir_exec(@restart_dir, Gem.ruby, *@restart_argv)
+        JRubyRestart.chdir_exec(@restart_dir, @restart_argv)
       else
+        redirects = {}
         @binder.listeners.each_with_index do |(l,io),i|
           ENV["PUMA_INHERIT_#{i}"] = "#{io.to_i}:#{l}"
+          redirects[io.to_i] = io.to_i
         end
 
         if cmd = @options[:restart_cmd]
@@ -117,6 +130,8 @@ module Puma
         end
 
         Dir.chdir @restart_dir
+
+        argv += [redirects] unless RUBY_VERSION < '1.9'
         Kernel.exec(*argv)
       end
     end
@@ -164,6 +179,7 @@ module Puma
         :binds => [],
         :workers => 0,
         :daemon => false,
+        :worker_boot => [],
         :environment => "development"
       }
 
@@ -201,6 +217,7 @@ module Puma
 
         o.on "--dir DIR", "Change to DIR before starting" do |d|
           @options[:directory] = d.to_s
+          @options[:worker_directory] = d.to_s
         end
 
         o.on "-e", "--environment ENVIRONMENT",
@@ -225,7 +242,7 @@ module Puma
           @options[:quiet] = true
         end
 
-        o.on "--restart-cmd CMD",
+        o.on "-R", "--restart-cmd CMD",
              "The puma command to run during a hot restart",
              "Default: inferred" do |cmd|
           @options[:restart_cmd] = cmd
@@ -246,6 +263,11 @@ module Puma
           end
         end
 
+        o.on "-V", "--version", "Print the version information" do
+          puts "puma version #{Puma::Const::VERSION}"
+          exit 1
+        end
+
         o.on "-w", "--workers COUNT",
                    "Activate cluster mode: How many worker processes to create" do |arg|
           unsupported "-w not supported on JRuby and Windows",
@@ -291,13 +313,16 @@ module Puma
     end
 
     def write_state
+      write_pid
+
       require 'yaml'
 
       if path = @options[:state]
         state = { "pid" => Process.pid }
 
         cfg = @config.dup
-        cfg.options.delete :on_restart
+        
+        [ :logger, :worker_boot, :on_restart ].each { |o| cfg.options.delete o }
 
         state["config"] = cfg
 
@@ -316,6 +341,10 @@ module Puma
       end
 
       @config = Puma::Configuration.new @options
+
+      # Advertise the Configuration
+      Puma.cli_config = @config
+
       @config.load
     end
 
@@ -333,11 +362,14 @@ module Puma
       append = @options[:redirect_append]
 
       if stdout
+        @io_redirected = true
         STDOUT.reopen stdout, (append ? "a" : "w")
+        STDOUT.puts "=== puma startup: #{Time.now} ==="
       end
 
       if stderr
-        STDOUT.reopen stderr, (append ? "a" : "w")
+        STDERR.reopen stderr, (append ? "a" : "w")
+        STDERR.puts "=== puma startup: #{Time.now} ==="
       end
     end
 
@@ -364,9 +396,6 @@ module Puma
 
       set_rack_environment
 
-      write_pid
-      write_state
-
       if clustered
         run_cluster
       else
@@ -384,6 +413,12 @@ module Puma
 
       @binder.parse @options[:binds], self
 
+      if @options[:daemon]
+        Process.daemon(true, @io_redirected)
+      end
+
+      write_state
+
       server = Puma::Server.new @config.app, @events
       server.binder = @binder
       server.min_threads = min_t
@@ -441,9 +476,7 @@ module Puma
         log "*** Sorry signal SIGTERM not implemented, gracefully stopping feature disabled!"
       end
 
-      if @options[:daemon]
-        Process.daemon(true)
-      else
+      unless @options[:daemon]
         log "Use Ctrl-C to stop"
       end
 
@@ -469,31 +502,51 @@ module Puma
       end
     end
 
-    def worker
+    def worker(upgrade)
       $0 = "puma: cluster worker: #{@master_pid}"
       Signal.trap "SIGINT", "IGNORE"
 
+      @master_read.close
       @suicide_pipe.close
 
       Thread.new do
         IO.select [@check_pipe]
-        log "! Detected parent died, dieing"
+        log "! Detected parent died, dying"
         exit! 1
       end
 
+      # Be sure to change the directory again before loading
+      # the app. This way we can pick up new code.
+      if upgrade
+        if dir = @options[:worker_directory]
+          log "+ Changing to #{dir}"
+          Dir.chdir dir
+        end
+      end
+
+      # Invoke any worker boot hooks so they can get
+      # things in shape before booting the app.
+      hooks = @options[:worker_boot]
+      hooks.each { |h| h.call }
+
       min_t = @options[:min_threads]
       max_t = @options[:max_threads]
 
       server = Puma::Server.new @config.app, @events
       server.min_threads = min_t
       server.max_threads = max_t
-      server.binder = @binder
+      server.inherit_binder @binder
 
       Signal.trap "SIGTERM" do
         server.stop
       end
 
+      @worker_write << "b#{Process.pid}\n"
+
       server.run.join
+
+    ensure
+      @worker_write.close
     end
 
     def stop_workers
@@ -509,12 +562,27 @@ module Puma
       end
     end
 
+    def start_phased_restart
+      @phase += 1
+      log "- Starting phased worker restart, phase: #{@phase}"
+    end
+
     class Worker
-      def initialize(pid)
+      def initialize(pid, phase)
         @pid = pid
+        @phase = phase
+        @stage = :started
       end
 
-      attr_reader :pid
+      attr_reader :pid, :phase
+
+      def booted?
+        @stage == :booted
+      end
+
+      def boot!
+        @stage = :booted
+      end
 
       def term
         begin
@@ -527,15 +595,25 @@ module Puma
     def spawn_workers
       diff = @options[:workers] - @workers.size
 
+      upgrade = (@phased_state == :waiting)
+
       diff.times do
-        pid = fork { worker }
+        pid = fork { worker(upgrade) }
         debug "Spawned worker: #{pid}"
-        @workers << Worker.new(pid)
+        @workers << Worker.new(pid, @phase)
+      end
+
+      if diff > 0
+        @phased_state = :idle
       end
     end
 
+    def all_workers_booted?
+      @workers.count { |w| !w.booted? } == 0
+    end
+
     def check_workers
-      while true
+      while @workers.any?
         pid = Process.waitpid(-1, Process::WNOHANG)
         break unless pid
 
@@ -543,6 +621,20 @@ module Puma
       end
 
       spawn_workers
+
+      if @phased_state == :idle && all_workers_booted?
+        # If we're running at proper capacity, check to see if
+        # we need to phase any workers out (which will restart
+        # in the right phase).
+        #
+        w = @workers.find { |x| x.phase != @phase }
+
+        if w
+          @phased_state = :waiting
+          log "- Stopping #{w.pid} for phased upgrade..."
+          w.term
+        end
+      end
     end
 
     def run_cluster
@@ -553,9 +645,7 @@ module Puma
 
       @binder.parse @options[:binds], self
 
-      @master_pid = Process.pid
-
-      read, write = IO.pipe
+      read, write = Puma::Util.pipe
 
       Signal.trap "SIGCHLD" do
         write.write "!"
@@ -572,9 +662,29 @@ module Puma
       rescue Exception
       end
 
+      master_pid = Process.pid
+
       begin
         Signal.trap "SIGTERM" do
-          stop = true
+          # The worker installs there own SIGTERM when booted.
+          # Until then, this is run by the worker and the worker
+          # should just exit if they get it.
+          if Process.pid != master_pid
+            log "Early termination of worker"
+            exit! 0
+          else
+            stop = true
+            write.write "!"
+          end
+        end
+      rescue Exception
+      end
+
+      phased_restart = false
+
+      begin
+        Signal.trap "SIGUSR1" do
+          phased_restart = true
           write.write "!"
         end
       rescue Exception
@@ -585,16 +695,21 @@ module Puma
       # master has exited and @suicide_pipe has been automatically
       # closed.
       #
-      @check_pipe, @suicide_pipe = IO.pipe
+      @check_pipe, @suicide_pipe = Puma::Util.pipe
 
       if @options[:daemon]
-        Process.daemon(true)
+        Process.daemon(true, @io_redirected)
       else
         log "Use Ctrl-C to stop"
       end
 
+      @master_pid = Process.pid
+
       redirect_io
 
+      write_state
+
+      @master_read, @worker_write = read, write
       spawn_workers
 
       Signal.trap "SIGINT" do
@@ -605,8 +720,30 @@ module Puma
       begin
         while !stop
           begin
-            IO.select([read], nil, nil, 5)
+            res = IO.select([read], nil, nil, 5)
+
+            if res
+              req = read.read_nonblock(1)
+
+              if req == "b"
+                pid = read.gets.to_i
+                w = @workers.find { |x| x.pid == pid }
+                if w
+                  w.boot!
+                  log "- Worker #{pid} booted, phase: #{w.phase}"
+                else
+                  log "! Out-of-sync worker list, no #{pid} worker"
+                end
+              end
+            end
+
             check_workers
+
+            if phased_restart
+              start_phased_restart
+              phased_restart = false
+            end
+
           rescue Interrupt
             stop = true
           end
@@ -615,6 +752,10 @@ module Puma
         stop_workers
       ensure
         delete_pidfile
+        @check_pipe.close
+        @suicide_pipe.close
+        read.close
+        write.close
       end
 
       if @restart