puma 5.0.0.beta1 → 5.0.0.beta2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d624c4973e123e4b085bd0080a85d76c996f26743dde5d8e55ca4f3971e6d77a
-  data.tar.gz: 8f3ebe99584b845456708a73ae5305b012058c09af0a5e75bead0322c1c8501c
+  metadata.gz: b265505e1f4f00fe8c7de1c1d66103664bdd5920c2aed54b66e3303ded01c881
+  data.tar.gz: 3c921f609fd679b3b8cfbbc807678c16d09b03f43a8104913f0e0dd54e7e93f1
 SHA512:
-  metadata.gz: 1dbaeeb104fa56b185b1b647fc865a3b25eb621dba523dba15d056bef66137279fbb57545330ab3f486e610a88ed487f3ccf2e6f0b9c75de12522dc9db9b0029
-  data.tar.gz: 1b2f86bb5275f3764b891da8c3bc3643e1544d7209385aae8d87b5aa572945ceb627ef70df23060bc27abc9714b16377b8fa371b3ac3a10738e76cbd69f88841
+  metadata.gz: 74dce09aab280163c048a0c3ae3a81cd16b2ac32c7641fd43640d65797a61edf6e25a11f5111324111cd0068a98fc2ec13fcb3ac44154cef39d4bb3eb839a392
+  data.tar.gz: c6cf2c5d6d29867130be2fa572d5aea12a45a50052841a2de26e0a4600004f6fffae80922a9c641e8a155c8967ea8349d759b62a7ae9c2f6cbc84433fd092d13

data/History.md

@@ -6,11 +6,12 @@
   * EXPERIMENTAL: Added `nakayoshi_fork` option. Reduce memory usage in preloaded cluster-mode apps by GCing before fork and compacting, where available. (#2093, #2256)
   * Added pumactl `thread-backtraces` command to print thread backtraces (#2054)
   * Added incrementing `requests_count` to `Puma.stats`. (#2106)
-  * Increased maximum URI path length from 2048 to 8196 bytes (#2167)
+  * Increased maximum URI path length from 2048 to 8192 bytes (#2167, #2344)
   * `lowlevel_error_handler` is now called during a forced threadpool shutdown, and if a callable with 3 arguments is set, we now also pass the status code (#2203)
   * Faster phased restart and worker timeout (#2220)
   * Added `state_permission` to config DSL to set state file permissions (#2238)
   * Added `Puma.stats_hash`, which returns a stats in Hash instead of a JSON string (#2086, #2253)
+  * `rack.multithread` and `rack.multiprocess` now dynamically resolved by `max_thread` and `workers` respectively (#2288)
 
 * Deprecations, Removals and Breaking API Changes
   * `--control` has been removed. Use `--control-url` (#1487)
@@ -24,8 +25,12 @@
   * Daemonization has been removed without replacement. (#2170)
   * Changed #connected_port to #connected_ports (#2076)
   * Configuration: `environment` is read from `RAILS_ENV`, if `RACK_ENV` can't be found (#2022)
+  * Log binding on http:// for TCP bindings to make it clickable
 
 * Bugfixes
+  * Fix JSON loading issues on phased-restarts (#2269)
+  * Improve shutdown reliability (#2312, #2338)
+  * Close client http connections made to an ssl server with TLSv1.3 (#2116)
   * Do not set user_config to quiet by default to allow for file config (#2074)
   * Always close SSL connection in Puma::ControlCLI (#2211)
   * Windows update extconf.rb for use with ssp and varied Ruby/MSYS2 combinations (#2069)
@@ -44,6 +49,15 @@
   * Fix `UserFileDefaultOptions#fetch` to properly use `default` (#2233)
   * Improvements to `out_of_band` hook (#2234)
   * Prefer the rackup file specified by the CLI (#2225)
+  * Fix for spawning subprocesses with fork_worker option (#2267)
+  * Set `CONTENT_LENGTH` for chunked requests (#2287)
+  * JRuby - Add Puma::MiniSSL::Engine#init? and #teardown methods, run all SSL tests (#2317)
+  * Improve shutdown reliability (#2312)
+  * Resolve issue with threadpool waiting counter decrement when thread is killed
+  * Constrain rake-compiler version to 0.9.4 to fix `ClassNotFound` exception when using MiniSSL with Java8.
+  * Fix recursive `prune_bundler` (#2319).
+  * Ensure that TCP_CORK is usable
+  * Fix corner case when request body is chunked (#2326)
 
 * Refactor
   * Remove unused loader argument from Plugin initializer (#2095)
@@ -54,6 +68,14 @@
   * ThreadPool concurrency refactoring (#2220)
   * JSON parse cluster worker stats instead of regex (#2124)
   * Support parallel tests in verbose progress reporting (#2223)
+  * Refactor error handling in server accept loop (#2239)
+
+## 4.3.4/4.3.5 and 3.12.5/3.12.6 / 2020-05-22
+
+Each patchlevel release contains a separate security fix. We recommend simply upgrading to 4.3.5/3.12.6.
+
+* Security
+  * Fix: Fixed two separate HTTP smuggling vulnerabilities that used the Transfer-Encoding header. CVE-2020-11076 and CVE-2020-11077.
 
 ## 4.3.3 and 3.12.4 / 2020-02-28
 

data/README.md

@@ -8,7 +8,7 @@
 
 [![Code Climate](https://codeclimate.com/github/puma/puma.svg)](https://codeclimate.com/github/puma/puma)
 [![SemVer](https://api.dependabot.com/badges/compatibility_score?dependency-name=puma&package-manager=bundler&version-scheme=semver)](https://dependabot.com/compatibility-score.html?dependency-name=puma&package-manager=bundler&version-scheme=semver)
-[![StackOverflow](http://img.shields.io/badge/stackoverflow-Puma-blue.svg)]( http://stackoverflow.com/questions/tagged/puma )
+[![StackOverflow](https://img.shields.io/badge/stackoverflow-Puma-blue.svg)]( https://stackoverflow.com/questions/tagged/puma )
 
 Puma is a **simple, fast, multi-threaded, and highly concurrent HTTP 1.1 server for Ruby/Rack applications**.
 
@@ -27,7 +27,7 @@ $ gem install puma
 $ puma
 ```
 
-Without arguments, puma will look for a rackup (.ru) file in 
+Without arguments, puma will look for a rackup (.ru) file in
 working directory called `config.ru`.
 
 ## Frameworks
@@ -135,7 +135,7 @@ Preloading can’t be used with phased restart, since phased restart kills and r
 If puma encounters an error outside of the context of your application, it will respond with a 500 and a simple
 textual error message (see `lowlevel_error` in [this file](https://github.com/puma/puma/blob/master/lib/puma/server.rb)).
 You can specify custom behavior for this scenario. For example, you can report the error to your third-party
-error-tracking service (in this example, [rollbar](http://rollbar.com)):
+error-tracking service (in this example, [rollbar](https://rollbar.com)):
 
 ```ruby
 lowlevel_error_handler do |e|

data/docs/architecture.md

@@ -2,7 +2,7 @@
 
 ## Overview
 
-![http://bit.ly/2iJuFky](images/puma-general-arch.png)
+![https://bit.ly/2iJuFky](images/puma-general-arch.png)
 
 Puma is a threaded web server, processing requests across a TCP or UNIX socket.
 
@@ -12,7 +12,7 @@ Clustered mode is shown/discussed here. Single mode is analogous to having a sin
 
 ## Connection pipeline
 
-![http://bit.ly/2zwzhEK](images/puma-connection-flow.png)
+![https://bit.ly/2zwzhEK](images/puma-connection-flow.png)
 
 * Upon startup, Puma listens on a TCP or UNIX socket.
   * The backlog of this socket is configured (with a default of 1024), determining how many established but unaccepted connections can exist concurrently.
@@ -29,7 +29,7 @@ Clustered mode is shown/discussed here. Single mode is analogous to having a sin
 
 ### Disabling `queue_requests`
 
-![http://bit.ly/2zxCJ1Z](images/puma-connection-flow-no-reactor.png)
+![https://bit.ly/2zxCJ1Z](images/puma-connection-flow-no-reactor.png)
 
 The `queue_requests` option is `true` by default, enabling the separate thread used to buffer requests as described above.
 

data/docs/deployment.md

@@ -20,7 +20,10 @@ Welcome back!
 Puma was originally conceived as a thread-only webserver, but grew the ability to
 also use processes in version 2.
 
-Here are some rules of thumb:
+To run puma in single mode (e.g. for a development environment) you will need to
+set the number of workers to 0, anything above will run in cluster mode.
+
+Here are some rules of thumb for cluster mode:
 
 ### MRI
 
@@ -66,7 +69,8 @@ thread to become available.
 
 * Have your upstream proxy set a header with the time it received the request:
     * nginx: `proxy_set_header X-Request-Start "${msec}";`
-    * haproxy: `http-request set-header X-Request-Start "%t";`
+    * haproxy >= 1.9: `http-request set-header X-Request-Start t=%[date()]%[date_us()]`
+    * haproxy < 1.9: `http-request set-header X-Request-Start t=%[date()]`
 * In your Rack middleware, determine the amount of time elapsed since `X-Request-Start`.
 * To improve accuracy, you will want to subtract time spent waiting for slow clients:
     * `env['puma.request_body_wait']` contains the number of milliseconds Puma spent

data/docs/signals.md

@@ -1,8 +1,8 @@
-The [unix signal](http://en.wikipedia.org/wiki/Unix_signal) is a method of sending messages between [processes](http://en.wikipedia.org/wiki/Process_(computing)). When a signal is sent, the operating system interrupts the target process's normal flow of execution. There are standard signals that are used to stop a process but there are also custom signals that can be used for other purposes. This document is an attempt to list all supported signals that Puma will respond to. In general, signals need only be sent to the master process of a cluster.
+The [unix signal](https://en.wikipedia.org/wiki/Unix_signal) is a method of sending messages between [processes](https://en.wikipedia.org/wiki/Process_(computing)). When a signal is sent, the operating system interrupts the target process's normal flow of execution. There are standard signals that are used to stop a process but there are also custom signals that can be used for other purposes. This document is an attempt to list all supported signals that Puma will respond to. In general, signals need only be sent to the master process of a cluster.
 
 ## Sending Signals
 
-If you are new to signals it can be useful to see how they can be used. When a process is created in a *nix like operating system it will have a [PID - or process identifier](http://en.wikipedia.org/wiki/Process_identifier) that can be used to send signals to the process. For demonstration we will create an infinitely running process by tailing a file:
+If you are new to signals it can be useful to see how they can be used. When a process is created in a *nix like operating system it will have a [PID - or process identifier](https://en.wikipedia.org/wiki/Process_identifier) that can be used to send signals to the process. For demonstration we will create an infinitely running process by tailing a file:
 
 ```sh
 $ echo "foo" >> my.log
@@ -17,13 +17,13 @@ $ ps aux | grep tail
 schneems        87152   0.0  0.0  2432772    492 s032  S+   12:46PM   0:00.00 tail -f my.log
 ```
 
-You can send a signal in Ruby using the [Process module](http://www.ruby-doc.org/core-2.1.1/Process.html#kill-method):
+You can send a signal in Ruby using the [Process module](https://www.ruby-doc.org/core-2.1.1/Process.html#kill-method):
 
 ```
 $ irb
 > puts pid
 => 87152
-Process.detach(pid) # http://ruby-doc.org/core-2.1.1/Process.html#method-c-detach
+Process.detach(pid) # https://ruby-doc.org/core-2.1.1/Process.html#method-c-detach
 Process.kill("TERM", pid)
 ```
 

data/ext/puma_http11/http11_parser.c

@@ -14,12 +14,14 @@
 
 /*
  * capitalizes all lower-case ASCII characters,
- * converts dashes to underscores.
+ * converts dashes to underscores, and underscores to commas.
  */
 static void snake_upcase_char(char *c)
 {
     if (*c >= 'a' && *c <= 'z')
       *c &= ~0x20;
+    else if (*c == '_')
+      *c = ',';
     else if (*c == '-')
       *c = '_';
 }

data/ext/puma_http11/http11_parser.rl

@@ -12,12 +12,14 @@
 
 /*
  * capitalizes all lower-case ASCII characters,
- * converts dashes to underscores.
+ * converts dashes to underscores, and underscores to commas.
  */
 static void snake_upcase_char(char *c)
 {
     if (*c >= 'a' && *c <= 'z')
       *c &= ~0x20;
+    else if (*c == '_')
+      *c = ',';
     else if (*c == '-')
       *c = '_';
 }

data/ext/puma_http11/mini_ssl.c

@@ -301,6 +301,7 @@ void raise_error(SSL* ssl, int result) {
   char msg[512];
   const char* err_str;
   int err = errno;
+  int mask = 4095;
   int ssl_err = SSL_get_error(ssl, result);
   int verify_err = (int) SSL_get_verify_result(ssl);
 
@@ -317,8 +318,8 @@ void raise_error(SSL* ssl, int result) {
     } else {
       err = (int) ERR_get_error();
       ERR_error_string_n(err, buf, sizeof(buf));
-      snprintf(msg, sizeof(msg), "OpenSSL error: %s - %d", buf, err);
-
+      int errexp = err & mask;
+      snprintf(msg, sizeof(msg), "OpenSSL error: %s - %d", buf, errexp);
     }
   } else {
     snprintf(msg, sizeof(msg), "Unknown OpenSSL error: %d", ssl_err);
@@ -462,6 +463,13 @@ VALUE engine_peercert(VALUE self) {
   return rb_cert_buf;
 }
 
+static VALUE
+engine_ssl_vers_st(VALUE self) {
+  ms_conn* conn;
+  Data_Get_Struct(self, ms_conn, conn);
+  return rb_ary_new3(2, rb_str_new2(SSL_get_version(conn->ssl)), rb_str_new2(SSL_state_string(conn->ssl)));
+}
+
 VALUE noop(VALUE self) {
   return Qnil;
 }
@@ -533,6 +541,8 @@ void Init_mini_ssl(VALUE puma) {
   rb_define_method(eng, "init?", engine_init, 0);
 
   rb_define_method(eng, "peercert", engine_peercert, 0);
+
+  rb_define_method(eng, "ssl_vers_st", engine_ssl_vers_st, 0);
 }
 
 #else

data/ext/puma_http11/org/jruby/puma/MiniSSL.java

@@ -120,6 +120,8 @@ public class MiniSSL extends RubyObject {
   }
 
   private SSLEngine engine;
+  private boolean closed;
+  private boolean handshake;
   private MiniSSLBuffer inboundNetData;
   private MiniSSLBuffer outboundAppData;
   private MiniSSLBuffer outboundNetData;
@@ -157,6 +159,8 @@ public class MiniSSL extends RubyObject {
     SSLContext sslCtx = SSLContext.getInstance("TLS");
 
     sslCtx.init(kmf.getKeyManagers(), tmf.getTrustManagers(), null);
+    closed = false;
+    handshake = false;
     engine = sslCtx.createSSLEngine();
 
     String[] protocols;
@@ -173,7 +177,7 @@ public class MiniSSL extends RubyObject {
     engine.setEnabledProtocols(protocols);
     engine.setUseClientMode(false);
 
-    long verify_mode = miniSSLContext.callMethod(threadContext, "verify_mode").convertToInteger().getLongValue();
+    long verify_mode = miniSSLContext.callMethod(threadContext, "verify_mode").convertToInteger("to_i").getLongValue();
     if ((verify_mode & 0x1) != 0) { // 'peer'
         engine.setWantClientAuth(true);
     }
@@ -240,14 +244,21 @@ public class MiniSSL extends RubyObject {
           // need to wait for more data to come in before we retry
           retryOp = false;
           break;
+        case CLOSED:
+          closed = true;
+          retryOp = false;
+          break;
         default:
-          // other cases are OK and CLOSED.  We're done here.
+          // other case is OK.  We're done here.
           retryOp = false;
       }
+      if (res.getHandshakeStatus() == HandshakeStatus.FINISHED) {
+        handshake = true;
+      }
     }
 
     // after each op, run any delegated tasks if needed
-    if(engine.getHandshakeStatus() == HandshakeStatus.NEED_TASK) {
+    if(res.getHandshakeStatus() == HandshakeStatus.NEED_TASK) {
       Runnable runnable;
       while ((runnable = engine.getDelegatedTask()) != null) {
         runnable.run();
@@ -271,13 +282,14 @@ public class MiniSSL extends RubyObject {
 
       HandshakeStatus handshakeStatus = engine.getHandshakeStatus();
       boolean done = false;
+      SSLEngineResult res = null;
       while (!done) {
         switch (handshakeStatus) {
           case NEED_WRAP:
-            doOp(SSLOperation.WRAP, inboundAppData, outboundNetData);
+            res = doOp(SSLOperation.WRAP, inboundAppData, outboundNetData);
             break;
           case NEED_UNWRAP:
-            SSLEngineResult res = doOp(SSLOperation.UNWRAP, inboundNetData, inboundAppData);
+            res = doOp(SSLOperation.UNWRAP, inboundNetData, inboundAppData);
             if (res.getStatus() == Status.BUFFER_UNDERFLOW) {
               // need more data before we can shake more hands
               done = true;
@@ -286,7 +298,9 @@ public class MiniSSL extends RubyObject {
           default:
             done = true;
         }
-        handshakeStatus = engine.getHandshakeStatus();
+        if (!done) {
+          handshakeStatus = res.getHandshakeStatus();
+        }
       }
 
       if (inboundNetData.hasRemaining()) {
@@ -360,4 +374,21 @@ public class MiniSSL extends RubyObject {
       return getRuntime().getNil();
     }
   }
+
+  @JRubyMethod(name = "init?")
+  public IRubyObject isInit(ThreadContext context) {
+    return handshake ? getRuntime().getFalse() : getRuntime().getTrue();
+  }
+
+  @JRubyMethod
+  public IRubyObject shutdown() {
+    if (closed || engine.isInboundDone() && engine.isOutboundDone()) {
+      if (engine.isOutboundDone()) {
+        engine.closeOutbound();
+      }
+      return getRuntime().getTrue();
+    } else {
+      return getRuntime().getFalse();
+    }
+  }
 }

data/ext/puma_http11/puma_http11.c

@@ -54,7 +54,7 @@ DEF_MAX_LENGTH(FIELD_NAME, 256);
 DEF_MAX_LENGTH(FIELD_VALUE, 80 * 1024);
 DEF_MAX_LENGTH(REQUEST_URI, 1024 * 12);
 DEF_MAX_LENGTH(FRAGMENT, 1024); /* Don't know if this length is specified somewhere or not */
-DEF_MAX_LENGTH(REQUEST_PATH, 8196);
+DEF_MAX_LENGTH(REQUEST_PATH, 8192);
 DEF_MAX_LENGTH(QUERY_STRING, (1024 * 10));
 DEF_MAX_LENGTH(HEADER, (1024 * (80 + 32)));
 

data/lib/puma.rb

@@ -20,6 +20,7 @@ module Puma
   end
 
   def self.stats
+    require 'json'
     @get_stats.stats.to_json
   end
 

data/lib/puma/app/status.rb

@@ -1,7 +1,5 @@
 # frozen_string_literal: true
 
-require 'json'
-
 module Puma
   module App
     # Check out {#call}'s source code to see what actions this web application
@@ -19,6 +17,10 @@ module Puma
           return rack_response(403, 'Invalid auth token', 'text/plain')
         end
 
+        if env['PATH_INFO'] =~ /\/(gc-stats|stats|thread-backtraces)$/
+          require 'json'
+        end
+
         case env['PATH_INFO']
         when /\/stop$/
           @cli.stop

data/lib/puma/binder.rb

@@ -6,6 +6,7 @@ require 'socket'
 require 'puma/const'
 require 'puma/util'
 require 'puma/minissl/context_builder'
+require 'puma/configuration'
 
 module Puma
   class Binder
@@ -13,7 +14,7 @@ module Puma
 
     RACK_VERSION = [1,6].freeze
 
-    def initialize(events)
+    def initialize(events, conf = Configuration.new)
       @events = events
       @listeners = []
       @inherited_fds = {}
@@ -23,8 +24,8 @@ module Puma
       @proto_env = {
         "rack.version".freeze => RACK_VERSION,
         "rack.errors".freeze => events.stderr,
-        "rack.multithread".freeze => true,
-        "rack.multiprocess".freeze => false,
+        "rack.multithread".freeze => conf.options[:max_threads] > 1,
+        "rack.multiprocess".freeze => conf.options[:workers] >= 1,
         "rack.run_once".freeze => false,
         "SCRIPT_NAME".freeze => ENV['SCRIPT_NAME'] || "",
 
@@ -113,7 +114,7 @@ module Puma
                 i.local_address.ip_unpack.join(':')
               end
 
-              logger.log "* #{log_msg} on tcp://#{addr}"
+              logger.log "* #{log_msg} on http://#{addr}"
             end
           end
 

data/lib/puma/client.rb

@@ -308,8 +308,16 @@ module Puma
 
       te = @env[TRANSFER_ENCODING2]
 
-      if te && CHUNKED.casecmp(te) == 0
-        return setup_chunked_body(body)
+      if te
+        if te.include?(",")
+          te.split(",").each do |part|
+            if CHUNKED.casecmp(part.strip) == 0
+              return setup_chunked_body(body)
+            end
+          end
+        elsif CHUNKED.casecmp(te) == 0
+          return setup_chunked_body(body)
+        end
       end
 
       @chunked_body = false
@@ -412,7 +420,10 @@ module Puma
           raise EOFError
         end
 
-        return true if decode_chunk(chunk)
+        if decode_chunk(chunk)
+          @env[CONTENT_LENGTH] = @chunked_content_length
+          return true
+        end
       end
     end
 
@@ -424,20 +435,36 @@ module Puma
       @body = Tempfile.new(Const::PUMA_TMP_BASE)
       @body.binmode
       @tempfile = @body
+      @chunked_content_length = 0
+
+      if decode_chunk(body)
+        @env[CONTENT_LENGTH] = @chunked_content_length
+        return true
+      end
+    end
 
-      return decode_chunk(body)
+    def write_chunk(str)
+      @chunked_content_length += @body.write(str)
     end
 
     def decode_chunk(chunk)
       if @partial_part_left > 0
         if @partial_part_left <= chunk.size
           if @partial_part_left > 2
-            @body << chunk[0..(@partial_part_left-3)] # skip the \r\n
+            write_chunk(chunk[0..(@partial_part_left-3)]) # skip the \r\n
           end
           chunk = chunk[@partial_part_left..-1]
           @partial_part_left = 0
         else
-          @body << chunk if @partial_part_left > 2 # don't include the last \r\n
+          if @partial_part_left > 2
+            if @partial_part_left == chunk.size + 1
+              # Don't include the last \r
+              write_chunk(chunk[0..(@partial_part_left-3)])
+            else
+              # don't include the last \r\n
+              write_chunk(chunk)
+            end
+          end
           @partial_part_left -= chunk.size
           return false
         end
@@ -484,12 +511,12 @@ module Puma
 
           case
           when got == len
-            @body << part[0..-3] # to skip the ending \r\n
+            write_chunk(part[0..-3]) # to skip the ending \r\n
           when got <= len - 2
-            @body << part
+            write_chunk(part)
             @partial_part_left = len - part.size
           when got == len - 1 # edge where we get just \r but not \n
-            @body << part[0..-2]
+            write_chunk(part[0..-2])
             @partial_part_left = len - part.size
           end
         else