unicorn 5.5.2 → 5.7.0

data/Links

@@ -2,7 +2,7 @@
 
 If you're interested in unicorn, you may be interested in some of the projects
 listed below.  If you have any links to add/change/remove, please tell us at
-mailto:unicorn-public@bogomips.org!
+mailto:unicorn-public@yhbt.net!
 
 == Disclaimer
 
@@ -23,10 +23,10 @@ or services behind them.
 * {golden_brindle}[https://github.com/simonoff/golden_brindle] - tool to
   manage multiple unicorn instances/applications on a single server
 
-* {raindrops}[https://bogomips.org/raindrops/] - real-time stats for
+* {raindrops}[https://yhbt.net/raindrops/] - real-time stats for
   preforking Rack servers
 
-* {UnXF}[https://bogomips.org/unxf/]  Un-X-Forward* the Rack environment,
+* {UnXF}[https://yhbt.net/unxf/]  Un-X-Forward* the Rack environment,
   useful since unicorn is designed to be deployed behind a reverse proxy.
 
 === unicorn is written to work with
@@ -52,7 +52,7 @@ or services behind them.
 * {Mongrel}[https://rubygems.org/gems/mongrel] - the awesome webserver
   unicorn is based on.  A historical archive of the mongrel dev list
   featuring early discussions of unicorn is available at:
-  https://bogomips.org/mongrel-devel/
+  https://yhbt.net/mongrel-devel/
 
-* {david}[https://bogomips.org/david.git] - a tool to explain why you need
+* {david}[https://yhbt.net/david.git] - a tool to explain why you need
   nginx in front of unicorn

data/README

@@ -80,12 +80,12 @@ You may install it via RubyGems on RubyGems.org:
 You can get the latest source via git from the following locations
 (these versions may not be stable):
 
-  https://bogomips.org/unicorn.git
+  https://yhbt.net/unicorn.git
   https://repo.or.cz/unicorn.git (mirror)
 
 You may browse the code from the web:
 
-* https://bogomips.org/unicorn.git
+* https://yhbt.net/unicorn.git
 * https://repo.or.cz/w/unicorn.git (gitweb)
 
 See the HACKING guide on how to contribute and build prerelease gems
@@ -133,13 +133,20 @@ and libraries which run on top of it.
 
 All feedback (bug reports, user/development dicussion, patches, pull
 requests) go to the mailing list/newsgroup.  See the ISSUES document for
-information on the {mailing list}[mailto:unicorn-public@bogomips.org].
+information on the {mailing list}[mailto:unicorn-public@yhbt.net].
+
+The mailing list is archived at https://yhbt.net/unicorn-public/
 
-The mailing list is archived at https://bogomips.org/unicorn-public/
 Read-only NNTP access is available at:
 nntp://news.public-inbox.org/inbox.comp.lang.ruby.unicorn and
-nntp://news.gmane.org/gmane.comp.lang.ruby.unicorn.general
+nntp://news.gmane.io/gmane.comp.lang.ruby.unicorn.general
+
+Read-only IMAP access is also avaialble at:
+imaps://news.public-inbox.org/inbox.comp.lang.ruby.unicorn.0 and
+imap://ou63pmih66umazou.onion/inbox.comp.lang.ruby.unicorn.0
+AUTH=ANONYMOUS mechanism is supported, as is any username+password
+combination.
 
 For the latest on unicorn releases, you may also finger us at
-unicorn@bogomips.org or check our NEWS page (and subscribe to our Atom
+unicorn@yhbt.net or check our NEWS page (and subscribe to our Atom
 feed).

data/SIGNALS

@@ -8,7 +8,7 @@ should be possible to easily share process management scripts between
 Unicorn and nginx.
 
 One example init script is distributed with unicorn:
-https://bogomips.org/unicorn/examples/init.sh
+https://yhbt.net/unicorn/examples/init.sh
 
 === Master Process
 

data/Sandbox

@@ -34,7 +34,7 @@ is the primary issue with sandboxing tools such as Bundler and Isolate.
 If you're bundling unicorn, use "bundle exec unicorn" (or "bundle exec
 unicorn_rails") to start unicorn with the correct environment variables
 
-ref: https://bogomips.org/unicorn-public/9ECF07C4-5216-47BE-961D-AFC0F0C82060@internetfamo.us/
+ref: https://yhbt.net/unicorn-public/9ECF07C4-5216-47BE-961D-AFC0F0C82060@internetfamo.us/
 
 Otherwise (if you choose to not sandbox your unicorn installation), we
 expect the tips for Isolate (below) apply, too.
@@ -44,7 +44,7 @@ expect the tips for Isolate (below) apply, too.
 This is no longer be an issue as of bundler 0.9.17
 
 ref:
-https://bogomips.org/unicorn-public/8FC34B23-5994-41CC-B5AF-7198EF06909E@tramchase.com/
+https://yhbt.net/unicorn-public/8FC34B23-5994-41CC-B5AF-7198EF06909E@tramchase.com/
 
 === BUNDLE_GEMFILE for Capistrano users
 

data/archive/slrnpull.conf

@@ -1,4 +1,4 @@
 # group_name                         max        expire     headers_only
 gmane.comp.lang.ruby.unicorn.general 1000000000 1000000000 0
 
-# usage: slrnpull -d $PWD -h news.gmane.org --no-post
+# usage: slrnpull -d $PWD -h news.gmane.io --no-post

data/examples/big_app_gc.rb

@@ -1,2 +1,2 @@
-# see {Unicorn::OobGC}[https://bogomips.org/unicorn/Unicorn/OobGC.html]
+# see {Unicorn::OobGC}[https://yhbt.net/unicorn/Unicorn/OobGC.html]
 # Unicorn::OobGC was broken in Unicorn v3.3.1 - v3.6.1 and fixed in v3.6.2

data/examples/logrotate.conf

@@ -5,7 +5,7 @@
 #    https://linux.die.net/man/8/logrotate
 #
 # public logrotate-related discussion in our archives:
-#    https://bogomips.org/unicorn-public/?q=logrotate
+#    https://yhbt.net/unicorn-public/?q=logrotate
 
 # Modify the following glob to match the logfiles your app writes to:
 /var/log/unicorn_app/*.log {
@@ -33,7 +33,7 @@
 		systemctl kill -s SIGUSR1 unicorn@2.service
 
 		# Examples for other process management systems appreciated
-		# Mail us at unicorn-public@bogomips.org
+		# Mail us at unicorn-public@yhbt.net
 		# (see above for archives)
 
 		# If you use a pid file and assuming your pid file

data/examples/nginx.conf

@@ -113,7 +113,7 @@ http {
     # try_files directive appeared in in nginx 0.7.27 and has stabilized
     # over time.  Older versions of nginx (e.g. 0.6.x) requires
     # "if (!-f $request_filename)" which was less efficient:
-    # https://bogomips.org/unicorn.git/tree/examples/nginx.conf?id=v3.3.1#n127
+    # https://yhbt.net/unicorn.git/tree/examples/nginx.conf?id=v3.3.1#n127
     try_files $uri/index.html $uri.html $uri @app;
 
     location @app {

data/examples/unicorn.conf.minimal.rb

@@ -1,9 +1,9 @@
 # Minimal sample configuration file for Unicorn (not Rack) when used
 # with daemonization (unicorn -D) started in your working directory.
 #
-# See https://bogomips.org/unicorn/Unicorn/Configurator.html for complete
+# See https://yhbt.net/unicorn/Unicorn/Configurator.html for complete
 # documentation.
-# See also https://bogomips.org/unicorn/examples/unicorn.conf.rb for
+# See also https://yhbt.net/unicorn/examples/unicorn.conf.rb for
 # a more verbose configuration using more features.
 
 listen 2007 # by default Unicorn listens on port 8080

data/examples/unicorn.conf.rb

@@ -2,10 +2,10 @@
 #
 # This configuration file documents many features of Unicorn
 # that may not be needed for some applications. See
-# https://bogomips.org/unicorn/examples/unicorn.conf.minimal.rb
+# https://yhbt.net/unicorn/examples/unicorn.conf.minimal.rb
 # for a much simpler configuration file.
 #
-# See https://bogomips.org/unicorn/Unicorn/Configurator.html for complete
+# See https://yhbt.net/unicorn/Unicorn/Configurator.html for complete
 # documentation.
 
 # Use at least one worker per core if you're on a dedicated server,

data/ext/unicorn_http/extconf.rb

@@ -1,6 +1,11 @@
 # -*- encoding: binary -*-
 require 'mkmf'
 
+unless RUBY_VERSION < '3.1'
+  warn "Unicorn was only tested against MRI up to 3.0.\n" \
+       "It might not properly work with #{RUBY_VERSION}"
+end
+
 have_macro("SIZEOF_OFF_T", "ruby.h") or check_sizeof("off_t", "sys/types.h")
 have_macro("SIZEOF_SIZE_T", "ruby.h") or check_sizeof("size_t", "sys/types.h")
 have_macro("SIZEOF_LONG", "ruby.h") or check_sizeof("long", "sys/types.h")

data/ext/unicorn_http/unicorn_http.rl

@@ -62,7 +62,8 @@ struct http_parser {
   } len;
 };
 
-static ID id_set_backtrace;
+static ID id_set_backtrace, id_is_chunked_p;
+static VALUE cHttpParser;
 
 #ifdef HAVE_RB_HASH_CLEAR /* Ruby >= 2.0 */
 #  define my_hash_clear(h) (void)rb_hash_clear(h)
@@ -220,6 +221,19 @@ static void write_cont_value(struct http_parser *hp,
   rb_str_buf_cat(hp->cont, vptr, end + 1);
 }
 
+static int is_chunked(VALUE v)
+{
+  /* common case first */
+  if (STR_CSTR_CASE_EQ(v, "chunked"))
+    return 1;
+
+  /*
+   * call Ruby function in unicorn/http_request.rb to deal with unlikely
+   * comma-delimited case
+   */
+  return rb_funcall(cHttpParser, id_is_chunked_p, 1, v) != Qfalse;
+}
+
 static void write_value(struct http_parser *hp,
                         const char *buffer, const char *p)
 {
@@ -246,7 +260,9 @@ static void write_value(struct http_parser *hp,
     f = uncommon_field(field, flen);
   } else if (f == g_http_connection) {
     hp_keepalive_connection(hp, v);
-  } else if (f == g_content_length) {
+  } else if (f == g_content_length && !HP_FL_TEST(hp, CHUNKED)) {
+    if (hp->len.content)
+      parser_raise(eHttpParserError, "Content-Length already set");
     hp->len.content = parse_length(RSTRING_PTR(v), RSTRING_LEN(v));
     if (hp->len.content < 0)
       parser_raise(eHttpParserError, "invalid Content-Length");
@@ -254,9 +270,30 @@ static void write_value(struct http_parser *hp,
       HP_FL_SET(hp, HASBODY);
     hp_invalid_if_trailer(hp);
   } else if (f == g_http_transfer_encoding) {
-    if (STR_CSTR_CASE_EQ(v, "chunked")) {
+    if (is_chunked(v)) {
+      if (HP_FL_TEST(hp, CHUNKED))
+        /*
+         * RFC 7230 3.3.1:
+         * A sender MUST NOT apply chunked more than once to a message body
+         * (i.e., chunking an already chunked message is not allowed).
+         */
+        parser_raise(eHttpParserError, "Transfer-Encoding double chunked");
+
       HP_FL_SET(hp, CHUNKED);
       HP_FL_SET(hp, HASBODY);
+
+      /* RFC 7230 3.3.3, 3: favor chunked if Content-Length exists */
+      hp->len.content = 0;
+    } else if (HP_FL_TEST(hp, CHUNKED)) {
+      /*
+       * RFC 7230 3.3.3, point 3 states:
+       * If a Transfer-Encoding header field is present in a request and
+       * the chunked transfer coding is not the final encoding, the
+       * message body length cannot be determined reliably; the server
+       * MUST respond with the 400 (Bad Request) status code and then
+       * close the connection.
+       */
+      parser_raise(eHttpParserError, "invalid Transfer-Encoding");
     }
     hp_invalid_if_trailer(hp);
   } else if (f == g_http_trailer) {
@@ -487,7 +524,7 @@ static void set_url_scheme(VALUE env, VALUE *server_port)
      * and X-Forwarded-Proto handling from this parser?  We've had it
      * forever and nobody has said anything against it, either.
      * Anyways, please send comments to our public mailing list:
-     * unicorn-public@bogomips.org (no HTML mail, no subscription necessary)
+     * unicorn-public@yhbt.net (no HTML mail, no subscription necessary)
      */
     scheme = rb_hash_aref(env, g_http_x_forwarded_ssl);
     if (!NIL_P(scheme) && STR_CSTR_EQ(scheme, "on")) {
@@ -931,7 +968,7 @@ static VALUE HttpParser_rssget(VALUE self)
 
 void Init_unicorn_http(void)
 {
-  VALUE mUnicorn, cHttpParser;
+  VALUE mUnicorn;
 
   mUnicorn = rb_define_module("Unicorn");
   cHttpParser = rb_define_class_under(mUnicorn, "HttpParser", rb_cObject);
@@ -991,5 +1028,6 @@ void Init_unicorn_http(void)
 #ifndef HAVE_RB_HASH_CLEAR
   id_clear = rb_intern("clear");
 #endif
+  id_is_chunked_p = rb_intern("is_chunked?");
 }
 #undef SET_GLOBAL

data/lib/unicorn.rb

@@ -96,7 +96,7 @@ def self.builder(ru, op)
 
   # returns an array of strings representing TCP listen socket addresses
   # and Unix domain socket paths.  This is useful for use with
-  # Raindrops::Middleware under Linux: https://bogomips.org/raindrops/
+  # Raindrops::Middleware under Linux: https://yhbt.net/raindrops/
   def self.listener_names
     Unicorn::HttpServer::LISTENERS.map do |io|
       Unicorn::SocketHelper.sock_name(io)

data/lib/unicorn/configurator.rb

@@ -3,11 +3,11 @@
 
 # Implements a simple DSL for configuring a unicorn server.
 #
-# See https://bogomips.org/unicorn/examples/unicorn.conf.rb and
-# https://bogomips.org/unicorn/examples/unicorn.conf.minimal.rb
+# See https://yhbt.net/unicorn/examples/unicorn.conf.rb and
+# https://yhbt.net/unicorn/examples/unicorn.conf.minimal.rb
 # example configuration files.  An example config file for use with
 # nginx is also available at
-# https://bogomips.org/unicorn/examples/nginx.conf
+# https://yhbt.net/unicorn/examples/nginx.conf
 #
 # See the link:/TUNING.html document for more information on tuning unicorn.
 class Unicorn::Configurator
@@ -53,6 +53,7 @@ class Unicorn::Configurator
         server.logger.info("worker=#{worker.nr} ready")
       },
     :pid => nil,
+    :early_hints => false,
     :worker_exec => false,
     :preload_app => false,
     :check_client_connection => false,
@@ -276,6 +277,15 @@ def default_middleware(bool)
     set_bool(:default_middleware, bool)
   end
 
+  # sets whether to enable the proposed early hints Rack API.
+  # If enabled, Rails 5.2+ will automatically send a 103 Early Hint
+  # for all the `javascript_include_tag` and `stylesheet_link_tag`
+  # in your response. See: https://api.rubyonrails.org/v5.2/classes/ActionDispatch/Request.html#method-i-send_early_hints
+  # See also https://tools.ietf.org/html/rfc8297
+  def early_hints(bool)
+    set_bool(:early_hints, bool)
+  end
+
   # sets listeners to the given +addresses+, replacing or augmenting the
   # current set.  This is for the global listener pool shared by all
   # worker processes.  For per-worker listeners, see the after_fork example

data/lib/unicorn/http_request.rb

@@ -188,4 +188,15 @@ def write_http_header(socket) # :nodoc:
       HTTP_RESPONSE_START.each { |c| socket.write(c) }
     end
   end
+
+  # called by ext/unicorn_http/unicorn_http.rl via rb_funcall
+  def self.is_chunked?(v) # :nodoc:
+    vals = v.split(/[ \t]*,[ \t]*/).map!(&:downcase)
+    if vals.pop == 'chunked'.freeze
+      return true unless vals.include?('chunked'.freeze)
+      raise Unicorn::HttpParserError, 'double chunked', []
+    end
+    return false unless vals.include?('chunked'.freeze)
+    raise Unicorn::HttpParserError, 'chunked not last', []
+  end
 end

data/lib/unicorn/http_server.rb

@@ -6,7 +6,7 @@
 # forked worker children.
 #
 # Users do not need to know the internals of this class, but reading the
-# {source}[https://bogomips.org/unicorn.git/tree/lib/unicorn/http_server.rb]
+# {source}[https://yhbt.net/unicorn.git/tree/lib/unicorn/http_server.rb]
 # is education for programmers wishing to learn how unicorn works.
 # See Unicorn::Configurator for information on how to configure unicorn.
 class Unicorn::HttpServer
@@ -15,7 +15,7 @@ class Unicorn::HttpServer
                 :before_fork, :after_fork, :before_exec,
                 :listener_opts, :preload_app,
                 :orig_app, :config, :ready_pipe, :user,
-                :default_middleware
+                :default_middleware, :early_hints
   attr_writer   :after_worker_exit, :after_worker_ready, :worker_exec
 
   attr_reader :pid, :logger
@@ -588,6 +588,25 @@ def handle_error(client, e)
   rescue
   end
 
+  def e103_response_write(client, headers)
+    response = if @request.response_start_sent
+      "103 Early Hints\r\n"
+    else
+      "HTTP/1.1 103 Early Hints\r\n"
+    end
+
+    headers.each_pair do |k, vs|
+      next if !vs || vs.empty?
+      values = vs.to_s.split("\n".freeze)
+      values.each do |v|
+        response << "#{k}: #{v}\r\n"
+      end
+    end
+    response << "\r\n".freeze
+    response << "HTTP/1.1 ".freeze if @request.response_start_sent
+    client.write(response)
+  end
+
   def e100_response_write(client, env)
     # We use String#freeze to avoid allocations under Ruby 2.1+
     # Not many users hit this code path, so it's better to reduce the
@@ -602,7 +621,15 @@ def e100_response_write(client, env)
   # once a client is accepted, it is processed in its entirety here
   # in 3 easy steps: read request, call app, write app response
   def process_client(client)
-    status, headers, body = @app.call(env = @request.read(client))
+    env = @request.read(client)
+
+    if early_hints
+      env["rack.early_hints"] = lambda do |headers|
+        e103_response_write(client, headers)
+      end
+    end
+
+    status, headers, body = @app.call(env)
 
     begin
       return if @request.hijacked?
@@ -686,6 +713,7 @@ def worker_loop(worker)
     trap(:USR1) { nr = -65536 }
 
     ready = readers.dup
+    nr_listeners = readers.size
     @after_worker_ready.call(self, worker)
 
     begin
@@ -708,7 +736,7 @@ def worker_loop(worker)
       # we're probably reasonably busy, so avoid calling select()
       # and do a speculative non-blocking accept() on ready listeners
       # before we sleep again in select().
-      unless nr == 0
+      if nr == nr_listeners
         tmp = ready.dup
         redo
       end

data/lib/unicorn/oob_gc.rb

@@ -43,8 +43,8 @@
 #     use Unicorn::OobGC, 2, %r{\A/(?:expensive/foo|more_expensive/foo)}
 #
 # Feedback from users of early implementations of this module:
-# * https://bogomips.org/unicorn-public/0BFC98E9-072B-47EE-9A70-05478C20141B@lukemelia.com/
-# * https://bogomips.org/unicorn-public/AANLkTilUbgdyDv9W1bi-s_W6kq9sOhWfmuYkKLoKGOLj@mail.gmail.com/
+# * https://yhbt.net/unicorn-public/0BFC98E9-072B-47EE-9A70-05478C20141B@lukemelia.com/
+# * https://yhbt.net/unicorn-public/AANLkTilUbgdyDv9W1bi-s_W6kq9sOhWfmuYkKLoKGOLj@mail.gmail.com/
 
 module Unicorn::OobGC
 

data/t/GNUmakefile

@@ -1,74 +1,5 @@
-# we can run tests in parallel with GNU make
+# there used to be more, here, but we stopped relying on recursive make
 all::
+	$(MAKE) -C .. test-integration
 
-pid := $(shell echo $$PPID)
-
-RUBY = ruby
-RAKE = rake
--include ../local.mk
-ifeq ($(RUBY_VERSION),)
-  RUBY_VERSION := $(shell $(RUBY) -e 'puts RUBY_VERSION')
-endif
-
-ifeq ($(RUBY_VERSION),)
-  $(error unable to detect RUBY_VERSION)
-endif
-
-RUBY_ENGINE := $(shell $(RUBY) -e 'puts((RUBY_ENGINE rescue "ruby"))')
-export RUBY_ENGINE
-
-MYLIBS := $(RUBYLIB)
-
-T = $(wildcard t[0-9][0-9][0-9][0-9]-*.sh)
-
-all:: $(T)
-
-# can't rely on "set -o pipefail" since we don't require bash or ksh93 :<
-t_pfx = trash/$@-$(RUBY_ENGINE)-$(RUBY_VERSION)
-TEST_OPTS =
-# TRACER = strace -f -o $(t_pfx).strace -s 100000
-# TRACER = /usr/bin/time -o $(t_pfx).time
-
-ifdef V
-  ifeq ($(V),2)
-    TEST_OPTS += --trace
-  else
-    TEST_OPTS += --verbose
-  endif
-endif
-
-random_blob:
-	dd if=/dev/urandom bs=1M count=30 of=$@.$(pid)
-	mv $@.$(pid) $@
-
-$(T): random_blob
-
-dependencies := socat curl
-deps := $(addprefix .dep+,$(dependencies))
-$(deps): dep_bin = $(lastword $(subst +, ,$@))
-$(deps):
-	@which $(dep_bin) > $@.$(pid) 2>/dev/null || :
-	@test -s $@.$(pid) || \
-	  { echo >&2 "E '$(dep_bin)' not found in PATH=$(PATH)"; exit 1; }
-	@mv $@.$(pid) $@
-dep: $(deps)
-
-test_prefix := $(CURDIR)/../test/$(RUBY_ENGINE)-$(RUBY_VERSION)
-$(test_prefix)/.stamp:
-	$(MAKE) -C .. test-install
-
-$(T): export RUBY := $(RUBY)
-$(T): export RAKE := $(RAKE)
-$(T): export PATH := $(test_prefix)/bin:$(PATH)
-$(T): export RUBYLIB := $(test_prefix)/lib:$(MYLIBS)
-$(T): dep $(test_prefix)/.stamp trash/.gitignore
-	$(TRACER) $(SHELL) $(SH_TEST_OPTS) $@ $(TEST_OPTS)
-
-trash/.gitignore:
-	mkdir -p $(@D)
-	echo '*' > $@
-
-clean:
-	$(RM) -r trash/*
-
-.PHONY: $(T) clean
+.PHONY: all