unicorn-maintained 6.2.0

data/TUNING

@@ -0,0 +1,119 @@
+= Tuning unicorn
+
+unicorn performance is generally as good as a (mostly) Ruby web server
+can provide.  Most often the performance bottleneck is in the web
+application running on Unicorn rather than Unicorn itself.
+
+== unicorn Configuration
+
+See Unicorn::Configurator for details on the config file format.
++worker_processes+ is the most-commonly needed tuning parameter.
+
+=== Unicorn::Configurator#worker_processes
+
+* worker_processes should be scaled to the number of processes your
+  backend system(s) can support.  DO NOT scale it to the number of
+  external network clients your application expects to be serving.
+  unicorn is NOT for serving slow clients, that is the job of nginx.
+
+* worker_processes should be *at* *least* the number of CPU cores on
+  a dedicated server (unless you do not have enough memory).
+  If your application has occasionally slow responses that are /not/
+  CPU-intensive, you may increase this to workaround those inefficiencies.
+
+* Under Ruby 2.2 or later, Etc.nprocessors may be used to determine
+  the number of CPU cores present.
+
+* worker_processes may be increased for Unicorn::OobGC users to provide
+  more consistent response times.
+
+* Never, ever, increase worker_processes to the point where the system
+  runs out of physical memory and hits swap.  Production servers should
+  never see heavy swap activity.
+
+=== Unicorn::Configurator#listen Options
+
+* Setting a very low value for the :backlog parameter in "listen"
+  directives can allow failover to happen more quickly if your
+  cluster is configured for it.
+
+* If you're doing extremely simple benchmarks and getting connection
+  errors under high request rates, increasing your :backlog parameter
+  above the already-generous default of 1024 can help avoid connection
+  errors.  Keep in mind this is not recommended for real traffic if
+  you have another machine to failover to (see above).
+
+* :rcvbuf and :sndbuf parameters generally do not need to be set for TCP
+  listeners under Linux 2.6 because auto-tuning is enabled.  UNIX domain
+  sockets do not have auto-tuning buffer sizes; so increasing those will
+  allow syscalls and task switches to be saved for larger requests
+  and responses.  If your app only generates small responses or expects
+  small requests, you may shrink the buffer sizes to save memory, too.
+
+* Having socket buffers too large can also be detrimental or have
+  little effect.  Huge buffers can put more pressure on the allocator
+  and may also thrash CPU caches, cancelling out performance gains
+  one would normally expect.
+
+* UNIX domain sockets are slightly faster than TCP sockets, but only
+  work if nginx is on the same machine.
+
+== Other unicorn settings
+
+* Setting "preload_app true" can allow copy-on-write-friendly GC to
+  be used to save memory.  It will probably not work out of the box with
+  applications that open sockets or perform random I/O on files.
+  Databases like TokyoCabinet use concurrency-safe pread()/pwrite()
+  functions for safe sharing of database file descriptors across
+  processes.
+
+* On POSIX-compliant filesystems, it is safe for multiple threads or
+  processes to append to one log file as long as all the processes are
+  have them unbuffered (File#sync = true) or they are
+  record(line)-buffered in userspace before any writes.
+
+== Kernel Parameters (Linux sysctl and sysfs)
+
+WARNING: Do not change system parameters unless you know what you're doing!
+
+* Transparent hugepages (THP) improves performance in many cases,
+  but can also increase memory use when relying on a
+  copy-on-write(CoW)-friendly GC (Ruby 2.0+) with "preload_app true".
+  CoW operates at the page level, so writing to a huge page would
+  trigger a 2 MB copy (x86-64), as opposed to a 4 KB copy on a
+  regular (non-huge) page.
+
+  Consider only allowing THP to be used when it is requested via the
+  madvise(2) syscall:
+
+	echo madvise >/sys/kernel/mm/transparent_hugepage/enabled
+
+  Or disabling it system-wide, via "never".
+
+  n.b. "page" in this context only applies to the OS kernel,
+  Ruby GC implementations also use this term for the same concept
+  in a way that is agnostic to the OS.
+
+* net.core.rmem_max and net.core.wmem_max can increase the allowed
+  size of :rcvbuf and :sndbuf respectively. This is mostly only useful
+  for UNIX domain sockets which do not have auto-tuning buffer sizes.
+
+* For load testing/benchmarking with UNIX domain sockets, you should
+  consider increasing net.core.somaxconn or else nginx will start
+  failing to connect under heavy load.  You may also consider setting
+  a higher :backlog to listen on as noted earlier.
+
+* If you're running out of local ports, consider lowering
+  net.ipv4.tcp_fin_timeout to 20-30 (default: 60 seconds).  Also
+  consider widening the usable port range by changing
+  net.ipv4.ip_local_port_range.
+
+* Setting net.ipv4.tcp_timestamps=1 will also allow setting
+  net.ipv4.tcp_tw_reuse=1 and net.ipv4.tcp_tw_recycle=1, which along
+  with the above settings can slow down port exhaustion.  Not all
+  networks are compatible with these settings, check with your friendly
+  network administrator before changing these.
+
+* Increasing the MTU size can reduce framing overhead for larger
+  transfers.  One often-overlooked detail is that the loopback
+  device (usually "lo") can have its MTU increased, too.

data/archive/.gitignore

@@ -0,0 +1,3 @@
+/data
+/news
+/requests

data/archive/slrnpull.conf

@@ -0,0 +1,4 @@
+# group_name                         max        expire     headers_only
+gmane.comp.lang.ruby.unicorn.general 1000000000 1000000000 0
+
+# usage: slrnpull -d $PWD -h news.gmane.io --no-post

data/bin/unicorn

@@ -0,0 +1,128 @@
+#!/this/will/be/overwritten/or/wrapped/anyways/do/not/worry/ruby
+# -*- encoding: binary -*-
+require 'unicorn/launcher'
+require 'optparse'
+
+ENV["RACK_ENV"] ||= "development"
+rackup_opts = Unicorn::Configurator::RACKUP
+options = rackup_opts[:options]
+set_no_default_middleware = true
+
+op = OptionParser.new("", 24, '  ') do |opts|
+  cmd = File.basename($0)
+  opts.banner = "Usage: #{cmd} " \
+                "[ruby options] [#{cmd} options] [rackup config file]"
+  opts.separator "Ruby options:"
+
+  lineno = 1
+  opts.on("-e", "--eval LINE", "evaluate a LINE of code") do |line|
+    eval line, TOPLEVEL_BINDING, "-e", lineno
+    lineno += 1
+  end
+
+  opts.on("-d", "--debug", "set debugging flags (set $DEBUG to true)") do
+    $DEBUG = true
+  end
+
+  opts.on("-w", "--warn", "turn warnings on for your script") do
+    $-w = true
+  end
+
+  opts.on("-I", "--include PATH",
+          "specify $LOAD_PATH (may be used more than once)") do |path|
+    $LOAD_PATH.unshift(*path.split(':'))
+  end
+
+  opts.on("-r", "--require LIBRARY",
+          "require the library, before executing your script") do |library|
+    require library
+  end
+
+  opts.separator "#{cmd} options:"
+
+  # some of these switches exist for rackup command-line compatibility,
+
+  opts.on("-o", "--host HOST",
+          "listen on HOST (default: #{Unicorn::Const::DEFAULT_HOST})") do |h|
+    rackup_opts[:host] = h
+    rackup_opts[:set_listener] = true
+  end
+
+  opts.on("-p", "--port PORT", Integer,
+          "use PORT (default: #{Unicorn::Const::DEFAULT_PORT})") do |port|
+    rackup_opts[:port] = port
+    rackup_opts[:set_listener] = true
+  end
+
+  opts.on("-E", "--env RACK_ENV",
+          "use RACK_ENV for defaults (default: development)") do |e|
+    ENV["RACK_ENV"] = e
+  end
+
+  opts.on("-N", "--no-default-middleware",
+          "do not load middleware implied by RACK_ENV") do |e|
+    rackup_opts[:no_default_middleware] = true if set_no_default_middleware
+  end
+
+  opts.on("-D", "--daemonize", "run daemonized in the background") do |d|
+    rackup_opts[:daemonize] = !!d
+  end
+
+  opts.on("-P", "--pid FILE", "DEPRECATED") do |f|
+    warn %q{Use of --pid/-P is strongly discouraged}
+    warn %q{Use the 'pid' directive in the Unicorn config file instead}
+    options[:pid] = f
+  end
+
+  opts.on("-s", "--server SERVER",
+          "this flag only exists for compatibility") do |s|
+    warn "-s/--server only exists for compatibility with rackup"
+  end
+
+  # Unicorn-specific stuff
+  opts.on("-l", "--listen {HOST:PORT|PATH}",
+          "listen on HOST:PORT or PATH",
+          "this may be specified multiple times",
+          "(default: #{Unicorn::Const::DEFAULT_LISTEN})") do |address|
+    options[:listeners] << address
+  end
+
+  opts.on("-c", "--config-file FILE", "Unicorn-specific config file") do |f|
+    options[:config_file] = f
+  end
+
+  # I'm avoiding Unicorn-specific config options on the command-line.
+  # IMNSHO, config options on the command-line are redundant given
+  # config files and make things unnecessarily complicated with multiple
+  # places to look for a config option.
+
+  opts.separator "Common options:"
+
+  opts.on_tail("-h", "--help", "Show this message") do
+    puts opts.to_s.gsub(/^.*DEPRECATED.*$/s, '')
+    exit
+  end
+
+  opts.on_tail("-v", "--version", "Show version") do
+    puts "#{cmd} v#{Unicorn::Const::UNICORN_VERSION}"
+    exit
+  end
+
+  opts.parse! ARGV
+end
+
+set_no_default_middleware = false
+app = Unicorn.builder(ARGV[0] || 'config.ru', op)
+op = nil
+
+if $DEBUG
+  require 'pp'
+  pp({
+    :unicorn_options => options,
+    :app => app,
+    :daemonize => rackup_opts[:daemonize],
+  })
+end
+
+Unicorn::Launcher.daemonize!(options) if rackup_opts[:daemonize]
+Unicorn::HttpServer.new(app, options).start.join

data/bin/unicorn_rails

@@ -0,0 +1,209 @@
+#!/this/will/be/overwritten/or/wrapped/anyways/do/not/worry/ruby
+# -*- encoding: binary -*-
+require 'unicorn/launcher'
+require 'optparse'
+require 'fileutils'
+
+ENV['RAILS_ENV'] ||= "development"
+rackup_opts = Unicorn::Configurator::RACKUP
+options = rackup_opts[:options]
+
+op = OptionParser.new("", 24, '  ') do |opts|
+  cmd = File.basename($0)
+  opts.banner = "Usage: #{cmd} " \
+                "[ruby options] [#{cmd} options] [rackup config file]"
+  opts.separator "Ruby options:"
+
+  lineno = 1
+  opts.on("-e", "--eval LINE", "evaluate a LINE of code") do |line|
+    eval line, TOPLEVEL_BINDING, "-e", lineno
+    lineno += 1
+  end
+
+  opts.on("-d", "--debug", "set debugging flags (set $DEBUG to true)") do
+    $DEBUG = true
+  end
+
+  opts.on("-w", "--warn", "turn warnings on for your script") do
+    $-w = true
+  end
+
+  opts.on("-I", "--include PATH",
+          "specify $LOAD_PATH (may be used more than once)") do |path|
+    $LOAD_PATH.unshift(*path.split(':'))
+  end
+
+  opts.on("-r", "--require LIBRARY",
+          "require the library, before executing your script") do |library|
+    require library
+  end
+
+  opts.separator "#{cmd} options:"
+
+  # some of these switches exist for rackup command-line compatibility,
+
+  opts.on("-o", "--host HOST",
+          "listen on HOST (default: #{Unicorn::Const::DEFAULT_HOST})") do |h|
+    rackup_opts[:host] = h
+    rackup_opts[:set_listener] = true
+  end
+
+  opts.on("-p", "--port PORT", Integer,
+          "use PORT (default: #{Unicorn::Const::DEFAULT_PORT})") do |port|
+    rackup_opts[:port] = port
+    rackup_opts[:set_listener] = true
+  end
+
+  opts.on("-E", "--env RAILS_ENV",
+          "use RAILS_ENV for defaults (default: development)") do |e|
+    ENV['RAILS_ENV'] = e
+  end
+
+  opts.on("-D", "--daemonize", "run daemonized in the background") do |d|
+    rackup_opts[:daemonize] = !!d
+  end
+
+  # Unicorn-specific stuff
+  opts.on("-l", "--listen {HOST:PORT|PATH}",
+          "listen on HOST:PORT or PATH",
+          "this may be specified multiple times",
+          "(default: #{Unicorn::Const::DEFAULT_LISTEN})") do |address|
+    options[:listeners] << address
+  end
+
+  opts.on("-c", "--config-file FILE", "Unicorn-specific config file") do |f|
+    options[:config_file] = f
+  end
+
+  opts.on("-P PATH", "DEPRECATED") do |v|
+    warn %q{Use of -P is ambiguous and discouraged}
+    warn %q{Use --path or RAILS_RELATIVE_URL_ROOT instead}
+    ENV['RAILS_RELATIVE_URL_ROOT'] = v
+  end
+
+  opts.on("--path PATH", "Runs Rails app mounted at a specific path.",
+          "(default: /)") do |v|
+    ENV['RAILS_RELATIVE_URL_ROOT'] = v
+  end
+
+  # I'm avoiding Unicorn-specific config options on the command-line.
+  # IMNSHO, config options on the command-line are redundant given
+  # config files and make things unnecessarily complicated with multiple
+  # places to look for a config option.
+
+  opts.separator "Common options:"
+
+  opts.on_tail("-h", "--help", "Show this message") do
+    puts opts.to_s.gsub(/^.*DEPRECATED.*$/s, '')
+    exit
+  end
+
+  opts.on_tail("-v", "--version", "Show version") do
+    puts "#{cmd} v#{Unicorn::Const::UNICORN_VERSION}"
+    exit
+  end
+
+  opts.parse! ARGV
+end
+
+def rails_dispatcher
+  if ::Rails::VERSION::MAJOR >= 3 && ::File.exist?('config/application.rb')
+    if ::File.read('config/application.rb') =~ /^module\s+([\w:]+)\s*$/
+      app_module = Object.const_get($1)
+      begin
+        result = app_module::Application
+      rescue NameError
+      end
+    end
+  end
+
+  if result.nil? && defined?(ActionController::Dispatcher)
+    result = ActionController::Dispatcher.new
+  end
+
+  result || abort("Unable to locate the application dispatcher class")
+end
+
+def rails_builder(ru, op, daemonize)
+  return Unicorn.builder(ru, op) if ru
+
+  # allow Configurator to parse cli switches embedded in the ru file
+  Unicorn::Configurator::RACKUP.update(:file => :rails, :optparse => op)
+
+  # this lambda won't run until after forking if preload_app is false
+  # this runs after config file reloading
+  lambda do |x, server|
+    # Rails 3 includes a config.ru, use it if we find it after
+    # working_directory is bound.
+    ::File.exist?('config.ru') and
+      return Unicorn.builder('config.ru', op).call(x, server)
+
+    # Load Rails and (possibly) the private version of Rack it bundles.
+    begin
+      require ::File.expand_path('config/boot')
+      require ::File.expand_path('config/environment')
+    rescue LoadError => err
+      abort "#$0 must be run inside RAILS_ROOT: #{err.inspect}"
+    end
+
+    defined?(::Rails::VERSION::STRING) or
+      abort "Rails::VERSION::STRING not defined by config/{boot,environment}"
+    # it seems Rails >=2.2 support Rack, but only >=2.3 requires it
+    old_rails = case ::Rails::VERSION::MAJOR
+    when 0, 1 then true
+    when 2 then Rails::VERSION::MINOR < 3 ? true : false
+    else
+      false
+    end
+
+    Rack::Builder.new do
+      map_path = ENV['RAILS_RELATIVE_URL_ROOT'] || '/'
+      if old_rails
+        if map_path != '/'
+          # patches + tests welcome, but I really cbf to deal with this
+          # since all apps I've ever dealt with just use "/" ...
+          warn "relative URL roots may not work for older Rails"
+        end
+        warn "LogTailer not available for Rails < 2.3" unless daemonize
+        warn "Debugger not available" if $DEBUG
+        require 'unicorn/app/old_rails'
+        map(map_path) do
+          use Unicorn::App::OldRails::Static
+          run Unicorn::App::OldRails.new
+        end
+      else
+        use Rails::Rack::LogTailer unless daemonize
+        use Rails::Rack::Debugger if $DEBUG
+        map(map_path) do
+          unless defined?(ActionDispatch::Static)
+            use Rails::Rack::Static
+          end
+          run rails_dispatcher
+        end
+      end
+    end.to_app
+  end
+end
+
+app = rails_builder(ARGV[0], op, rackup_opts[:daemonize])
+op = nil
+
+if $DEBUG
+  require 'pp'
+  pp({
+    :unicorn_options => options,
+    :app => app,
+    :daemonize => rackup_opts[:daemonize],
+  })
+end
+
+# ensure Rails standard tmp paths exist
+options[:after_reload] = lambda do
+  FileUtils.mkdir_p(%w(cache pids sessions sockets).map! { |d| "tmp/#{d}" })
+end
+
+if rackup_opts[:daemonize]
+  options[:pid] = "tmp/pids/unicorn.pid"
+  Unicorn::Launcher.daemonize!(options)
+end
+Unicorn::HttpServer.new(app, options).start.join

data/examples/big_app_gc.rb

@@ -0,0 +1,2 @@
+# see {Unicorn::OobGC}[https://yhbt.net/unicorn/Unicorn/OobGC.html]
+# Unicorn::OobGC was broken in Unicorn v3.3.1 - v3.6.1 and fixed in v3.6.2

data/examples/echo.ru

@@ -0,0 +1,26 @@
+#\-E none
+#
+# Example application that echoes read data back to the HTTP client.
+# This emulates the old echo protocol people used to run.
+#
+# An example of using this in a client would be to run:
+#   curl --no-buffer -T- http://host:port/
+#
+# Then type random stuff in your terminal to watch it get echoed back!
+
+class EchoBody < Struct.new(:input)
+
+  def each(&block)
+    while buf = input.read(4096)
+      yield buf
+    end
+    self
+  end
+
+end
+
+run lambda { |env|
+  /\A100-continue\z/i =~ env['HTTP_EXPECT'] and return [100, {}, []]
+  [ 200, { 'Content-Type' => 'application/octet-stream' },
+    EchoBody.new(env['rack.input']) ]
+}

data/examples/init.sh

@@ -0,0 +1,102 @@
+#!/bin/sh
+set -e
+### BEGIN INIT INFO
+# Provides:          unicorn
+# Required-Start:    $local_fs $network
+# Required-Stop:     $local_fs $network
+# Default-Start:     2 3 4 5
+# Default-Stop:      0 1 6
+# Short-Description: Start/stop unicorn Rack app server
+### END INIT INFO
+
+# Example init script, this can be used with nginx, too,
+# since nginx and unicorn accept the same signals.
+
+# Feel free to change any of the following variables for your app:
+TIMEOUT=${TIMEOUT-60}
+APP_ROOT=/home/x/my_app/current
+PID=$APP_ROOT/tmp/pids/unicorn.pid
+CMD="/usr/bin/unicorn -D -c $APP_ROOT/config/unicorn.rb"
+INIT_CONF=$APP_ROOT/config/init.conf
+UPGRADE_DELAY=${UPGRADE_DELAY-2}
+action="$1"
+set -u
+
+test -f "$INIT_CONF" && . $INIT_CONF
+
+OLD="$PID.oldbin"
+
+cd $APP_ROOT || exit 1
+
+sig () {
+	test -s "$PID" && kill -$1 $(cat $PID)
+}
+
+oldsig () {
+	test -s "$OLD" && kill -$1 $(cat $OLD)
+}
+
+case $action in
+start)
+	sig 0 && echo >&2 "Already running" && exit 0
+	$CMD
+	;;
+stop)
+	sig QUIT && exit 0
+	echo >&2 "Not running"
+	;;
+force-stop)
+	sig TERM && exit 0
+	echo >&2 "Not running"
+	;;
+restart|reload)
+	sig HUP && echo reloaded OK && exit 0
+	echo >&2 "Couldn't reload, starting '$CMD' instead"
+	$CMD
+	;;
+upgrade)
+	if oldsig 0
+	then
+		echo >&2 "Old upgraded process still running with $OLD"
+		exit 1
+	fi
+
+	cur_pid=
+	if test -s "$PID"
+	then
+		cur_pid=$(cat $PID)
+	fi
+
+	if test -n "$cur_pid" &&
+			kill -USR2 "$cur_pid" &&
+			sleep $UPGRADE_DELAY &&
+			new_pid=$(cat $PID) &&
+			test x"$new_pid" != x"$cur_pid" &&
+			kill -0 "$new_pid" &&
+			kill -QUIT "$cur_pid"
+	then
+		n=$TIMEOUT
+		while kill -0 "$cur_pid" 2>/dev/null && test $n -ge 0
+		do
+			printf '.' && sleep 1 && n=$(( $n - 1 ))
+		done
+		echo
+
+		if test $n -lt 0 && kill -0 "$cur_pid" 2>/dev/null
+		then
+			echo >&2 "$cur_pid still running after $TIMEOUT seconds"
+			exit 1
+		fi
+		exit 0
+	fi
+	echo >&2 "Couldn't upgrade, starting '$CMD' instead"
+	$CMD
+	;;
+reopen-logs)
+	sig USR1
+	;;
+*)
+	echo >&2 "Usage: $0 <start|stop|restart|upgrade|force-stop|reopen-logs>"
+	exit 1
+	;;
+esac

data/examples/logger_mp_safe.rb

@@ -0,0 +1,25 @@
+# Multi-Processing-safe monkey patch for Logger
+#
+# This monkey patch fixes the case where "preload_app true" is used and
+# the application spawns a background thread upon being loaded.
+#
+# This removes all lock from the Logger code and solely relies on the
+# underlying filesystem to handle write(2) system calls atomically when
+# O_APPEND is used.  This is safe in the presence of both multiple
+# threads (native or green) and multiple processes when writing to
+# a filesystem with POSIX O_APPEND semantics.
+#
+# It should be noted that the original locking on Logger could _never_ be
+# considered reliable on non-POSIX filesystems with multiple processes,
+# either, so nothing is lost in that case.
+
+require 'logger'
+class Logger::LogDevice
+  def write(message)
+    @dev.syswrite(message)
+  end
+
+  def close
+    @dev.close
+  end
+end

data/examples/logrotate.conf

@@ -0,0 +1,44 @@
+# example logrotate config file, I usually keep this in
+# /etc/logrotate.d/unicorn_app on my Debian systems
+#
+# See the logrotate(8) manpage for more information:
+#    https://linux.die.net/man/8/logrotate
+#
+# public logrotate-related discussion in our archives:
+#    https://yhbt.net/unicorn-public/?q=logrotate
+
+# Modify the following glob to match the logfiles your app writes to:
+/var/log/unicorn_app/*.log {
+	# this first block is mostly just personal preference, though
+	# I wish logrotate offered an "hourly" option...
+	daily
+	missingok
+	rotate 180
+	compress # must use with delaycompress below
+	dateext
+
+	# this is important if using "compress" since we need to call
+	# the "lastaction" script below before compressing:
+	delaycompress
+
+	# note the lack of the evil "copytruncate" option in this
+	# config.  Unicorn supports the USR1 signal and we send it
+	# as our "lastaction" action:
+	lastaction
+		# For systemd users, assuming you use two services
+		# (as recommended) to allow zero-downtime upgrades.
+		# Only one service needs to be started, but signaling
+		# both here is harmless as long as they're both enabled
+		systemctl kill -s SIGUSR1 unicorn@1.service
+		systemctl kill -s SIGUSR1 unicorn@2.service
+
+		# Examples for other process management systems appreciated
+		# Mail us at unicorn-public@yhbt.net
+		# (see above for archives)
+
+		# If you use a pid file and assuming your pid file
+		# is in /var/run/unicorn_app/pid
+		pid=/var/run/unicorn_app/pid
+		test -s $pid && kill -USR1 "$(cat $pid)"
+	endscript
+}