pitchfork 0.1.0

data/docs/REFORKING.md

@@ -0,0 +1,113 @@
+## Reforking
+
+Reforking is `pitchfork`'s main feature. To understand how it works, you must first understand Copy-on-Write.
+
+### Copy-on-Write
+
+In old UNIX systems of the ’70s or ’80s, forking a process involved copying its entire addressable memory over
+to the new process address space, effectively doubling the memory usage. But since the mid ’90s, that’s no longer
+true as, most, if not all, fork implementations are now sophisticated enough to trick the processes into thinking
+they have their own private memory regions, while in reality they’re sharing it with other processes.
+
+When the child process is forked, its pages tables are initialized to point to the parent’s memory pages. Later on,
+if either the parent or the child tries to write in one of these pages, the operating system is notified and will
+actually copy the page before it’s modified.
+
+This means that if neither the child nor the parent write in these shared pages after the fork happens,
+forked processes are essentially free.
+
+### Shared Memory Invalidation in Ruby
+
+So in theory, preforking servers shouldn't use more memory than threaded servers.
+
+However, in a Ruby process, there is generally a lot of memory regions that are lazily initialized.
+This include the Ruby Virtual Machine inline caches, JITed code if you use YJIT, and also
+some common patterns in applications, such as memoization:
+
+```ruby
+module MyCode
+  def self.some_data
+    @some_data ||= File.read("path/to/something")
+  end
+end
+```
+
+However, since workers are forked right after boot, most codepaths have never been executed,
+so most of these caches are not yet initialized.
+
+As more code get executed, more an more memory pages get invalidated. If you were to graph the ratio
+of shared memory of a ruby process over time, you'd likely see a logarithmic curve, with a quick degradation
+during the first few processed request as the most common code paths get warmed up, and then a stabilization.
+
+### Reforking
+
+That is where reforking helps. Since move of these invalidations only happens when a codepath is executed for the
+first time, if you take a warmed up worker out of rotation, and use it to fork new workers, warmed up pages will
+be shared again, and most of them won't be invalidated anymore.
+
+
+When you start `pitchfork` it forks the desired amount of workers:
+
+```
+PID   COMMAND
+100   \_ pitchfork master
+101       \_ pitchfork worker[0] (gen:0)
+102       \_ pitchfork worker[1] (gen:0)
+103       \_ pitchfork worker[2] (gen:0)
+104       \_ pitchfork worker[3] (gen:0)
+```
+
+When a reforking is triggered, one of the workers is selected to become a `mold`, and is taken out of rotation.
+When promoted, molds no longer process any incoming HTTP requests, they become inactive:
+
+```
+PID   COMMAND
+100   \_ pitchfork master
+101       \_ pitchfork mold (gen:1)
+102       \_ pitchfork worker[1] (gen:0)
+103       \_ pitchfork worker[2] (gen:0)
+104       \_ pitchfork worker[3] (gen:0)
+```
+
+When a new mold has been promoted, `pitchfork` starts a slow rollout of older workers and replace them with fresh workers
+forked from the mold:
+
+```
+PID   COMMAND
+100   \_ pitchfork master
+101       \_ pitchfork mold (gen:1)
+105       \_ pitchfork worker[0] (gen:1)
+102       \_ pitchfork worker[1] (gen:0)
+103       \_ pitchfork worker[2] (gen:0)
+104       \_ pitchfork worker[3] (gen:0)
+```
+
+```
+PID   COMMAND
+100   \_ pitchfork master
+101       \_ pitchfork mold (gen:1)
+105       \_ pitchfork worker[0] (gen:1)
+106       \_ pitchfork worker[1] (gen:1)
+103       \_ pitchfork worker[2] (gen:0)
+104       \_ pitchfork worker[3] (gen:0)
+```
+
+etc.
+
+### Forking Sibling Processes 
+
+Normally on unix systems, when calling `fork(2)`, the newly created process is a child of the original one, so forking from the mold should create
+a process tree such as:
+
+```
+PID   COMMAND
+100   \_ pitchfork master
+101       \_ pitchfork mold (gen:1)
+105          \_ pitchfork worker[0] (gen:1)
+```
+
+However the `pitchfork` master process registers itself as a "child subreaper" via [`PR_SET_CHILD_SUBREAPER`](https://man7.org/linux/man-pages/man2/prctl.2.html).
+This means any descendant process that is orphaned will be re-parented as a child of the master rather than a child of the init process (pid 1).
+
+With this in mind, the mold fork twice to create an orphaned process that will get re-attached to the master, effectively forking a sibling rather than a child.
+The need for `PR_SET_CHILD_SUBREAPER` is the main reason why reforking is only available on Linux. 

data/docs/SIGNALS.md

@@ -0,0 +1,38 @@
+## Signal handling
+
+In general, signals need only be sent to the master process. However,
+the signals Pitchfork uses internally to communicate with the worker
+processes are documented here as well.
+
+### Master Process
+
+* `INT/TERM` - quick shutdown, kills all workers immediately
+
+* `QUIT` - graceful shutdown, waits for workers to finish their
+  current request before finishing.
+
+* `USR2` - trigger a manual refork. A worker is promoted as
+  a new mold, and existing workers progressively replaced
+  by fresh ones.
+
+* `TTIN` - increment the number of worker processes by one
+
+* `TTOU` - decrement the number of worker processes by one
+
+### Worker Processes
+
+Note: the master uses a pipe to signal workers
+instead of `kill(2)` for most cases.  Using signals still (and works and
+remains supported for external tools/libraries), however.
+
+Sending signals directly to the worker processes should not normally be
+needed.  If the master process is running, any exited worker will be
+automatically respawned.
+
+* `INT/TERM` - Quick shutdown, immediately exit.
+  The master process will respawn a worker to replace this one.
+  Immediate shutdown is still triggered using kill(2) and not the
+  internal pipe as of unicorn 4.8
+
+* `QUIT` - Gracefully exit after finishing the current request.
+  The master process will respawn a worker to replace this one.

data/docs/TUNING.md

@@ -0,0 +1,106 @@
+# Tuning pitchfork
+
+unicorn performance is generally as good as a (mostly) Ruby web server
+can provide. Most often the performance bottleneck is in the web
+application running on Pitchfork rather than Pitchfork itself.
+
+## pitchfork Configuration
+
+See Pitchfork::Configurator for details on the config file format.
+`worker_processes` is the most-commonly needed tuning parameter.
+
+### Pitchfork::Configurator#worker_processes
+
+* `worker_processes` should be scaled to the number of processes your
+  backend system(s) can support. DO NOT scale it to the number of
+  external network clients your application expects to be serving.
+  unicorn is NOT for serving slow clients, that is the job of nginx.
+
+* `worker_processes` should be *at* *least* the number of CPU cores on
+  a dedicated server (unless you do not have enough memory).
+  If your application has occasionally slow responses that are /not/
+  CPU-intensive, you may increase this to workaround those inefficiencies.
+
+* `Etc.nprocessors` may be used to determine the number of CPU cores present.
+
+* Never, ever, increase `worker_processes` to the point where the system
+  runs out of physical memory and hits swap. Production servers should
+  never see swap activity.
+
+* Bigger is better. The more `worker_processes` you run, the more you'll
+  benefit from Copy-on-Write. If your application use 1GiB of memory after boot,
+  running `10` worker process, the relative memory usage per worker will only be
+  `~100MiB`, whereas if you only run `5` worker processes, there relative usage will be
+  `~200MiB`.
+  So if you can chose your hardware, it's preferable to use a smaller number
+  of bigger servers rather than a large number of smaller servers.
+  The same applies for containers, it's preferable to run a smaller number of larger containers.
+
+### Pitchfork::Configurator#refork_after
+
+* Reforking allows to share again memory pages that have been written into.
+
+* In general, the main source of shared memory pages invalidation in Ruby
+  is inline caches and JITed code. This means that calling a method for the
+  first time tend to degrade Copy-on-Write performance, and that over time
+  as more and more codepaths get executed at least once, less and less memory
+  is shared until it stabilize as most codepaths have been warmed up.
+
+* This is why automatic reforking is based on the number of processed requests.
+  You want to refork relatively frequently when the `pitchfork` server is fresh,
+  and then less and less frequently over time.
+
+### Pitchfork::Configurator#listen Options
+
+* Setting a very low value for the :backlog parameter in "listen"
+  directives can allow failover to happen more quickly if your
+  cluster is configured for it.
+
+* If you're doing extremely simple benchmarks and getting connection
+  errors under high request rates, increasing your :backlog parameter
+  above the already-generous default of 1024 can help avoid connection
+  errors.  Keep in mind this is not recommended for real traffic if
+  you have another machine to failover to (see above).
+
+* :rcvbuf and :sndbuf parameters generally do not need to be set for TCP
+  listeners under Linux 2.6 because auto-tuning is enabled.  UNIX domain
+  sockets do not have auto-tuning buffer sizes; so increasing those will
+  allow syscalls and task switches to be saved for larger requests
+  and responses.  If your app only generates small responses or expects
+  small requests, you may shrink the buffer sizes to save memory, too.
+
+* Having socket buffers too large can also be detrimental or have
+  little effect.  Huge buffers can put more pressure on the allocator
+  and may also thrash CPU caches, cancelling out performance gains
+  one would normally expect.
+
+* UNIX domain sockets are slightly faster than TCP sockets, but only
+  work if nginx is on the same machine.
+
+## Kernel Parameters (Linux sysctl and sysfs)
+
+WARNING: Do not change system parameters unless you know what you're doing!
+
+* net.core.rmem_max and net.core.wmem_max can increase the allowed
+  size of :rcvbuf and :sndbuf respectively. This is mostly only useful
+  for UNIX domain sockets which do not have auto-tuning buffer sizes.
+
+* For load testing/benchmarking with UNIX domain sockets, you should
+  consider increasing net.core.somaxconn or else nginx will start
+  failing to connect under heavy load.  You may also consider setting
+  a higher :backlog to listen on as noted earlier.
+
+* If you're running out of local ports, consider lowering
+  net.ipv4.tcp_fin_timeout to 20-30 (default: 60 seconds).  Also
+  consider widening the usable port range by changing
+  net.ipv4.ip_local_port_range.
+
+* Setting net.ipv4.tcp_timestamps=1 will also allow setting
+  net.ipv4.tcp_tw_reuse=1 and net.ipv4.tcp_tw_recycle=1, which along
+  with the above settings can slow down port exhaustion.  Not all
+  networks are compatible with these settings, check with your friendly
+  network administrator before changing these.
+
+* Increasing the MTU size can reduce framing overhead for larger
+  transfers.  One often-overlooked detail is that the loopback
+  device (usually "lo") can have its MTU increased, too.

data/examples/constant_caches.ru

@@ -0,0 +1,43 @@
+require "pitchfork/mem_info"
+
+module App
+  CONST_NUM = Integer(ENV.fetch("NUM", 100_000))
+
+  CONST_NUM.times do |i|
+    class_eval(<<~RUBY, __FILE__, __LINE__ + 1)
+      Const#{i} = Module.new
+
+      def self.lookup_#{i}
+        Const#{i}
+      end
+    RUBY
+  end
+
+  class_eval(<<~RUBY, __FILE__, __LINE__ + 1)
+    def self.warmup
+      #{CONST_NUM.times.map { |i| "lookup_#{i}"}.join("\n")}
+    end
+  RUBY
+end
+
+run lambda { |env|
+  parent_meminfo = Pitchfork::MemInfo.new(Process.ppid)
+  siblings_pids = File.read("/proc/#{Process.ppid}/task/#{Process.ppid}/children").split
+  siblings = siblings_pids.map do |pid|
+    Pitchfork::MemInfo.new(pid)
+  rescue Errno::ENOENT, Errno::ESRCH # The process just died
+    nil
+  end.compact
+
+  total_pss = parent_meminfo.pss + siblings.map(&:pss).sum
+  self_info = Pitchfork::MemInfo.new(Process.pid)
+
+  body = <<~EOS
+    Single Worker Memory Usage: #{(self_info.pss / 1024.0).round(1)} MiB
+    Total Cluster Memory Usage: #{(total_pss / 1024.0).round(1)} MiB
+  EOS
+
+  App.warmup
+
+  [ 200, { 'content-type' => 'text/plain' }, [ body ] ]
+}

data/examples/echo.ru

@@ -0,0 +1,25 @@
+# Example application that echoes read data back to the HTTP client.
+# This emulates the old echo protocol people used to run.
+#
+# An example of using this in a client would be to run:
+#   curl --no-buffer -T- http://host:port/
+#
+# Then type random stuff in your terminal to watch it get echoed back!
+
+class EchoBody < Struct.new(:input)
+
+  def each(&block)
+    while buf = input.read(4096)
+      yield buf
+    end
+    self
+  end
+
+end
+
+use Rack::Chunked
+run lambda { |env|
+  /\A100-continue\z/i =~ env['HTTP_EXPECT'] and return [100, {}, []]
+  [ 200, { 'content-type' => 'application/octet-stream' },
+    EchoBody.new(env['rack.input']) ]
+}

data/examples/hello.ru

@@ -0,0 +1,5 @@
+run lambda { |env|
+  /\A100-continue\z/i =~ env['HTTP_EXPECT'] and return [100, {}, []]
+  body = "Hello World!\n"
+  [ 200, { 'content-type' => 'text/plain', 'content-length' => body.bytesize.to_s }, [ body ] ]
+}

data/examples/nginx.conf

@@ -0,0 +1,156 @@
+# This is example contains the bare mininum to get nginx going with
+# unicorn servers.  Generally these configuration settings
+# are applicable to other HTTP application servers (and not just Ruby
+# ones), so if you have one working well for proxying another app
+# server, feel free to continue using it.
+#
+# The only setting we feel strongly about is the fail_timeout=0
+# directive in the "upstream" block.  max_fails=0 also has the same
+# effect as fail_timeout=0 for current versions of nginx and may be
+# used in its place.
+#
+# Users are strongly encouraged to refer to nginx documentation for more
+# details and search for other example configs.
+
+# you generally only need one nginx worker unless you're serving
+# large amounts of static files which require blocking disk reads
+worker_processes 1;
+
+# # drop privileges, root is needed on most systems for binding to port 80
+# # (or anything < 1024).  Capability-based security may be available for
+# # your system and worth checking out so you won't need to be root to
+# # start nginx to bind on 80
+user nobody nogroup; # for systems with a "nogroup"
+# user nobody nobody; # for systems with "nobody" as a group instead
+
+# Feel free to change all paths to suite your needs here, of course
+pid /path/to/nginx.pid;
+error_log /path/to/nginx.error.log;
+
+events {
+  worker_connections 1024; # increase if you have lots of clients
+  accept_mutex off; # "on" if nginx worker_processes > 1
+  # use epoll; # enable for Linux 2.6+
+  # use kqueue; # enable for FreeBSD, OSX
+}
+
+http {
+  # nginx will find this file in the config directory set at nginx build time
+  include mime.types;
+
+  # fallback in case we can't determine a type
+  default_type application/octet-stream;
+
+  # click tracking!
+  access_log /path/to/nginx.access.log combined;
+
+  # you generally want to serve static files with nginx since
+  # unicorn is not and will never be optimized for it
+  sendfile on;
+
+  tcp_nopush on; # off may be better for *some* Comet/long-poll stuff
+  tcp_nodelay off; # on may be better for some Comet/long-poll stuff
+
+  # we haven't checked to see if Rack::Deflate on the app server is
+  # faster or not than doing compression via nginx.  It's easier
+  # to configure it all in one place here for static files and also
+  # to disable gzip for clients who don't get gzip/deflate right.
+  # There are other gzip settings that may be needed used to deal with
+  # bad clients out there, see
+  # https://nginx.org/en/docs/http/ngx_http_gzip_module.html
+  gzip on;
+  gzip_http_version 1.0;
+  gzip_proxied any;
+  gzip_min_length 500;
+  gzip_disable "MSIE [1-6]\.";
+  gzip_types text/plain text/html text/xml text/css
+             text/comma-separated-values
+             text/javascript application/x-javascript
+             application/atom+xml;
+
+  # this can be any application server, not just unicorn
+  upstream app_server {
+    # fail_timeout=0 means we always retry an upstream even if it failed
+    # to return a good HTTP response (in case the unicorn master nukes a
+    # single worker for timing out).
+
+    # for UNIX domain socket setups:
+    server unix:/path/to/.unicorn.sock fail_timeout=0;
+
+    # for TCP setups, point these to your backend servers
+    # server 192.168.0.7:8080 fail_timeout=0;
+    # server 192.168.0.8:8080 fail_timeout=0;
+    # server 192.168.0.9:8080 fail_timeout=0;
+  }
+
+  server {
+    # enable one of the following if you're on Linux or FreeBSD
+    # listen 80 default deferred; # for Linux
+    # listen 80 default accept_filter=httpready; # for FreeBSD
+
+    # If you have IPv6, you'll likely want to have two separate listeners.
+    # One on IPv4 only (the default), and another on IPv6 only instead
+    # of a single dual-stack listener.  A dual-stack listener will make
+    # for ugly IPv4 addresses in $remote_addr (e.g ":ffff:10.0.0.1"
+    # instead of just "10.0.0.1") and potentially trigger bugs in
+    # some software.
+    # listen [::]:80 ipv6only=on; # deferred or accept_filter recommended
+
+    client_max_body_size 4G;
+    server_name _;
+
+    # ~2 seconds is often enough for most folks to parse HTML/CSS and
+    # retrieve needed images/icons/frames, connections are cheap in
+    # nginx so increasing this is generally safe...
+    keepalive_timeout 5;
+
+    # path for static files
+    root /path/to/app/current/public;
+
+    # Prefer to serve static files directly from nginx to avoid unnecessary
+    # data copies from the application server.
+    #
+    # try_files directive appeared in in nginx 0.7.27 and has stabilized
+    # over time.  Older versions of nginx (e.g. 0.6.x) requires
+    # "if (!-f $request_filename)" which was less efficient:
+    # https://yhbt.net/unicorn.git/tree/examples/nginx.conf?id=v3.3.1#n127
+    try_files $uri/index.html $uri.html $uri @app;
+
+    location @app {
+      # an HTTP header important enough to have its own Wikipedia entry:
+      #   https://en.wikipedia.org/wiki/X-Forwarded-For
+      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+
+      # enable this if you forward HTTPS traffic to unicorn,
+      # this helps Rack set the proper URL scheme for doing redirects:
+      # proxy_set_header X-Forwarded-Proto $scheme;
+
+      # pass the Host: header from the client right along so redirects
+      # can be set properly within the Rack application
+      proxy_set_header Host $http_host;
+
+      # we don't want nginx trying to do something clever with
+      # redirects, we set the Host: header above already.
+      proxy_redirect off;
+
+      # It's also safe to set if you're using only serving fast clients
+      # with unicorn + nginx, but not slow clients.  You normally want
+      # nginx to buffer responses to slow clients, even with Rails 3.1
+      # streaming because otherwise a slow client can become a bottleneck
+      # of unicorn.
+      #
+      # The Rack application may also set "X-Accel-Buffering (yes|no)"
+      # in the response headers do disable/enable buffering on a
+      # per-response basis.
+      # proxy_buffering off;
+
+      proxy_pass http://app_server;
+    }
+
+    # Rails error pages
+    error_page 500 502 503 504 /500.html;
+    location = /500.html {
+      root /path/to/app/current/public;
+    }
+  }
+}

data/examples/pitchfork.conf.minimal.rb

@@ -0,0 +1,5 @@
+# Minimal sample configuration file for Pitchfork
+
+listen 2007 # by default Pitchfork listens on port 8080
+worker_processes 4 # this should be >= nr_cpus
+refork_after [50, 100, 1000]

data/examples/pitchfork.conf.rb

@@ -0,0 +1,77 @@
+# Sample verbose configuration file for Pitchfork
+
+# Use at least one worker per core if you're on a dedicated server,
+# more will usually help for _short_ waits on databases/caches.
+worker_processes 4
+
+# listen on both a Unix domain socket and a TCP port,
+# we use a shorter backlog for quicker failover when busy
+listen "/path/to/.pitchfork.sock", :backlog => 64
+listen 8080, :tcp_nopush => true
+
+# nuke workers after 30 seconds instead of 60 seconds (the default)
+timeout 30
+
+# Enable this flag to have pitchfork test client connections by writing the
+# beginning of the HTTP headers before calling the application.  This
+# prevents calling the application for connections that have disconnected
+# while queued.  This is only guaranteed to detect clients on the same
+# host pitchfork runs on, and unlikely to detect disconnects even on a
+# fast LAN.
+check_client_connection false
+
+# local variable to guard against running a hook multiple times
+run_once = true
+
+before_fork do |server, worker|
+  # the following is highly recommended for Rails
+  # as there's no need for the master process to hold a connection
+  defined?(ActiveRecord::Base) and
+    ActiveRecord::Base.connection.disconnect!
+
+  # Occasionally, it may be necessary to run non-idempotent code in the
+  # master before forking.  Keep in mind the above disconnect! example
+  # is idempotent and does not need a guard.
+  if run_once
+    # do_something_once_here ...
+    run_once = false # prevent from firing again
+  end
+
+  # The following is only recommended for memory/DB-constrained
+  # installations.  It is not needed if your system can house
+  # twice as many worker_processes as you have configured.
+  #
+  # # This allows a new master process to incrementally
+  # # phase out the old master process with SIGTTOU to avoid a
+  # # thundering herd
+  # # when doing a transparent upgrade.  The last worker spawned
+  # # will then kill off the old master process with a SIGQUIT.
+  # old_pid = "#{server.config[:pid]}.oldbin"
+  # if old_pid != server.pid
+  #   begin
+  #     sig = (worker.nr + 1) >= server.worker_processes ? :QUIT : :TTOU
+  #     Process.kill(sig, File.read(old_pid).to_i)
+  #   rescue Errno::ENOENT, Errno::ESRCH
+  #   end
+  # end
+  #
+  # Throttle the master from forking too quickly by sleeping.  Due
+  # to the implementation of standard Unix signal handlers, this
+  # helps (but does not completely) prevent identical, repeated signals
+  # from being lost when the receiving process is busy.
+  # sleep 1
+end
+
+after_fork do |server, worker|
+  # per-process listener ports for debugging/admin/migrations
+  # addr = "127.0.0.1:#{9293 + worker.nr}"
+  # server.listen(addr, :tries => -1, :delay => 5, :tcp_nopush => true)
+
+  # the following is *required* for Rails
+  defined?(ActiveRecord::Base) and
+    ActiveRecord::Base.establish_connection
+
+  # You may also want to check and
+  # restart any other shared sockets/descriptors such as Memcached,
+  # and Redis.
+end

data/examples/unicorn.socket

@@ -0,0 +1,11 @@
+# ==> /etc/systemd/system/unicorn.socket <==
+[Unit]
+Description = unicorn sockets
+
+[Socket]
+ListenStream = 127.0.0.1:8080
+ListenStream = /tmp/path/to/.unicorn.sock
+Service = unicorn@1.service
+
+[Install]
+WantedBy = sockets.target