pitchfork 0.1.0

data/docs/CONFIGURATION.md

@@ -0,0 +1,388 @@
+# Configuration
+
+Most of Pitchfork configuration is directly inherited from Unicorn, however several
+options have been removed, and a few added.
+
+## Basic configurations
+
+### `worker_processes`
+
+```ruby
+worker_processes 16
+```
+
+Sets the number of desired worker processes.
+Each worker process will serve exactly one client at a time.
+
+### `listen`
+
+By default pitchfork listen to port 8080.
+
+```ruby
+listen 2007
+listen "/path/to/.pitchfork.sock", backlog: 64
+listen 8080, tcp_nopush: true
+```
+
+Adds an address to the existing listener set. May be specified more
+than once. address may be an Integer port number for a TCP port, an
+`IP_ADDRESS:PORT` for TCP listeners or a pathname for UNIX domain sockets.
+
+```ruby
+listen 3000 # listen to port 3000 on all TCP interfaces
+listen "127.0.0.1:3000"  # listen to port 3000 on the loopback interface
+listen "/path/to/.pitchfork.sock" # listen on the given Unix domain socket
+listen "[::1]:3000" # listen to port 3000 on the IPv6 loopback interface
+```
+
+When using Unix domain sockets, be sure:
+1) the path matches the one used by nginx
+2) uses the same filesystem namespace as the nginx process
+For systemd users using PrivateTmp=true (for either nginx or pitchfork),
+this means Unix domain sockets must not be placed in /tmp
+
+The following options may be specified (but are generally not needed):
+
+- `backlog: number of clients`
+
+  This is the backlog of the listen() syscall.
+
+  Some operating systems allow negative values here to specify the
+  maximum allowable value.  In most cases, this number is only
+  recommendation and there are other OS-specific tunables and
+  variables that can affect this number.  See the listen(2)
+  syscall documentation of your OS for the exact semantics of
+  this.
+
+  If you are running pitchfork on multiple machines, lowering this number
+  can help your load balancer detect when a machine is overloaded
+  and give requests to a different machine.
+
+  Default: `1024`
+
+  Note: with the Linux kernel, the net.core.somaxconn sysctl defaults
+  to 128, capping this value to 128.  Raising the sysctl allows a
+  larger backlog (which may not be desirable with multiple,
+  load-balanced machines).
+
+- `rcvbuf: bytes, sndbuf: bytes`
+
+  Maximum receive and send buffer sizes (in bytes) of sockets.
+
+  These correspond to the SO_RCVBUF and SO_SNDBUF settings which
+  can be set via the setsockopt(2) syscall.  Some kernels
+  (e.g. Linux 2.4+) have intelligent auto-tuning mechanisms and
+  there is no need (and it is sometimes detrimental) to specify them.
+
+  See the socket API documentation of your operating system
+  to determine the exact semantics of these settings and
+  other operating system-specific knobs where they can be
+  specified.
+
+  Defaults: operating system defaults
+
+- `tcp_nodelay: false`
+
+  Enables Nagle's algorithm on TCP sockets if +false+.
+
+  Default: +true+ (Nagle's algorithm disabled)
+
+  Setting this to +false+ can help in situation where the network between
+  pitchfork and the reverse proxy may be congested. In most case it's not
+  necessary.
+
+  This has no effect on UNIX sockets.
+
+- `tcp_nopush: true or false`
+
+  Enables/disables TCP_CORK in Linux or TCP_NOPUSH in FreeBSD
+
+  This prevents partial TCP frames from being sent out and reduces
+  wakeups in nginx if it is on a different machine.
+  Since pitchfork is only designed for applications that send the response body
+  quickly without keepalive, sockets will always be flushed on close to prevent delays.
+
+  This has no effect on UNIX sockets.
+
+  Default: `false` (disabled)
+
+- `ipv6only: true or false`
+
+  This option makes IPv6-capable TCP listeners IPv6-only and unable
+  to receive IPv4 queries on dual-stack systems.
+  A separate IPv4-only listener is required if this is true.
+
+  Enabling this option for the IPv6-only listener and having a
+  separate IPv4 listener is recommended if you wish to support IPv6
+  on the same TCP port.  Otherwise, the value of `env["REMOTE_ADDR"]`
+  will appear as an ugly IPv4-mapped-IPv6 address for IPv4 clients
+  (e.g `:ffff:10.0.0.1` instead of just `10.0.0.1`).
+
+  Default: Operating-system dependent
+
+- `reuseport: true or false`
+
+  This enables multiple, independently-started pitchfork instances to
+  bind to the same port (as long as all the processes enable this).
+
+  This option must be used when pitchfork first binds the listen socket.
+
+  Note: there is a chance of connections being dropped if
+  one of the pitchfork instances is stopped while using this.
+
+  This is supported on *BSD systems and Linux 3.9 or later.
+
+  ref: https://lwn.net/Articles/542629/
+
+  Default: `false` (unset)
+
+- `umask: mode`
+
+  Sets the file mode creation mask for UNIX sockets.
+  If specified, this is usually in octal notation.
+
+  Typically UNIX domain sockets are created with more liberal
+  file permissions than the rest of the application.
+  By default, we create UNIX domain sockets to be readable and writable by
+  all local users to give them the same accessibility as locally-bound TCP listeners.
+
+  This has no effect on TCP listeners.
+
+  Default: `0000` (world-read/writable)
+
+- `tcp_defer_accept: Integer`
+
+  Defer `accept()` until data is ready (Linux-only)
+
+  For Linux 2.6.32 and later, this is the number of retransmits to
+  defer an `accept()` for if no data arrives, but the client will
+  eventually be accepted after the specified number of retransmits
+  regardless of whether data is ready.
+
+  For Linux before 2.6.32, this is a boolean option, and
+  accepts are _always_ deferred indefinitely if no data arrives.
+
+  Specifying `true` is synonymous for the default value(s) below,
+  and `false` or `nil` is synonymous for a value of zero.
+
+  A value of `1` is a good optimization for local networks and trusted clients.
+  There is no good reason to ever disable this with a +zero+ value with pitchfork.
+
+  Default: `1`
+
+### `timeout`
+
+```ruby
+timeout 10
+```
+
+Sets the timeout of worker processes to a number of seconds.
+Workers handling the request/app.call/response cycle taking longer than
+this time period will be forcibly killed (via `SIGKILL`).
+
+This timeout mecanism shouldn't be routinely relying on, and should
+instead be considered as a last line of defense in case you application
+is impacted by bugs causing unexpectedly slow response time, or fully stuck
+processes.
+
+Make sure to read the guide on [application timeouts](Application_Timeouts.md).
+
+This configuration defaults to a (too) generous 20 seconds, it is
+highly recommended to set a stricter one based on your application
+profile.
+
+This timeout is enforced by the master process itself and not subject
+to the scheduling limitations by the worker process.
+Due the low-complexity, low-overhead implementation, timeouts of less
+than 3.0 seconds can be considered inaccurate and unsafe.
+
+For running Pitchfork behind nginx, it is recommended to set
+"fail_timeout=0" for in your nginx configuration like this
+to have nginx always retry backends that may have had workers
+SIGKILL-ed due to timeouts.
+
+```
+   upstream pitchfork_backend {
+     # for UNIX domain socket setups:
+     server unix:/path/to/.pitchfork.sock fail_timeout=0;
+
+     # for TCP setups
+     server 192.168.0.7:8080 fail_timeout=0;
+     server 192.168.0.8:8080 fail_timeout=0;
+     server 192.168.0.9:8080 fail_timeout=0;
+   }
+```
+
+See https://nginx.org/en/docs/http/ngx_http_upstream_module.html
+for more details on nginx upstream configuration.
+
+### `logger`
+
+```ruby
+logger Logger.new("path/to/logs")
+```
+
+Replace the default logger by the provided one.
+The passed logger must respond to the standard Ruby Logger interface.
+The default Logger will log its output to STDERR.
+
+## Callbacks
+
+Because pitchfork several callbacks around the lifecycle of workers.
+It is often necessary to use these callbacks to close inherited connection after fork.
+
+Note that when reforking is available, the `pitchfork` master process won't load your application
+at all. As such for hooks executed in the master, you may need to explicitly load the parts of your
+application that are used in hooks.
+
+`pitchfork` also don't attempt to rescue hook errors. Raising from a worker hook will crash the worker,
+and raising from a master hook will bring the whole cluster down.
+
+### `before_fork`
+
+```ruby
+before_fork do |server, worker|
+  Database.disconnect!
+end
+```
+
+Called in the context of the parent or mold.
+For most protocols connections can be closed after fork, but some
+stateful protocols require to close connections before fork.
+
+That is the case for instance of many SQL databases protocols.
+
+### `after_fork`
+
+```ruby
+after_fork do |server, worker|
+  NetworkClient.disconnect!
+end
+```
+
+Called in the worker after forking. Generally used to close inherited connections
+or to restart backgrounds threads for libraries that don't do it automatically.
+
+
+### `after_worker_ready`
+
+Called by a worker process after it has been fully loaded, directly before it
+starts responding to requests:
+
+```ruby
+after_worker_ready do |server, worker|
+  server.logger.info("worker #{worker.nr} ready")
+end
+```
+
+### `after_worker_exit`
+
+Called in the master process after a worker exits.
+
+```ruby
+after_worker_exit do |server, worker, status|
+  # status is a Process::Status instance for the exited worker process
+  unless status.success?
+    server.logger.error("worker process failure: #{status.inspect}")
+  end
+end
+```
+
+## Reforking
+
+### `refork_after`
+
+```ruby
+refork_after [50, 100, 1000]
+```
+
+Sets a number of requests threshold for triggering an automatic refork.
+The limit is per-worker, for instance with `refork_after [50]` a refork is triggered
+once at least one worker processed `50` requests.
+
+Each element is a limit for the next generation. On the example above a new generation
+is triggered when a worker has processed 50 requests, then the second generation when
+a worker from the new generation processed an additional 100 requests and finally after another
+1000 requests.
+
+Generally speaking Copy-on-Write efficiency tend to degrade fast during the early requests,
+and then less and less frequently.
+
+As such you likely want to refork exponentially less and less over time.
+
+By default automatic reforking isn't enabled.
+
+Make sure to read the [fork safety guide](FORK_SAFETY.md) before enabling reforking.
+
+### `mold_selector`
+
+Sets the mold selector implementation.
+
+```ruby
+mold_selector do |server|
+  candidate = server.children.workers.sample # return an random worker
+  server.logger.info("worker=#{worker.nr} pid=#{worker.pid} selected as new mold")
+  candidate
+end
+```
+
+The has access to `server.children` a `Pitchfork::Children` instance.
+This object can be used to introspect the state of the cluster and select the most
+appropriate worker to be used as the new mold from which workers will be reforked.
+
+The default implementation selects the worker with the least
+amount of shared memory. This heuristic aim to select the most
+warmed up worker.
+
+This should be considered a very advanced API and it is discouraged
+to use it unless you are confident you have a clear understanding
+of pitchfork's architecture.
+
+## Rack Features
+
+### `default_middleware`
+
+Sets whether to add Pitchfork's default middlewares. Defaults to `true`.
+
+### `early_hints`
+
+Sets whether to enable the proposed early hints Rack API. Defaults to `false`.
+
+If enabled, Rails 5.2+ will automatically send a 103 Early Hint for all the `javascript_include_tag` and `stylesheet_link_tag`
+in your response. See: https://api.rubyonrails.org/v5.2/classes/ActionDispatch/Request.html#method-i-send_early_hints
+See also https://tools.ietf.org/html/rfc8297
+
+## Advanced Tuning Configurations
+
+Make sure to read the tuning guide before tweaking any of these.
+Also note that most of these options are inherited from Unicorn, so
+most guides on how to tune Unicorn likely apply here.
+
+### `rewindable_input`
+
+Toggles making `env["rack.input"]` rewindable.
+Disabling rewindability can improve performance by lowering I/O and memory usage for applications that accept uploads.
+Keep in mind that the Rack 1.x spec requires `env["rack.input"]` to be rewindable, but the Rack 2.x spec does not.
+
+`rewindable_input` defaults to `true` for compatibility.
+Setting it to `false` may be safe for applications and frameworks developed for Rack 2.x and later.
+
+### `client_body_buffer_size`
+
+The maximum size in bytes to buffer in memory before resorting to a temporary file.
+Default is `112` kilobytes.
+This option has no effect if `rewindable_input` is set to `false`.
+
+### `check_client_connection`
+
+When enabled, pitchfork will check the client connection by writing
+the beginning of the HTTP headers before calling the application.
+
+This will prevent calling the application for clients who have
+disconnected while their connection was queued.
+
+This only affects clients connecting over Unix domain sockets
+and TCP via loopback (`127.*.*.*`).
+It is unlikely to detect disconnects if the client is on a remote host (even on a fast LAN).
+
+This option cannot be used in conjunction with `tcp_nopush`.

data/docs/DESIGN.md

@@ -0,0 +1,86 @@
+## Design
+
+* Simplicity: Pitchfork is a traditional UNIX prefork web server.
+  No threads are used at all, this makes applications easier to debug
+  and fix.
+  
+* Resiliency: If something in goes catastrophically wrong and your application
+  is dead locked or somehow stuck, once the request timeout is reached the master
+  process will take care of sending `kill -9` to the affected worker and
+  spawn a new one to replace it.
+
+* Leverage Copy-on-Write: The only real disadvantage of prefork servers is
+  their increased memory usage. But thanks to reforking, `pitchfork` is able
+  to drastically improve Copy-on-Write performance, hence reduce memory usage
+  enough that it's no longer a concern.
+
+* The Ragel+C HTTP parser is taken from Mongrel.
+
+* All HTTP parsing and I/O is done much like Mongrel:
+    1. read/parse HTTP request headers in full
+    2. call Rack application
+    3. write HTTP response back to the client
+
+* Like Mongrel, neither keepalive nor pipelining are supported.
+  These aren't needed since Pitchfork is only designed to serve
+  fast, low-latency clients directly.  Do one thing, do it well;
+  let nginx handle slow clients.
+
+* Configuration is purely in Ruby. Ruby is less
+  ambiguous than YAML and lets lambdas for
+  before_fork/after_fork hooks be defined inline. An
+  optional, separate config_file may be used to modify supported
+  configuration changes.
+
+* One master process spawns and reaps worker processes.
+
+* The number of worker processes should be scaled to the number of
+  CPUs or memory you have. If you have an existing
+  Unicorn cluster on a single-threaded app, using the same amount of
+  processes should work. Let a full-HTTP-request-buffering reverse
+  proxy like nginx manage concurrency to thousands of slow clients for
+  you. Pitchfork scaling should only be concerned about limits of your
+  backend system(s).
+
+* Load balancing between worker processes is done by the OS kernel.
+  All workers share a common set of listener sockets and does
+  non-blocking accept() on them.  The kernel will decide which worker
+  process to give a socket to and workers will sleep if there is
+  nothing to accept().
+
+* Since non-blocking accept() is used, there can be a thundering
+  herd when an occasional client connects when application
+  *is not busy*.  The thundering herd problem should not affect
+  applications that are running all the time since worker processes
+  will only select()/accept() outside of the application dispatch.
+
+* Additionally, thundering herds are much smaller than with
+  configurations using existing prefork servers.  Process counts should
+  only be scaled to backend resources, _never_ to the number of expected
+  clients like is typical with blocking prefork servers.  So while we've
+  seen instances of popular prefork servers configured to run many
+  hundreds of worker processes, Pitchfork deployments are typically between
+  1 and 2 processes per-core.
+
+* Blocking I/O is used for clients. This allows a simpler code path
+  to be followed within the Ruby interpreter and fewer syscalls.
+
+* `SIGKILL` is used to terminate the timed-out workers from misbehaving apps
+  as reliably as possible on a UNIX system. The default timeout is a
+  generous 20 seconds.
+
+* The poor performance of select() on large FD sets is avoided
+  as few file descriptors are used in each worker.
+  There should be no gain from moving to highly scalable but
+  unportable event notification solutions for watching few
+  file descriptors.
+
+* If the master process dies unexpectedly for any reason,
+  workers will notice within :timeout/2 seconds and follow
+  the master to its death.
+
+* There is never any explicit real-time dependency or communication
+  between the worker processes nor to the master process.
+  Synchronization is handled entirely by the OS kernel and shared
+  resources are never accessed by the worker when it is servicing
+  a client.

data/docs/FORK_SAFETY.md

@@ -0,0 +1,80 @@
+# Fork Safety
+
+Because `pitchfork` is a preforking server, your application code and libraries
+must be fork safe.
+
+Generally code might be fork-unsafe for one of two reasons
+
+## Inherited Connection
+
+When a process is forked, any open file descriptor (sockets, files, pipes, etc)
+end up shared between the parent and child process. This is never what you
+want, so any code keeping persistent connections should close them either
+before or after the fork happens.
+
+`pitchfork` provide two callbacks in its configuration file to do so:
+
+```ruby
+# pitchfork.conf.rb
+
+before_fork do
+  Sequel::DATABASES.each(&:disconnect)
+end
+
+after_fork do
+  SomeLibary.connection.close
+end
+```
+
+The documentation of any database client or network library you use should be
+read with care to figure out how to disconnect it, and whether it is best to
+do it before or after fork.
+
+Since the most common Ruby application servers `Puma`, `Unicorn` and `Passenger`
+have forking at least as an option, the requirements are generally well documented.
+
+However what is novel with `Pitchfork`, is that processes can be forked more than once.
+So just because an application works fine with existing pre-fork servers doesn't necessarily
+mean it will work fine with `Pitchfork`.
+
+It's not uncommon for applications to not close connections after fork, but for it to go
+unnoticed because these connections are lazily created when the first request is handled.
+
+So if you enable reforking for the first time, you may discover some issues.
+
+Also note that rather than to expose a callback, some libraries take on them to detect
+that a fork happened, and automatically close inherited connections.
+
+## Background Threads
+
+When a process is forked, only the main threads will stay alive in the child process.
+So any libraries that spawn a background thread for periodical work may need to be notified
+that a fork happened and that it should restart its thread.
+
+Just like with connections, some libraries take on them to automatically restart their background
+thread when they detect a fork happened.
+
+# Refork Safety
+
+Some code might happen to work without issue in other forking servers such as Unicorn or Puma,
+but not work in Pitchfork when reforking is enabled.
+
+This is because it is not uncommon for network connections or background threads to only be
+initialized upon the first request. As such they're not inherited on the first fork.
+
+However when reforking is enabled, new processes as forked out of warmed up process, as such
+any lazily created connection is much more likely to have been created.
+
+As such, if you enable reforking for the first time, it is heavily recommended to first do it
+in some sort of staging environment, or on a small subset of production servers as to limit the
+impact of discovering such bug.
+
+## Known Incompatible Gems
+
+- [The `grpc` isn't fork safe](https://github.com/grpc/grpc/issues/8798) and doesn't provide any before or after fork callback to re-establish connection.
+  It can only be used in forking environment if the client is never used in the parent before fork.
+  If you application uses `grpc`, you shouldn't enable reforking.
+  But frankly, that gem is such a tire fire, you shouldn't use it regardless.
+  If you really have to consume a gRPC API, you can consider `grpc_kit` as a replacement.
+
+No other gem is known to be incompatible, but if you find one please open an issue to add it to the list.

data/docs/PHILOSOPHY.md

@@ -0,0 +1,90 @@
+# The Philosophy Behind pitchfork
+
+## Avoid Complexity
+
+Instead of attempting to be efficient at serving slow clients, pitchfork
+relies on a buffering reverse proxy to efficiently deal with slow
+clients.
+
+## Threads and Events Are Not Well Suited For Transactional Web Applications
+
+`pitchfork` uses a preforking worker model with blocking I/O.
+Our processing model is the antithesis of processing models using threads or
+non-blocking I/O with events or fibers.
+
+It is only meant to serve fast, transactional HTTP/1.1 applications.
+These applications rarely if ever spend more than half their time on IOs, and
+any remote API call made by the application should either have a strict SLA
+and timeout, or be deferred to a background job processing system.
+
+As such, when they are ran in a threaded or fiber processing model, they suffer
+from GVL contention and GC pauses, which hurts latency.
+
+`pitchfork` is not suited for all applications. `pitchfork` is optimized for
+applications that are CPU/memory/disk intensive and spend little time
+waiting on external resources (e.g. a database server or external API).
+
+WebSocket, Server-Sent Events and applications that mainly act as a light proxy
+to another service have radically different performance profiles and requirements,
+and shouldn't be handled by the same process. It's preferable to host these with
+a threaded or evented processing model (`falcon`, `puma`, etc).
+
+No processing model can efficiently handle both types of workload. Use
+the right tool with the right configuration for the right job.
+
+## Improved Performance Through Reverse Proxying
+
+By acting as a buffer to shield unicorn from slow I/O, a reverse proxy
+will inevitably incur overhead in the form of extra data copies.
+However, as I/O within a local network is fast (and faster still
+with local sockets), this overhead is negligible for the vast majority
+of HTTP requests and responses.
+
+The ideal reverse proxy complements the weaknesses of `pitchfork`.
+A reverse proxy for `pitchfork` should meet the following requirements:
+
+1. It should fully buffer all HTTP requests (and large responses).
+   Each request should be "corked" in the reverse proxy and sent
+   as fast as possible to the backend unicorn processes.  This is
+   the most important feature to look for when choosing a
+   reverse proxy for `pitchfork`.
+
+2. It should handle SSL/TLS termination. Requests should arrive
+   decrypted to `pitchfork`. Reverse proxy can do this much more
+   efficiently. If you don't trust your local network enough to
+   make unencrypted traffic go through it, you can have a reverse
+   proxy on the same server than `pitchfork` to handle decryption.
+
+3. It should handle HTTP/2 or HTTP/3 termination. Newer HTTP protocols
+   do not provide any feature or improvements that are useful or even desirable
+   for transactional HTTP applications. Your reverse proxy or load balancer
+   should handle the HTTP/2 or HTTP/3 protocol with the client, but forward
+   requests to `pitchfork` as HTTP/1.1.
+
+4. It should efficiently manage persistent connections (and
+   pipelining) to slow clients.
+
+5. It should not be "sticky". Even if the client has a persistent
+   connection, every request made as part of that persistent connection
+   should be load balanced individually.
+
+6. It should (optionally) serve static files. If you have static
+   files on your site (especially large ones), they are far more
+   efficiently served with as few data copies as possible (e.g. with
+   sendfile() to completely avoid copying the data to userspace).
+
+Suitable options include `nginx`, `caddy` and likely several others.
+
+## Leverage Copy-on-Write to reduce memory usage.
+
+One of the main advantages of threaded servers over preforking servers is their
+lower memory usage.
+
+However `pitchfork` solves this with its reforking feature. If enabled and properly configured
+it very significantly increase Copy-on-Write performance, closing the gap with threaded servers.
+
+## Assume Modern Deployment Methods
+
+Pitchfork assumes it is deployed using modern tools such as either containers or
+advanced init systems such as systemd. As such it doesn't provide classic daemon
+functionallty like pidfile management, log rediction and reopening, config reloading etc.