pitchfork 0.1.0

data/Gemfile

@@ -0,0 +1,9 @@
+# frozen_string_literal: true
+
+source "https://rubygems.org"
+
+gem 'minitest'
+gem 'rake'
+gem 'rake-compiler'
+
+gemspec

data/Gemfile.lock

@@ -0,0 +1,30 @@
+PATH
+  remote: .
+  specs:
+    pitchfork (0.1.0)
+      rack (>= 2.0)
+      raindrops (~> 0.7)
+
+GEM
+  remote: https://rubygems.org/
+  specs:
+    minitest (5.16.3)
+    rack (3.0.0)
+    raindrops (0.20.0)
+    rake (13.0.6)
+    rake-compiler (1.2.0)
+      rake
+
+PLATFORMS
+  aarch64-linux
+  arm64-darwin-21
+  x86_64-linux
+
+DEPENDENCIES
+  minitest
+  pitchfork!
+  rake
+  rake-compiler
+
+BUNDLED WITH
+   2.3.22

data/LICENSE

@@ -0,0 +1,67 @@
+Unicorn is copyrighted free software by all contributors, see logs in
+revision control for names and email addresses of all of them.
+
+You can redistribute it and/or modify it under either the terms of the
+GNU General Public License (GPL) as published by the Free Software
+Foundation (FSF), either version 2 of the License, or (at your option)
+any later version.  We currently prefer the GPLv3 or later for
+derivative works, but the GPLv2 is fine.
+
+The complete texts of the GPLv2 and GPLv3 are below:
+GPLv2 - https://www.gnu.org/licenses/gpl-2.0.txt
+GPLv3 - https://www.gnu.org/licenses/gpl-3.0.txt
+
+You may (against our _preference_) also use the Ruby 1.8 license terms
+which we inherited from the original Mongrel project when we forked it:
+
+=== Ruby 1.8-specific terms (if you're not using the GPL)
+
+  1. You may make and give away verbatim copies of the source form of the
+     software without restriction, provided that you duplicate all of the
+     original copyright notices and associated disclaimers.
+
+  2. You may modify your copy of the software in any way, provided that
+     you do at least ONE of the following:
+
+       a) place your modifications in the Public Domain or otherwise make them
+       Freely Available, such as by posting said modifications to Usenet or an
+       equivalent medium, or by allowing the author to include your
+       modifications in the software.
+
+       b) use the modified software only within your corporation or
+          organization.
+
+       c) rename any non-standard executables so the names do not conflict with
+       standard executables, which must also be provided.
+
+       d) make other distribution arrangements with the author.
+
+  3. You may distribute the software in object code or executable
+     form, provided that you do at least ONE of the following:
+
+       a) distribute the executables and library files of the software,
+       together with instructions (in the manual page or equivalent) on where
+       to get the original distribution.
+
+       b) accompany the distribution with the machine-readable source of the
+       software.
+
+       c) give non-standard executables non-standard names, with
+          instructions on where to get the original software distribution.
+
+       d) make other distribution arrangements with the author.
+
+  4. You may modify and include the part of the software into any other
+     software (possibly commercial).  But some files in the distribution
+     are not written by the author, so that they are not under this terms.
+
+  5. The scripts and library files supplied as input to or produced as
+     output from the software do not automatically fall under the
+     copyright of the software, but belong to whomever generated them,
+     and may be sold commercially, and may be aggregated with this
+     software.
+
+  6. THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR
+     IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED
+     WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+     PURPOSE.

data/README.md

@@ -0,0 +1,123 @@
+# pitchfork: Rack HTTP server for shared-nothing architecture
+
+`pitchfork` is a preforking HTTP server for Rack applications designed
+to minimize memory usage by maximizing Copy-on-Write performance.
+
+Like [`unicorn`](https://yhbt.net/unicorn/README.html) (which `pitchfork` is a derivative of), is it designed to
+only serve fast clients on low-latency, high-bandwidth connections and take
+advantage of features in Unix/Unix-like kernels. Slow clients should
+only be served by placing a reverse proxy capable of fully buffering
+both the request and response in between `pitchfork` and slow clients.
+
+## Disclaimer
+
+Until this notice is removed from the README, `pitchfork` should be
+considered experimental. As such it is not encouraged to run it in
+production just yet unless you feel capable of debugging yourself
+any issue that may arise.
+
+## Features
+
+* Designed for Rack, Linux, fast clients, and ease-of-debugging. We
+  cut out everything that is better supported by the operating system,
+  [nginx](https://nginx.org/) or [Rack](https://rack.github.io/).
+
+* Shared-Nothing architecture: workers all run within their own isolated
+  address space and only serve one client at a time for maximum performance
+  and robustness. Concurrent requests don't need to compete for the GVL,
+  or impact each others latency when triggering garbage collection.
+  It also does not care if your application is thread-safe or not.
+
+* Reforking: `pitchfork` can be configured to periodically promote a warmed up worker
+  as the new template from which workers are forked. This dramatically improves
+  the proportion of shared memory, making processes use only marginally more
+  memory than threads would.
+
+* Compatible with Ruby 2.5.0 and later.
+
+* Process management: `pitchfork` will reap and restart workers that
+  die from broken apps. There is no need to manage multiple processes
+  or ports yourself. `pitchfork` can spawn and manage any number of
+  worker processes you choose to scale to your backend.
+
+* Load balancing is done entirely by the operating system kernel.
+  Requests never pile up behind a busy worker process.
+
+## Requirements
+
+Ruby(MRI) Version 2.5 and above.
+
+`pitchfork` can be used on any Unix-like system, however the reforking
+feature requires `PR_SET_CHILD_SUBREAPER` which is a Linux 3.4 (May 2012) feature.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem "pitchfork"
+```
+
+And then execute:
+
+```bash
+$ bundle install
+```
+
+Or install it yourself as:
+
+```
+$ gem install pitchfork
+```
+
+## Usage
+
+### Rack
+
+In your application root directory, run:
+
+```bash
+$ bundle exec pitchfork
+```
+
+`pitchfork` will bind to all interfaces on TCP port 8080 by default.
+You may use the `--listen` switch to bind to a different
+address:port or a UNIX socket.
+
+### Configuration File(s)
+
+`pitchfork` will look for the config.ru file used by rackup in APP_ROOT.
+
+For deployments, it can use a config file for pitchfork-specific options
+specified by the `--config-file/-c` command-line switch.
+See the [configuration documentation](docs/CONFIGURATION.md) for the syntax
+of the pitchfork-specific options.
+
+The default settings are designed for maximum out-of-the-box
+compatibility with existing applications.
+
+Most command-line options for other Rack applications (above) are also
+supported.  Run `pitchfork -h` to see command-line options.
+
+## License
+
+pitchfork is copyright 2022 Shopify Inc and all contributors.
+It is based on Unicorn 6.1.0.
+
+Unicorn is copyright 2009-2018 by all contributors (see logs in git).
+It is based on Mongrel 1.1.5.
+Mongrel is copyright 2007 Zed A. Shaw and contributors.
+
+pitchfork is licensed under (your choice) of the GPLv2 or later
+(GPLv3+ preferred), or Ruby (1.8)-specific terms.
+See the included LICENSE file for details.
+
+## Thanks
+
+Thanks to Eric Wong and all Unicorn and Mongrel contributors over the years.
+Pitchfork would have been much harder to implement otherwise.
+
+Thanks to Will Jordan who implemented Puma's "fork worker" experimental feature
+which have been a significant inspiration for Pitchfork.
+
+Thanks to Peter Bui for letting us use the `pitchfork` name on Rubygems.

data/Rakefile

@@ -0,0 +1,72 @@
+# frozen_string_literal: true
+require "bundler/gem_tasks"
+require "rake/testtask"
+require "rake/extensiontask"
+
+Rake::TestTask.new("test:unit") do |t|
+  t.libs << "test"
+  t.libs << "lib"
+  t.test_files = FileList["test/unit/**/test_*.rb"]
+  t.options = '-v' if ENV['CI'] || ENV['VERBOSE']
+  t.warning = true
+end
+
+namespace :test do
+  # It's not so much that these tests are slow, but they tend to fork
+  # and/or register signal handlers, so they if something goes wrong
+  # they are likely to get stuck forever.
+  # The unicorn test suite has historically ran them in individual process
+  # so we continue to do that.
+  task slow: :compile do
+    tests = Dir["test/slow/**/*.rb"].flat_map do |test_file|
+      File.read(test_file).scan(/def (test_\w+)/).map do |test|
+        [test_file] + test
+      end
+    end
+    tests.each do |file, test|
+      sh "ruby", "-Ilib:test", file, "-n", test, "-v"
+    end
+  end
+
+  # Unicorn had its own POSIX-shell based testing framework.
+  # It's quite hard to work with and it would be good to convert all this
+  # to Ruby integration tests, but while pitchfork is a moving target it's
+  # preferable to edit the test suite as little as possible.
+  task integration: :compile do
+    File.write("test/integration/random_blob", File.read("/dev/random", 1_000_000))
+    lib = File.expand_path("lib", __dir__)
+    path = "#{File.expand_path("exe", __dir__)}:#{ENV["PATH"]}"
+    old_path = ENV["PATH"]
+    ENV["PATH"] = "#{path}:#{old_path}"
+    begin
+      Dir.chdir("test/integration") do
+        Dir["t[0-9]*.sh"].each do |integration_test|
+          sh("rm", "-rf", "trash")
+          sh("mkdir", "trash")
+          command = ["sh", integration_test]
+          command << "-v" if ENV["VERBOSE"] || ENV["CI"]
+          sh(*command)
+        end
+      end
+    ensure
+      ENV["PATH"] = old_path
+    end
+  end
+end
+
+Rake::ExtensionTask.new("pitchfork_http") do |ext|
+  ext.ext_dir = 'ext/pitchfork_http'
+  ext.lib_dir = 'lib/pitchfork'
+end
+
+task :ragel do
+  Dir.chdir(File.expand_path("ext/pitchfork_http", __dir__)) do
+    puts "* compiling pitchfork_http.rl"
+    cmd = ["ragel", "-G2", "pitchfork_http.rl", "-o", "pitchfork_http.c"]
+    system(*cmd) or raise "== #{cmd.join(' ')} failed =="
+  end
+end
+
+task test: %i(test:unit test:slow test:integration)
+
+task default: %i(ragel compile test)

data/docs/Application_Timeouts.md

@@ -0,0 +1,74 @@
+# Application Timeouts
+
+This article focuses on _application_ setup for Rack applications, but
+can be expanded to all applications that connect to external resources
+and expect short response times.
+
+This article is not specific to `pitchfork`, but exists to discourage
+the overuse of the built-in `timeout` directive in `pitchfork`.
+
+## ALL External Resources Are Considered Unreliable
+
+Network reliability can _never_ be guaranteed.  Network failures cannot
+be detected reliably by the client (Rack application) in a reasonable
+timeframe, not even on a LAN.
+
+Thus, application authors must configure timeouts when interacting with
+external resources.
+
+Most database adapters allow configurable timeouts.
+
+`Net::HTTP` and `Net::SMTP` in the Ruby standard library allow
+configurable timeouts.
+
+Even for things as fast as [memcached](https://memcached.org/),
+[dalli](https://rubygems.org/gems/dalli) and other memcached clients,
+all offer configurable timeouts.
+
+Consult the relevant documentation for the libraries you use on
+how to configure these timeouts.
+
+## Timeout module in the Ruby standard library
+
+Ruby offers a Timeout module in its standard library.  It has several
+caveats and is not always reliable:
+
+* /Some/ Ruby C extensions are not interrupted/timed-out gracefully by
+  this module (report these bugs to extension authors, please) but
+  pure-Ruby components should be.
+
+* `Timeout` uses [`Thread#raise` which most code don't and probably can't
+  handle properly](https://www.mikeperham.com/2015/05/08/timeout-rubys-most-dangerous-api/).
+  A process in which a `Timeout.timeout` block expired
+  should be considered corrupted and should exit as soon as possible.
+
+* Long-running tasks may run inside `ensure' clauses after timeout
+  fires, causing the timeout to be ineffective.
+
+The Timeout module is a second-to-last-resort solution, timeouts using
+`IO.select` (or similar) are more reliable. If you depend on libraries
+that do not offer timeouts when connecting to external resources, kindly
+ask those library authors to provide configurable timeouts.
+
+### A Note About Filesystems
+
+Most operations to regular files on POSIX filesystems are NOT
+interruptible. Thus, the "timeout" module in the Ruby standard library
+can not reliably timeout systems with massive amounts of iowait.
+
+If your app relies on the filesystem, ensure all the data your
+application works with is small enough to fit in the kernel page cache.
+Otherwise increase the amount of physical memory you have to match, or
+employ a fast, low-latency storage system (solid state).
+
+Volumes mounted over NFS (and thus a potentially unreliable network)
+must be mounted with timeouts and applications must be prepared to
+handle network/server failures.
+
+## The Last Line Of Defense
+
+The `timeout` mechanism in pitchfork is an extreme solution that should
+be avoided whenever possible.
+It will help preserve the platform if your application or a dependency
+has a bug that cause it to either get stuck or two slow, but it is not a
+solution to such bugs, merely a mitigation.