unicorn 0.1.0

data/.document

@@ -0,0 +1,12 @@
+README
+DESIGN
+CHANGELOG
+CONTRIBUTORS
+LICENSE
+SIGNALS
+TODO
+bin
+lib
+ext/**/*.c
+ext/**/*.rl
+ext/**/*.h

data/.gitignore

@@ -0,0 +1,12 @@
+*.bundle
+*.log
+*.so
+.DS_Store
+/.config
+/InstalledFiles
+/doc
+/local.mk
+/test/install-*
+ext/unicorn/http11/Makefile
+log/
+pkg/

data/CHANGELOG

@@ -0,0 +1,19 @@
+v0.1.0 - Unicorn - UNIX-only fork of Mongrel free of threading
+
+v2.0. (WIP) Rack support.
+
+v1.1.4. Fix camping handler. Correct treatment of @throttle parameter.
+
+v1.1.3. Fix security flaw of DirHandler; reported on mailing list.
+
+v1.1.2. Fix worker termination bug; fix JRuby 1.0.3 load order issue; fix require issue on systems without Rubygems.
+
+v1.1.1. Fix mongrel_rails restart bug; fix bug with Rack status codes.
+
+v1.1. Pure Ruby URIClassifier. More modular architecture. JRuby support. Move C URIClassifier into mongrel_experimental project.
+
+v1.0.4. Backport fixes for versioning inconsistency, mongrel_rails bug, and DirHandler bug.
+
+v1.0.3. Fix user-switching bug; make people upgrade to the latest from the RC.
+
+v1.0.2. Signed gem; many minor bugfixes and patches.

data/CONTRIBUTORS

@@ -0,0 +1,20 @@
+Unicorn would not be possible without Zed and all the contributors to Mongrel.
+
+Eric Wong
+Ezra Zygmuntowicz
+Zed A. Shaw
+Luis Lavena
+Wilson Bilkovich
+Why the Lucky Stiff
+Dan Kubb
+MenTaLguY
+Filipe Lautert
+Rick Olson
+Wayne E. Seguin
+Kirk Haines
+Bradley Taylor
+Matt Pelletier
+Ry Dahl
+Nick Sieger
+Evan Weaver
+Marc-André Cournoyer

data/DESIGN

@@ -0,0 +1,80 @@
+== Design
+
+* Simplicity: Unicorn is a traditional UNIX prefork web server.
+  No threads are used at all, this makes applications easier to debug
+  and fix.  When your application goes awry, a BOFH can just
+  "kill -9" the runaway worker process without worrying about tearing
+  all clients down, just one.  Only UNIX-like systems supporting
+  fork() and file descriptor inheritance are supported.
+
+* The Ragel->C HTTP parser is taken from Mongrel.  This is the
+  only non-Ruby part and there are no plans to add any more
+  non-Ruby components.
+
+* All HTTP protocol parsing and I/O is done just like Mongrel:
+    1. read/parse HTTP request in full
+    2. call Rack application
+    3. write HTTP response back to the client
+
+* Like Mongrel, neither keepalive nor pipelining are supported.
+  These aren't needed since Unicorn is only designed to serve
+  fast, low-latency clients directly.  Do one thing, do it well;
+  let nginx handle slow clients.
+
+* Configuration is purely in Ruby and eval().  Ruby is less
+  ambiguous than YAML and lets lambdas for
+  before_fork/after_fork/before_exec hooks be defined inline.  An
+  optional, separate config_file may be used to modify supported
+  configuration changes (and also gives you plenty of rope if you RTFS
+  :>)
+
+* One master process spawns and reaps worker processes.  The
+  Rack application itself is called only within the worker process (but
+  can be loaded within the master).  A copy-on-write friendly garbage
+  collector like Ruby Enterprise Edition can be used to minimize memory
+  usage along with the "preload_app true" directive.
+
+* The number of worker processes should be scaled to the number of
+  CPUs, memory or even spindles you have.  If you have an existing
+  Mongrel cluster, using the same amount of processes should work.
+  Let a full-HTTP-request-buffering reverse proxy like nginx manage
+  concurrency to thousands of slow clients for you.  Unicorn scaling
+  should only be concerned about limits of your backend system(s).
+
+* Load balancing between worker processes is done by the OS kernel.
+  All workers share a common set of listener sockets and does
+  non-blocking accept() on them.  The kernel will decide which worker
+  process to give a socket to and workers will sleep if there is
+  nothing to accept().
+
+* Since non-blocking accept() is used, there can be a thundering
+  herd when an occasional client connects when application
+  *is not busy*.  The thundering herd problem should not affect
+  applications that are running all the time since worker processes
+  will only select()/accept() outside of the application dispatch.
+
+* Blocking I/O is used for clients.  This allows a simpler code path
+  to be followed within the Ruby interpreter and fewer syscalls.
+  Applications that use threads should continue to work if Unicorn
+  is serving LAN or localhost clients.
+
+* Timeout implementation is done via fchmod(2) in each worker
+  on a shared file descriptor to update st_ctime on the inode.
+  Master process wakeups for checking on timeouts is throttled
+  one a second to minimize the performance impact and simplify
+  the code path within the worker.  Neither futimes(2) nor
+  pwrite(2)/pread(2) are supported by base MRI, nor are they as
+  portable on UNIX systems as fchmod(2).
+
+* SIGKILL is used to terminate the timed-out workers as reliably
+  as possible on a UNIX system.
+
+* The poor performance of select() on large FD sets is avoided
+  as few file descriptors are used in each worker.
+  There should be no gain from moving to highly scalable but
+  unportable event notification solutions for watching few
+  file descriptors.
+
+* If the master process dies unexpectedly for any reason,
+  workers will notice within :timeout/2 seconds and follow
+  the master to its death.

data/GNUmakefile

@@ -0,0 +1,111 @@
+# use GNU Make to run tests in parallel, and without depending on Rubygems
+all:: test
+ruby = ruby
+-include local.mk
+ruby_bin := $(shell which $(ruby))
+ifeq ($(DLEXT),) # "so" for Linux
+  DLEXT := $(shell $(ruby) -rrbconfig -e 'puts Config::CONFIG["DLEXT"]')
+endif
+ifeq ($(RUBY_VERSION),)
+  RUBY_VERSION := $(shell $(ruby) -e 'puts RUBY_VERSION')
+endif
+
+# dunno how to implement this as concisely in Ruby, and hell, I love awk
+awk_slow := awk '/def test_/{print FILENAME"--"$$2".n"}' 2>/dev/null
+
+slow_tests := test/unit/test_server.rb test/exec/test_exec.rb
+log_suffix = .$(RUBY_VERSION).log
+T := $(filter-out $(slow_tests),$(wildcard test/*/test*.rb))
+T_n := $(shell $(awk_slow) $(slow_tests))
+T_log := $(subst .rb,$(log_suffix),$(T))
+T_n_log := $(subst .n,$(log_suffix),$(T_n))
+test_prefix = $(CURDIR)/test/install-$(RUBY_VERSION)
+
+http11_deps := $(addprefix ext/unicorn/http11/, \
+                 ext_help.h http11.c http11_parser.c http11_parser.h \
+                 http11_parser.rl http11_parser_common.rl)
+inst_deps := $(wildcard bin/*) $(wildcard lib/*.rb) \
+  $(wildcard lib/*/*.rb) $(http11_deps)
+
+ext/unicorn/http11/http11_parser.c: ext/unicorn/http11/http11_parser.rl
+	cd $(@D) && ragel $(<F) -C -G2 -o $(@F)
+ext/unicorn/http11/Makefile: ext/unicorn/http11/extconf.rb
+	cd $(@D) && $(ruby) $(<F)
+ext/unicorn/http11/http11.$(DLEXT): $(http11_deps) ext/unicorn/http11/Makefile
+	$(MAKE) -C $(@D)
+lib/unicorn/http11.$(DLEXT): ext/unicorn/http11/http11.$(DLEXT)
+	@mkdir -p lib
+	install -m644 $< $@
+http11: lib/unicorn/http11.$(DLEXT)
+
+$(test_prefix)/.stamp: $(inst_deps)
+	$(MAKE) clean-http11
+	$(MAKE) install-test
+	> $@
+
+install-test:
+	mkdir -p $(test_prefix)/.ccache
+	tar c bin ext lib GNUmakefile | (cd $(test_prefix) && tar x)
+	$(MAKE) -C $(test_prefix) http11 shebang
+
+# this is only intended to be run within $(test_prefix)
+shebang: bin/unicorn
+	$(ruby) -i -p -e '$$_.gsub!(%r{^#!.*$$},"#!$(ruby_bin)")' $<
+
+t_log := $(T_log) $(T_n_log)
+test: $(T) $(T_n)
+	@cat $(t_log) | $(ruby) test/aggregate.rb
+	@$(RM) $(t_log)
+
+slow-tests: $(slow_tests)
+$(slow_tests):
+	@$(MAKE) $(shell $(awk_slow) $@)
+
+TEST_OPTS = -v
+run_test = @echo '*** $(arg) ***'; \
+  setsid $(ruby) $(arg) $(TEST_OPTS) >$(t) 2>&1 || \
+  (cat >&2 < $(t); exit 1)
+
+%.n: arg = $(subst .n,,$(subst --, -n ,$@))
+%.n: t = $(subst .n,$(log_suffix),$@)
+%.n: export PATH := $(test_prefix)/bin:$(PATH)
+%.n: export RUBYLIB := $(test_prefix)/lib:$(RUBYLIB)
+%.n: $(test_prefix)/.stamp
+	$(run_test)
+
+$(T): arg = $@
+$(T): t = $(subst .rb,$(log_suffix),$@)
+$(T): export PATH := $(test_prefix)/bin:$(PATH)
+$(T): export RUBYLIB := $(test_prefix)/lib:$(RUBYLIB)
+$(T): $(test_prefix)/.stamp
+	$(run_test)
+
+install: bin/unicorn
+	$(prep_setup_rb)
+	git diff --quiet $<
+	$(ruby) setup.rb all
+	git checkout $<
+	$(prep_setup_rb)
+
+clean-http11:
+	-$(MAKE) -C ext/unicorn/http11 clean
+	$(RM) ext/unicorn/http11/Makefile lib/unicorn/http11.$(DLEXT)
+
+setup_rb_files := .config InstalledFiles
+prep_setup_rb := @-$(RM) $(setup_rb_files);$(MAKE) -C ext/unicorn/http11 clean
+
+clean: clean-http11
+	$(RM) $(setup_rb_files)
+	$(RM) $(t_log)
+	$(RM) -r $(test_prefix)
+
+Manifest:
+	git ls-files > $@+
+	cmp $@+ $@ || mv $@+ $@
+	$(RM) -f $@+
+
+# using rdoc 2.4.1
+doc: .document
+	rdoc -Na -m README -t "$(shell sed -ne '1s/^= //p' README)"
+
+.PHONY: doc $(T) $(slow_tests) Manifest

data/LICENSE

@@ -0,0 +1,53 @@
+Unicorn Web Server (unicorn) is copyrighted free software by Eric Wong
+(normalperson@yhbt.net) and contributors. You can redistribute it
+and/or modify it under either the terms of the GPL2 or the conditions below:
+
+1. You may make and give away verbatim copies of the source form of the
+   software without restriction, provided that you duplicate all of the
+   original copyright notices and associated disclaimers.
+
+2. You may modify your copy of the software in any way, provided that
+   you do at least ONE of the following:
+
+     a) place your modifications in the Public Domain or otherwise make them
+     Freely Available, such as by posting said modifications to Usenet or an
+     equivalent medium, or by allowing the author to include your
+     modifications in the software.
+
+     b) use the modified software only within your corporation or
+        organization.
+
+     c) rename any non-standard executables so the names do not conflict with
+     standard executables, which must also be provided.
+
+     d) make other distribution arrangements with the author.
+
+3. You may distribute the software in object code or executable
+   form, provided that you do at least ONE of the following:
+
+     a) distribute the executables and library files of the software,
+     together with instructions (in the manual page or equivalent) on where
+     to get the original distribution.
+
+     b) accompany the distribution with the machine-readable source of the
+     software.
+
+     c) give non-standard executables non-standard names, with
+        instructions on where to get the original software distribution.
+
+     d) make other distribution arrangements with the author.
+
+4. You may modify and include the part of the software into any other
+   software (possibly commercial).  But some files in the distribution
+   are not written by the author, so that they are not under this terms.
+
+5. The scripts and library files supplied as input to or produced as
+   output from the software do not automatically fall under the
+   copyright of the software, but belong to whomever generated them,
+   and may be sold commercially, and may be aggregated with this
+   software.
+
+6. THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OR
+   IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED
+   WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+   PURPOSE.

data/Manifest

@@ -0,0 +1,41 @@
+.document
+.gitignore
+CHANGELOG
+CONTRIBUTORS
+DESIGN
+GNUmakefile
+LICENSE
+Manifest
+README
+Rakefile
+SIGNALS
+TODO
+bin/unicorn
+ext/unicorn/http11/ext_help.h
+ext/unicorn/http11/extconf.rb
+ext/unicorn/http11/http11.c
+ext/unicorn/http11/http11_parser.c
+ext/unicorn/http11/http11_parser.h
+ext/unicorn/http11/http11_parser.rl
+ext/unicorn/http11/http11_parser_common.rl
+lib/unicorn.rb
+lib/unicorn/configurator.rb
+lib/unicorn/const.rb
+lib/unicorn/http_request.rb
+lib/unicorn/http_response.rb
+lib/unicorn/socket.rb
+lib/unicorn/util.rb
+setup.rb
+test/aggregate.rb
+test/benchmark/previous.rb
+test/benchmark/simple.rb
+test/benchmark/utils.rb
+test/exec/README
+test/exec/test_exec.rb
+test/test_helper.rb
+test/tools/trickletest.rb
+test/unit/test_configurator.rb
+test/unit/test_http_parser.rb
+test/unit/test_response.rb
+test/unit/test_server.rb
+test/unit/test_upload.rb

data/README

@@ -0,0 +1,73 @@
+= Unicorn: UNIX + LAN/localhost-only fork of Mongrel
+
+Only run this behind a full-HTTP-request-buffering reverse proxy if
+you're serving slow clients.  That said, nginx is the only reverse
+proxy we know of that meets this requirement.
+
+== Features
+
+* process management: Unicorn will reap and restart workers that
+  die because of broken apps and there is no need to manage
+  multiple processes yourself.
+
+* does not care if your application is thread-safe or not, workers
+  all run within their own isolated address space and only serve one
+  client at a time...
+
+* able to listen on multiple interfaces, including UNIX sockets,
+  each worker process can also bind to a private port via the
+  after_fork hook for easy debugging.
+
+* supports all Rack applications
+
+* nginx-style binary re-execution without losing connections.
+  You can upgrade unicorn, your entire application, libraries
+  and even your Ruby interpreter as long as unicorn is
+  installed in the same path.
+
+* before_fork and after_fork hooks in case your application
+  has special needs when dealing with forked processes.
+
+* builtin log rotation via USR1 signal
+
+* Ruby 1.9-compatible (at least the test cases all pass :>)
+
+== License
+
+Unicorn is copyright 2009 Eric Wong and contributors.
+It is based on Mongrel:
+
+Mongrel is copyright 2007 Zed A. Shaw and contributors. It is licensed
+under the Ruby license and the GPL2. See the include LICENSE file for
+details.
+
+== Install
+
+The library consists of a C extension so you'll need a C compiler or at
+least a friend who can build it for you.
+
+Finally, the source includes a setup.rb for those who hate RubyGems.
+
+You can get the source via git via the following locations:
+
+  git://git.bogomips.org/unicorn.git
+
+  http://git.bogomips.org/unicorn.git
+
+== Usage
+
+Unicorn will look for the config.ru file used by rackup in APP_ROOT.
+Optionally, it can use a config file specified by the --config-file/-c
+command-line switch.
+
+Unicorn should be capable of running all Rack applications.  Since this
+is a preforking webserver, you do not have to worry about thread-safety
+of your application or libraries. However, your Rack application may use
+threads internally (and should even be able to continue running threads
+after the request is complete).
+
+== Contact
+
+Newsgroup and mailing list coming, or it'll be a part of the Mongrel project...
+
+Email Eric Wong at normalperson@yhbt.net for now.

data/Rakefile

@@ -0,0 +1,37 @@
+
+require 'rubygems'
+require 'echoe'
+
+Echoe.new("unicorn") do |p|
+  p.summary = "A small fast HTTP library and server for Rack applications."
+  p.author = "Eric Wong"
+  p.email = "normalperson@yhbt.net"
+  p.clean_pattern = ['ext/unicorn/http11/*.{bundle,so,o,obj,pdb,lib,def,exp}',
+                     'lib/*.{bundle,so,o,obj,pdb,lib,def,exp}',
+                     'ext/unicorn/http11/Makefile',
+                     'pkg', 'lib/*.bundle', '*.gem',
+                     'site/output', '.config', 'coverage',
+                     'test_*.log', 'log', 'doc']
+  p.url = "http://unicorn.bogomips.org"
+  p.ignore_pattern = /^(pkg|site|projects|doc|log)|CVS|\.log/
+  p.need_tar_gz = false
+  p.need_tgz = true
+
+  p.extension_pattern = ["ext/**/extconf.rb"]
+
+  # Eric hasn't bothered to figure out running exec tests properly
+  # from Rake, but Eric prefers GNU make to Rake for tests anyways...
+  p.test_pattern = [ 'test/unit/test*.rb' ]
+end
+
+#### Ragel builder
+
+desc "Rebuild the Ragel sources"
+task :ragel do
+  Dir.chdir "ext/unicorn/http11" do
+    target = "http11_parser.c"
+    File.unlink target if File.exist? target
+    sh "ragel http11_parser.rl -C -G2 -o #{target}"
+    raise "Failed to build C source" unless File.exist? target
+  end
+end