unicorn-simon 0.0.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +7 -0
- data/.CHANGELOG.old +25 -0
- data/.document +28 -0
- data/.gitattributes +5 -0
- data/.gitignore +25 -0
- data/.mailmap +26 -0
- data/.manifest +156 -0
- data/.olddoc.yml +18 -0
- data/Application_Timeouts +77 -0
- data/CONTRIBUTORS +35 -0
- data/COPYING +674 -0
- data/DESIGN +95 -0
- data/Documentation/.gitignore +5 -0
- data/Documentation/GNUmakefile +30 -0
- data/Documentation/unicorn.1.txt +187 -0
- data/Documentation/unicorn_rails.1.txt +175 -0
- data/FAQ +70 -0
- data/GIT-VERSION-FILE +1 -0
- data/GIT-VERSION-GEN +39 -0
- data/GNUmakefile +253 -0
- data/HACKING +120 -0
- data/ISSUES +90 -0
- data/KNOWN_ISSUES +79 -0
- data/LATEST +30 -0
- data/LICENSE +67 -0
- data/Links +56 -0
- data/NEWS +2465 -0
- data/PHILOSOPHY +139 -0
- data/README +138 -0
- data/Rakefile +16 -0
- data/SIGNALS +123 -0
- data/Sandbox +104 -0
- data/TODO +3 -0
- data/TUNING +119 -0
- data/archive/.gitignore +3 -0
- data/archive/slrnpull.conf +4 -0
- data/bin/unicorn +126 -0
- data/bin/unicorn_rails +209 -0
- data/examples/big_app_gc.rb +2 -0
- data/examples/echo.ru +27 -0
- data/examples/init.sh +102 -0
- data/examples/logger_mp_safe.rb +25 -0
- data/examples/logrotate.conf +44 -0
- data/examples/nginx.conf +155 -0
- data/examples/unicorn.conf.minimal.rb +13 -0
- data/examples/unicorn.conf.rb +110 -0
- data/examples/unicorn.socket +11 -0
- data/examples/unicorn@.service +33 -0
- data/ext/unicorn_http/CFLAGS +13 -0
- data/ext/unicorn_http/c_util.h +124 -0
- data/ext/unicorn_http/common_field_optimization.h +111 -0
- data/ext/unicorn_http/ext_help.h +62 -0
- data/ext/unicorn_http/extconf.rb +11 -0
- data/ext/unicorn_http/global_variables.h +97 -0
- data/ext/unicorn_http/httpdate.c +78 -0
- data/ext/unicorn_http/unicorn_http.c +4274 -0
- data/ext/unicorn_http/unicorn_http.rl +980 -0
- data/ext/unicorn_http/unicorn_http_common.rl +76 -0
- data/lib/unicorn/app/old_rails/static.rb +59 -0
- data/lib/unicorn/app/old_rails.rb +35 -0
- data/lib/unicorn/cgi_wrapper.rb +147 -0
- data/lib/unicorn/configurator.rb +664 -0
- data/lib/unicorn/const.rb +21 -0
- data/lib/unicorn/http_request.rb +122 -0
- data/lib/unicorn/http_response.rb +60 -0
- data/lib/unicorn/http_server.rb +824 -0
- data/lib/unicorn/launcher.rb +62 -0
- data/lib/unicorn/oob_gc.rb +82 -0
- data/lib/unicorn/preread_input.rb +33 -0
- data/lib/unicorn/socket_helper.rb +195 -0
- data/lib/unicorn/stream_input.rb +146 -0
- data/lib/unicorn/tee_input.rb +133 -0
- data/lib/unicorn/tmpio.rb +27 -0
- data/lib/unicorn/util.rb +90 -0
- data/lib/unicorn/version.rb +1 -0
- data/lib/unicorn/worker.rb +140 -0
- data/lib/unicorn.rb +123 -0
- data/man/man1/unicorn.1 +221 -0
- data/man/man1/unicorn_rails.1 +212 -0
- data/setup.rb +1586 -0
- data/t/.gitignore +4 -0
- data/t/GNUmakefile +74 -0
- data/t/README +42 -0
- data/t/bin/content-md5-put +36 -0
- data/t/bin/sha1sum.rb +17 -0
- data/t/bin/unused_listen +40 -0
- data/t/broken-app.ru +12 -0
- data/t/detach.ru +11 -0
- data/t/env.ru +3 -0
- data/t/fails-rack-lint.ru +5 -0
- data/t/heartbeat-timeout.ru +12 -0
- data/t/hijack.ru +43 -0
- data/t/listener_names.ru +4 -0
- data/t/my-tap-lib.sh +201 -0
- data/t/oob_gc.ru +20 -0
- data/t/oob_gc_path.ru +20 -0
- data/t/pid.ru +3 -0
- data/t/preread_input.ru +17 -0
- data/t/rack-input-tests.ru +21 -0
- data/t/t0000-http-basic.sh +50 -0
- data/t/t0001-reload-bad-config.sh +53 -0
- data/t/t0002-config-conflict.sh +49 -0
- data/t/t0002-parser-error.sh +94 -0
- data/t/t0003-working_directory.sh +51 -0
- data/t/t0004-heartbeat-timeout.sh +69 -0
- data/t/t0004-working_directory_broken.sh +24 -0
- data/t/t0005-working_directory_app.rb.sh +40 -0
- data/t/t0006-reopen-logs.sh +83 -0
- data/t/t0006.ru +13 -0
- data/t/t0007-working_directory_no_embed_cli.sh +44 -0
- data/t/t0008-back_out_of_upgrade.sh +110 -0
- data/t/t0009-broken-app.sh +56 -0
- data/t/t0009-winch_ttin.sh +59 -0
- data/t/t0010-reap-logging.sh +55 -0
- data/t/t0011-active-unix-socket.sh +79 -0
- data/t/t0012-reload-empty-config.sh +85 -0
- data/t/t0013-rewindable-input-false.sh +24 -0
- data/t/t0013.ru +12 -0
- data/t/t0014-rewindable-input-true.sh +24 -0
- data/t/t0014.ru +12 -0
- data/t/t0015-configurator-internals.sh +25 -0
- data/t/t0018-write-on-close.sh +23 -0
- data/t/t0019-max_header_len.sh +49 -0
- data/t/t0020-at_exit-handler.sh +49 -0
- data/t/t0021-process_detach.sh +29 -0
- data/t/t0022-listener_names-preload_app.sh +32 -0
- data/t/t0100-rack-input-tests.sh +124 -0
- data/t/t0116-client_body_buffer_size.sh +80 -0
- data/t/t0116.ru +16 -0
- data/t/t0200-rack-hijack.sh +30 -0
- data/t/t0300-no-default-middleware.sh +20 -0
- data/t/t9000-preread-input.sh +48 -0
- data/t/t9001-oob_gc.sh +47 -0
- data/t/t9002-oob_gc-path.sh +75 -0
- data/t/test-lib.sh +128 -0
- data/t/write-on-close.ru +11 -0
- data/test/aggregate.rb +15 -0
- data/test/benchmark/README +50 -0
- data/test/benchmark/dd.ru +18 -0
- data/test/benchmark/stack.ru +8 -0
- data/test/exec/README +5 -0
- data/test/exec/test_exec.rb +1099 -0
- data/test/test_helper.rb +298 -0
- data/test/unit/test_configurator.rb +175 -0
- data/test/unit/test_droplet.rb +28 -0
- data/test/unit/test_http_parser.rb +886 -0
- data/test/unit/test_http_parser_ng.rb +633 -0
- data/test/unit/test_request.rb +182 -0
- data/test/unit/test_response.rb +111 -0
- data/test/unit/test_server.rb +268 -0
- data/test/unit/test_signals.rb +188 -0
- data/test/unit/test_socket_helper.rb +197 -0
- data/test/unit/test_stream_input.rb +203 -0
- data/test/unit/test_tee_input.rb +304 -0
- data/test/unit/test_upload.rb +306 -0
- data/test/unit/test_util.rb +105 -0
- data/unicorn.gemspec +50 -0
- metadata +310 -0
data/PHILOSOPHY
ADDED
@@ -0,0 +1,139 @@
|
|
1
|
+
= The Philosophy Behind unicorn
|
2
|
+
|
3
|
+
Being a server that only runs on Unix-like platforms, unicorn is
|
4
|
+
strongly tied to the Unix philosophy of doing one thing and (hopefully)
|
5
|
+
doing it well. Despite using HTTP, unicorn is strictly a _backend_
|
6
|
+
application server for running Rack-based Ruby applications.
|
7
|
+
|
8
|
+
== Avoid Complexity
|
9
|
+
|
10
|
+
Instead of attempting to be efficient at serving slow clients, unicorn
|
11
|
+
relies on a buffering reverse proxy to efficiently deal with slow
|
12
|
+
clients.
|
13
|
+
|
14
|
+
unicorn uses an old-fashioned preforking worker model with blocking I/O.
|
15
|
+
Our processing model is the antithesis of more modern (and theoretically
|
16
|
+
more efficient) server processing models using threads or non-blocking
|
17
|
+
I/O with events.
|
18
|
+
|
19
|
+
=== Threads and Events Are Hard
|
20
|
+
|
21
|
+
...to many developers. Reasons for this is beyond the scope of this
|
22
|
+
document. unicorn avoids concurrency within each worker process so you
|
23
|
+
have fewer things to worry about when developing your application. Of
|
24
|
+
course unicorn can use multiple worker processes to utilize multiple
|
25
|
+
CPUs or spindles. Applications can still use threads internally, however.
|
26
|
+
|
27
|
+
== Slow Clients Are Problematic
|
28
|
+
|
29
|
+
Most benchmarks we've seen don't tell you this, and unicorn doesn't
|
30
|
+
care about slow clients... but <i>you</i> should.
|
31
|
+
|
32
|
+
A "slow client" can be any client outside of your datacenter. Network
|
33
|
+
traffic within a local network is always faster than traffic that
|
34
|
+
crosses outside of it. The laws of physics do not allow otherwise.
|
35
|
+
|
36
|
+
Persistent connections were introduced in HTTP/1.1 reduce latency from
|
37
|
+
connection establishment and TCP slow start. They also waste server
|
38
|
+
resources when clients are idle.
|
39
|
+
|
40
|
+
Persistent connections mean one of the unicorn worker processes
|
41
|
+
(depending on your application, it can be very memory hungry) would
|
42
|
+
spend a significant amount of its time idle keeping the connection alive
|
43
|
+
<i>and not doing anything else</i>. Being single-threaded and using
|
44
|
+
blocking I/O, a worker cannot serve other clients while keeping a
|
45
|
+
connection alive. Thus unicorn does not implement persistent
|
46
|
+
connections.
|
47
|
+
|
48
|
+
If your application responses are larger than the socket buffer or if
|
49
|
+
you're handling large requests (uploads), worker processes will also be
|
50
|
+
bottlenecked by the speed of the *client* connection. You should
|
51
|
+
not allow unicorn to serve clients outside of your local network.
|
52
|
+
|
53
|
+
== Application Concurrency != Network Concurrency
|
54
|
+
|
55
|
+
Performance is asymmetric across the different subsystems of the machine
|
56
|
+
and parts of the network. CPUs and main memory can process gigabytes of
|
57
|
+
data in a second; clients on the Internet are usually only capable of a
|
58
|
+
tiny fraction of that. unicorn deployments should avoid dealing with
|
59
|
+
slow clients directly and instead rely on a reverse proxy to shield it
|
60
|
+
from the effects of slow I/O.
|
61
|
+
|
62
|
+
== Improved Performance Through Reverse Proxying
|
63
|
+
|
64
|
+
By acting as a buffer to shield unicorn from slow I/O, a reverse proxy
|
65
|
+
will inevitably incur overhead in the form of extra data copies.
|
66
|
+
However, as I/O within a local network is fast (and faster still
|
67
|
+
with local sockets), this overhead is negligible for the vast majority
|
68
|
+
of HTTP requests and responses.
|
69
|
+
|
70
|
+
The ideal reverse proxy complements the weaknesses of unicorn.
|
71
|
+
A reverse proxy for unicorn should meet the following requirements:
|
72
|
+
|
73
|
+
1. It should fully buffer all HTTP requests (and large responses).
|
74
|
+
Each request should be "corked" in the reverse proxy and sent
|
75
|
+
as fast as possible to the backend unicorn processes. This is
|
76
|
+
the most important feature to look for when choosing a
|
77
|
+
reverse proxy for unicorn.
|
78
|
+
|
79
|
+
2. It should spend minimal time in userspace. Network (and disk) I/O
|
80
|
+
are system-level tasks and usually managed by the kernel.
|
81
|
+
This may change if userspace TCP stacks become more popular in the
|
82
|
+
future; but the reverse proxy should not waste time with
|
83
|
+
application-level logic. These concerns should be separated
|
84
|
+
|
85
|
+
3. It should avoid context switches and CPU scheduling overhead.
|
86
|
+
In many (most?) cases, network devices and their interrupts are
|
87
|
+
only be handled by one CPU at a time. It should avoid contention
|
88
|
+
within the system by serializing all network I/O into one (or few)
|
89
|
+
userspace processes. Network I/O is not a CPU-intensive task and
|
90
|
+
it is not helpful to use multiple CPU cores (at least not for GigE).
|
91
|
+
|
92
|
+
4. It should efficiently manage persistent connections (and
|
93
|
+
pipelining) to slow clients. If you care to serve slow clients
|
94
|
+
outside your network, then these features of HTTP/1.1 will help.
|
95
|
+
|
96
|
+
5. It should (optionally) serve static files. If you have static
|
97
|
+
files on your site (especially large ones), they are far more
|
98
|
+
efficiently served with as few data copies as possible (e.g. with
|
99
|
+
sendfile() to completely avoid copying the data to userspace).
|
100
|
+
|
101
|
+
nginx is the only (Free) solution we know of that meets the above
|
102
|
+
requirements.
|
103
|
+
|
104
|
+
Indeed, the folks behind unicorn have deployed nginx as a reverse-proxy not
|
105
|
+
only for Ruby applications, but also for production applications running
|
106
|
+
Apache/mod_perl, Apache/mod_php and Apache Tomcat. In every single
|
107
|
+
case, performance improved because application servers were able to use
|
108
|
+
backend resources more efficiently and spend less time waiting on slow
|
109
|
+
I/O.
|
110
|
+
|
111
|
+
== Worse Is Better
|
112
|
+
|
113
|
+
Requirements and scope for applications change frequently and
|
114
|
+
drastically. Thus languages like Ruby and frameworks like Rails were
|
115
|
+
built to give developers fewer things to worry about in the face of
|
116
|
+
rapid change.
|
117
|
+
|
118
|
+
On the other hand, stable protocols which host your applications (HTTP
|
119
|
+
and TCP) only change rarely. This is why we recommend you NOT tie your
|
120
|
+
rapidly-changing application logic directly into the processes that deal
|
121
|
+
with the stable outside world. Instead, use HTTP as a common RPC
|
122
|
+
protocol to communicate between your frontend and backend.
|
123
|
+
|
124
|
+
In short: separate your concerns.
|
125
|
+
|
126
|
+
Of course a theoretical "perfect" solution would combine the pieces
|
127
|
+
and _maybe_ give you better performance at the end of the day, but
|
128
|
+
that is not the Unix way.
|
129
|
+
|
130
|
+
== Just Worse in Some Cases
|
131
|
+
|
132
|
+
unicorn is not suited for all applications. unicorn is optimized for
|
133
|
+
applications that are CPU/memory/disk intensive and spend little time
|
134
|
+
waiting on external resources (e.g. a database server or external API).
|
135
|
+
|
136
|
+
unicorn is highly inefficient for Comet/reverse-HTTP/push applications
|
137
|
+
where the HTTP connection spends a large amount of time idle.
|
138
|
+
Nevertheless, the ease of troubleshooting, debugging, and management of
|
139
|
+
unicorn may still outweigh the drawbacks for these applications.
|
data/README
ADDED
@@ -0,0 +1,138 @@
|
|
1
|
+
= unicorn: Rack HTTP server for fast clients and Unix
|
2
|
+
|
3
|
+
unicorn is an HTTP server for Rack applications designed to only serve
|
4
|
+
fast clients on low-latency, high-bandwidth connections and take
|
5
|
+
advantage of features in Unix/Unix-like kernels. Slow clients should
|
6
|
+
only be served by placing a reverse proxy capable of fully buffering
|
7
|
+
both the the request and response in between unicorn and slow clients.
|
8
|
+
|
9
|
+
== Features
|
10
|
+
|
11
|
+
* Designed for Rack, Unix, fast clients, and ease-of-debugging. We
|
12
|
+
cut out everything that is better supported by the operating system,
|
13
|
+
{nginx}[http://nginx.org/] or {Rack}[http://rack.github.io/].
|
14
|
+
|
15
|
+
* Compatible with Ruby 1.9.3 and later.
|
16
|
+
unicorn 4.x remains supported for Ruby 1.8 users.
|
17
|
+
|
18
|
+
* Process management: unicorn will reap and restart workers that
|
19
|
+
die from broken apps. There is no need to manage multiple processes
|
20
|
+
or ports yourself. unicorn can spawn and manage any number of
|
21
|
+
worker processes you choose to scale to your backend.
|
22
|
+
|
23
|
+
* Load balancing is done entirely by the operating system kernel.
|
24
|
+
Requests never pile up behind a busy worker process.
|
25
|
+
|
26
|
+
* Does not care if your application is thread-safe or not, workers
|
27
|
+
all run within their own isolated address space and only serve one
|
28
|
+
client at a time for maximum robustness.
|
29
|
+
|
30
|
+
* Builtin reopening of all log files in your application via
|
31
|
+
USR1 signal. This allows logrotate to rotate files atomically and
|
32
|
+
quickly via rename instead of the racy and slow copytruncate method.
|
33
|
+
unicorn also takes steps to ensure multi-line log entries from one
|
34
|
+
request all stay within the same file.
|
35
|
+
|
36
|
+
* nginx-style binary upgrades without losing connections.
|
37
|
+
You can upgrade unicorn, your entire application, libraries
|
38
|
+
and even your Ruby interpreter without dropping clients.
|
39
|
+
|
40
|
+
* before_fork and after_fork hooks in case your application
|
41
|
+
has special needs when dealing with forked processes. These
|
42
|
+
should not be needed when the "preload_app" directive is
|
43
|
+
false (the default).
|
44
|
+
|
45
|
+
* Can be used with copy-on-write-friendly memory management
|
46
|
+
to save memory (by setting "preload_app" to true).
|
47
|
+
|
48
|
+
* Able to listen on multiple interfaces including UNIX sockets,
|
49
|
+
each worker process can also bind to a private port via the
|
50
|
+
after_fork hook for easy debugging.
|
51
|
+
|
52
|
+
* Simple and easy Ruby DSL for configuration.
|
53
|
+
|
54
|
+
* Decodes chunked requests on-the-fly.
|
55
|
+
|
56
|
+
== License
|
57
|
+
|
58
|
+
unicorn is copyright 2009-2016 by all contributors (see logs in git).
|
59
|
+
It is based on Mongrel 1.1.5.
|
60
|
+
Mongrel is copyright 2007 Zed A. Shaw and contributors.
|
61
|
+
|
62
|
+
unicorn is licensed under (your choice) of the GPLv2 or later
|
63
|
+
(GPLv3+ preferred), or Ruby (1.8)-specific terms.
|
64
|
+
See the included LICENSE file for details.
|
65
|
+
|
66
|
+
unicorn is 100% Free Software (including all development tools used).
|
67
|
+
|
68
|
+
== Install
|
69
|
+
|
70
|
+
The library consists of a C extension so you'll need a C compiler
|
71
|
+
and Ruby development libraries/headers.
|
72
|
+
|
73
|
+
You may install it via RubyGems on RubyGems.org:
|
74
|
+
|
75
|
+
gem install unicorn
|
76
|
+
|
77
|
+
You can get the latest source via git from the following locations
|
78
|
+
(these versions may not be stable):
|
79
|
+
|
80
|
+
git://bogomips.org/unicorn.git
|
81
|
+
git://repo.or.cz/unicorn.git (mirror)
|
82
|
+
|
83
|
+
You may browse the code from the web:
|
84
|
+
|
85
|
+
* https://bogomips.org/unicorn.git
|
86
|
+
* http://repo.or.cz/w/unicorn.git (gitweb)
|
87
|
+
|
88
|
+
See the HACKING guide on how to contribute and build prerelease gems
|
89
|
+
from git.
|
90
|
+
|
91
|
+
== Usage
|
92
|
+
|
93
|
+
=== Rack (including Rails 3+) applications
|
94
|
+
|
95
|
+
In APP_ROOT, run:
|
96
|
+
|
97
|
+
unicorn
|
98
|
+
|
99
|
+
unicorn will bind to all interfaces on TCP port 8080 by default.
|
100
|
+
You may use the +--listen/-l+ switch to bind to a different
|
101
|
+
address:port or a UNIX socket.
|
102
|
+
|
103
|
+
=== Configuration File(s)
|
104
|
+
|
105
|
+
unicorn will look for the config.ru file used by rackup in APP_ROOT.
|
106
|
+
|
107
|
+
For deployments, it can use a config file for unicorn-specific options
|
108
|
+
specified by the +--config-file/-c+ command-line switch. See
|
109
|
+
Unicorn::Configurator for the syntax of the unicorn-specific options.
|
110
|
+
The default settings are designed for maximum out-of-the-box
|
111
|
+
compatibility with existing applications.
|
112
|
+
|
113
|
+
Most command-line options for other Rack applications (above) are also
|
114
|
+
supported. Run `unicorn -h` to see command-line options.
|
115
|
+
|
116
|
+
== Disclaimer
|
117
|
+
|
118
|
+
There is NO WARRANTY whatsoever if anything goes wrong, but
|
119
|
+
{let us know}[link:ISSUES.html] and we'll try our best to fix it.
|
120
|
+
|
121
|
+
unicorn is designed to only serve fast clients either on the local host
|
122
|
+
or a fast LAN. See the PHILOSOPHY and DESIGN documents for more details
|
123
|
+
regarding this.
|
124
|
+
|
125
|
+
== Contact
|
126
|
+
|
127
|
+
All feedback (bug reports, user/development dicussion, patches, pull
|
128
|
+
requests) go to the mailing list/newsgroup. See the ISSUES document for
|
129
|
+
information on the {mailing list}[mailto:unicorn-public@bogomips.org].
|
130
|
+
|
131
|
+
The mailing list is archived at https://bogomips.org/unicorn-public/
|
132
|
+
Read-only NNTP access is available at:
|
133
|
+
nntp://news.public-inbox.org/inbox.comp.lang.ruby.unicorn and
|
134
|
+
nntp://news.gmane.org/gmane.comp.lang.ruby.unicorn.general
|
135
|
+
|
136
|
+
For the latest on unicorn releases, you may also finger us at
|
137
|
+
unicorn@bogomips.org or check our NEWS page (and subscribe to our Atom
|
138
|
+
feed).
|
data/Rakefile
ADDED
@@ -0,0 +1,16 @@
|
|
1
|
+
# optional rake-compiler support in case somebody needs to cross compile
|
2
|
+
begin
|
3
|
+
mk = "ext/unicorn_http/Makefile"
|
4
|
+
if File.readable?(mk)
|
5
|
+
warn "run 'gmake -C ext/unicorn_http clean' and\n" \
|
6
|
+
"remove #{mk} before using rake-compiler"
|
7
|
+
elsif ENV['VERSION']
|
8
|
+
unless File.readable?("ext/unicorn_http/unicorn_http.c")
|
9
|
+
abort "run 'gmake ragel' or 'make ragel' to generate the Ragel source"
|
10
|
+
end
|
11
|
+
spec = Gem::Specification.load('unicorn.gemspec')
|
12
|
+
require 'rake/extensiontask'
|
13
|
+
Rake::ExtensionTask.new('unicorn_http', spec)
|
14
|
+
end
|
15
|
+
rescue LoadError
|
16
|
+
end
|
data/SIGNALS
ADDED
@@ -0,0 +1,123 @@
|
|
1
|
+
== Signal handling
|
2
|
+
|
3
|
+
In general, signals need only be sent to the master process. However,
|
4
|
+
the signals Unicorn uses internally to communicate with the worker
|
5
|
+
processes are documented here as well. With the exception of TTIN/TTOU,
|
6
|
+
signal handling matches the behavior of {nginx}[http://nginx.org/] so it
|
7
|
+
should be possible to easily share process management scripts between
|
8
|
+
Unicorn and nginx.
|
9
|
+
|
10
|
+
One example init script is distributed with unicorn:
|
11
|
+
https://bogomips.org/unicorn/examples/init.sh
|
12
|
+
|
13
|
+
=== Master Process
|
14
|
+
|
15
|
+
* HUP - reloads config file and gracefully restart all workers.
|
16
|
+
If the "preload_app" directive is false (the default), then workers
|
17
|
+
will also pick up any application code changes when restarted. If
|
18
|
+
"preload_app" is true, then application code changes will have no
|
19
|
+
effect; USR2 + QUIT (see below) must be used to load newer code in
|
20
|
+
this case. When reloading the application, +Gem.refresh+ will
|
21
|
+
be called so updated code for your application can pick up newly
|
22
|
+
installed RubyGems. It is not recommended that you uninstall
|
23
|
+
libraries your application depends on while Unicorn is running,
|
24
|
+
as respawned workers may enter a spawn loop when they fail to
|
25
|
+
load an uninstalled dependency.
|
26
|
+
|
27
|
+
* INT/TERM - quick shutdown, kills all workers immediately
|
28
|
+
|
29
|
+
* QUIT - graceful shutdown, waits for workers to finish their
|
30
|
+
current request before finishing.
|
31
|
+
|
32
|
+
* USR1 - reopen all logs owned by the master and all workers
|
33
|
+
See Unicorn::Util.reopen_logs for what is considered a log.
|
34
|
+
|
35
|
+
* USR2 - reexecute the running binary. A separate QUIT
|
36
|
+
should be sent to the original process once the child is verified to
|
37
|
+
be up and running.
|
38
|
+
|
39
|
+
* WINCH - gracefully stops workers but keep the master running.
|
40
|
+
This will only work for daemonized processes.
|
41
|
+
|
42
|
+
* TTIN - increment the number of worker processes by one
|
43
|
+
|
44
|
+
* TTOU - decrement the number of worker processes by one
|
45
|
+
|
46
|
+
=== Worker Processes
|
47
|
+
|
48
|
+
Note: as of unicorn 4.8, the master uses a pipe to signal workers
|
49
|
+
instead of kill(2) for most cases. Using signals still (and works and
|
50
|
+
remains supported for external tools/libraries), however.
|
51
|
+
|
52
|
+
Sending signals directly to the worker processes should not normally be
|
53
|
+
needed. If the master process is running, any exited worker will be
|
54
|
+
automatically respawned.
|
55
|
+
|
56
|
+
* INT/TERM - Quick shutdown, immediately exit.
|
57
|
+
Unless WINCH has been sent to the master (or the master is killed),
|
58
|
+
the master process will respawn a worker to replace this one.
|
59
|
+
Immediate shutdown is still triggered using kill(2) and not the
|
60
|
+
internal pipe as of unicorn 4.8
|
61
|
+
|
62
|
+
* QUIT - Gracefully exit after finishing the current request.
|
63
|
+
Unless WINCH has been sent to the master (or the master is killed),
|
64
|
+
the master process will respawn a worker to replace this one.
|
65
|
+
|
66
|
+
* USR1 - Reopen all logs owned by the worker process.
|
67
|
+
See Unicorn::Util.reopen_logs for what is considered a log.
|
68
|
+
Log files are not reopened until it is done processing
|
69
|
+
the current request, so multiple log lines for one request
|
70
|
+
(as done by Rails) will not be split across multiple logs.
|
71
|
+
|
72
|
+
It is NOT recommended to send the USR1 signal directly to workers via
|
73
|
+
"killall -USR1 unicorn" if you are using user/group-switching support
|
74
|
+
in your workers. You will encounter incorrect file permissions and
|
75
|
+
workers will need to be respawned. Sending USR1 to the master process
|
76
|
+
first will ensure logs have the correct permissions before the master
|
77
|
+
forwards the USR1 signal to workers.
|
78
|
+
|
79
|
+
=== Procedure to replace a running unicorn executable
|
80
|
+
|
81
|
+
You may replace a running instance of unicorn with a new one without
|
82
|
+
losing any incoming connections. Doing so will reload all of your
|
83
|
+
application code, Unicorn config, Ruby executable, and all libraries.
|
84
|
+
The only things that will not change (due to OS limitations) are:
|
85
|
+
|
86
|
+
1. The path to the unicorn executable script. If you want to change to
|
87
|
+
a different installation of Ruby, you can modify the shebang
|
88
|
+
line to point to your alternative interpreter.
|
89
|
+
|
90
|
+
The procedure is exactly like that of nginx:
|
91
|
+
|
92
|
+
1. Send USR2 to the master process
|
93
|
+
|
94
|
+
2. Check your process manager or pid files to see if a new master spawned
|
95
|
+
successfully. If you're using a pid file, the old process will have
|
96
|
+
".oldbin" appended to its path. You should have two master instances
|
97
|
+
of unicorn running now, both of which will have workers servicing
|
98
|
+
requests. Your process tree should look something like this:
|
99
|
+
|
100
|
+
unicorn master (old)
|
101
|
+
\_ unicorn worker[0]
|
102
|
+
\_ unicorn worker[1]
|
103
|
+
\_ unicorn worker[2]
|
104
|
+
\_ unicorn worker[3]
|
105
|
+
\_ unicorn master
|
106
|
+
\_ unicorn worker[0]
|
107
|
+
\_ unicorn worker[1]
|
108
|
+
\_ unicorn worker[2]
|
109
|
+
\_ unicorn worker[3]
|
110
|
+
|
111
|
+
3. You can now send WINCH to the old master process so only the new workers
|
112
|
+
serve requests. If your unicorn process is bound to an interactive
|
113
|
+
terminal (not daemonized), you can skip this step. Step 5 will be more
|
114
|
+
difficult but you can also skip it if your process is not daemonized.
|
115
|
+
|
116
|
+
4. You should now ensure that everything is running correctly with the
|
117
|
+
new workers as the old workers die off.
|
118
|
+
|
119
|
+
5. If everything seems ok, then send QUIT to the old master. You're done!
|
120
|
+
|
121
|
+
If something is broken, then send HUP to the old master to reload
|
122
|
+
the config and restart its workers. Then send QUIT to the new master
|
123
|
+
process.
|
data/Sandbox
ADDED
@@ -0,0 +1,104 @@
|
|
1
|
+
= Tips for using unicorn with Sandbox installation tools
|
2
|
+
|
3
|
+
Since unicorn includes executables and is usually used to start a Ruby
|
4
|
+
process, there are certain caveats to using it with tools that sandbox
|
5
|
+
RubyGems installations such as
|
6
|
+
{Bundler}[http://bundler.io/] or
|
7
|
+
{Isolate}[https://github.com/jbarnette/isolate].
|
8
|
+
|
9
|
+
== General deployment
|
10
|
+
|
11
|
+
If you're sandboxing your unicorn installation and using Capistrano (or
|
12
|
+
similar), it's required that you sandbox your RubyGems in a per-application
|
13
|
+
shared directory that can be used between different revisions.
|
14
|
+
|
15
|
+
unicorn will stash its original command-line at startup for the USR2
|
16
|
+
upgrades, and cleaning up old revisions will cause revision-specific
|
17
|
+
installations of unicorn to go missing and upgrades to fail. If you
|
18
|
+
find yourself in this situation and can't afford downtime, you can
|
19
|
+
override the existing unicorn executable path in the config file like
|
20
|
+
this:
|
21
|
+
|
22
|
+
Unicorn::HttpServer::START_CTX[0] = "/some/path/to/bin/unicorn"
|
23
|
+
|
24
|
+
Then use HUP to reload, and then continue with the USR2+QUIT upgrade
|
25
|
+
sequence.
|
26
|
+
|
27
|
+
Environment variable pollution when exec-ing a new process (with USR2)
|
28
|
+
is the primary issue with sandboxing tools such as Bundler and Isolate.
|
29
|
+
|
30
|
+
== Bundler
|
31
|
+
|
32
|
+
=== Running
|
33
|
+
|
34
|
+
If you're bundling unicorn, use "bundle exec unicorn" (or "bundle exec
|
35
|
+
unicorn_rails") to start unicorn with the correct environment variables
|
36
|
+
|
37
|
+
ref: https://bogomips.org/unicorn-public/9ECF07C4-5216-47BE-961D-AFC0F0C82060@internetfamo.us/
|
38
|
+
|
39
|
+
Otherwise (if you choose to not sandbox your unicorn installation), we
|
40
|
+
expect the tips for Isolate (below) apply, too.
|
41
|
+
|
42
|
+
=== RUBYOPT pollution from SIGUSR2 upgrades
|
43
|
+
|
44
|
+
This is no longer be an issue as of bundler 0.9.17
|
45
|
+
|
46
|
+
ref:
|
47
|
+
https://bogomips.org/unicorn-public/8FC34B23-5994-41CC-B5AF-7198EF06909E@tramchase.com/
|
48
|
+
|
49
|
+
=== BUNDLE_GEMFILE for Capistrano users
|
50
|
+
|
51
|
+
You may need to set or reset the BUNDLE_GEMFILE environment variable in
|
52
|
+
the before_exec hook:
|
53
|
+
|
54
|
+
before_exec do |server|
|
55
|
+
ENV["BUNDLE_GEMFILE"] = "/path/to/app/current/Gemfile"
|
56
|
+
end
|
57
|
+
|
58
|
+
=== Other ENV pollution issues
|
59
|
+
|
60
|
+
If you're using an older Bundler version (0.9.x), you may need to set or
|
61
|
+
reset GEM_HOME, GEM_PATH and PATH environment variables in the
|
62
|
+
before_exec hook as illustrated by https://gist.github.com/534668
|
63
|
+
|
64
|
+
=== Ruby 2.0.0 close-on-exec and SIGUSR2 incompatibility
|
65
|
+
|
66
|
+
Ruby 2.0.0 enforces FD_CLOEXEC on file descriptors by default. unicorn
|
67
|
+
has been prepared for this behavior since unicorn 4.1.0, and bundler
|
68
|
+
needs the "--keep-file-descriptors" option for "bundle exec":
|
69
|
+
http://bundler.io/man/bundle-exec.1.html
|
70
|
+
|
71
|
+
== Isolate
|
72
|
+
|
73
|
+
=== Running
|
74
|
+
|
75
|
+
Installing "unicorn" as a system-wide Rubygem and using the
|
76
|
+
isolate gem may cause issues if you're using any of the bundled
|
77
|
+
application-level libraries in unicorn/app/* (for compatibility
|
78
|
+
with CGI-based applications, Rails <= 2.2.2, or ExecCgi).
|
79
|
+
For now workarounds include doing one of the following:
|
80
|
+
|
81
|
+
1. Isolating unicorn, setting GEM_HOME to your Isolate path,
|
82
|
+
and running the isolated version of unicorn. You *must* set
|
83
|
+
GEM_HOME before running your isolated unicorn install in this way.
|
84
|
+
|
85
|
+
2. Installing the same version of unicorn as a system-wide Rubygem
|
86
|
+
*and* isolating unicorn as well.
|
87
|
+
|
88
|
+
3. Explicitly setting RUBYLIB or $LOAD_PATH to include any gem path
|
89
|
+
where the unicorn gem is installed
|
90
|
+
(e.g. /usr/lib/ruby/gems/1.9.3/gems/unicorn-VERSION/lib)
|
91
|
+
|
92
|
+
=== RUBYOPT pollution from SIGUSR2 upgrades
|
93
|
+
|
94
|
+
If you are using Isolate, using Isolate 2.x is strongly recommended as
|
95
|
+
environment modifications are idempotent.
|
96
|
+
|
97
|
+
If you are stuck with 1.x versions of Isolate, it is recommended that
|
98
|
+
you disable it with the <tt>before_exec</tt> hook prevent the PATH and
|
99
|
+
RUBYOPT environment variable modifications from propagating between
|
100
|
+
upgrades in your Unicorn config file:
|
101
|
+
|
102
|
+
before_exec do |server|
|
103
|
+
Isolate.disable
|
104
|
+
end
|
data/TUNING
ADDED
@@ -0,0 +1,119 @@
|
|
1
|
+
= Tuning unicorn
|
2
|
+
|
3
|
+
unicorn performance is generally as good as a (mostly) Ruby web server
|
4
|
+
can provide. Most often the performance bottleneck is in the web
|
5
|
+
application running on Unicorn rather than Unicorn itself.
|
6
|
+
|
7
|
+
== unicorn Configuration
|
8
|
+
|
9
|
+
See Unicorn::Configurator for details on the config file format.
|
10
|
+
+worker_processes+ is the most-commonly needed tuning parameter.
|
11
|
+
|
12
|
+
=== Unicorn::Configurator#worker_processes
|
13
|
+
|
14
|
+
* worker_processes should be scaled to the number of processes your
|
15
|
+
backend system(s) can support. DO NOT scale it to the number of
|
16
|
+
external network clients your application expects to be serving.
|
17
|
+
unicorn is NOT for serving slow clients, that is the job of nginx.
|
18
|
+
|
19
|
+
* worker_processes should be *at* *least* the number of CPU cores on
|
20
|
+
a dedicated server (unless you do not have enough memory).
|
21
|
+
If your application has occasionally slow responses that are /not/
|
22
|
+
CPU-intensive, you may increase this to workaround those inefficiencies.
|
23
|
+
|
24
|
+
* Under Ruby 2.2 or later, Etc.nprocessors may be used to determine
|
25
|
+
the number of CPU cores present.
|
26
|
+
|
27
|
+
* worker_processes may be increased for Unicorn::OobGC users to provide
|
28
|
+
more consistent response times.
|
29
|
+
|
30
|
+
* Never, ever, increase worker_processes to the point where the system
|
31
|
+
runs out of physical memory and hits swap. Production servers should
|
32
|
+
never see heavy swap activity.
|
33
|
+
|
34
|
+
=== Unicorn::Configurator#listen Options
|
35
|
+
|
36
|
+
* Setting a very low value for the :backlog parameter in "listen"
|
37
|
+
directives can allow failover to happen more quickly if your
|
38
|
+
cluster is configured for it.
|
39
|
+
|
40
|
+
* If you're doing extremely simple benchmarks and getting connection
|
41
|
+
errors under high request rates, increasing your :backlog parameter
|
42
|
+
above the already-generous default of 1024 can help avoid connection
|
43
|
+
errors. Keep in mind this is not recommended for real traffic if
|
44
|
+
you have another machine to failover to (see above).
|
45
|
+
|
46
|
+
* :rcvbuf and :sndbuf parameters generally do not need to be set for TCP
|
47
|
+
listeners under Linux 2.6 because auto-tuning is enabled. UNIX domain
|
48
|
+
sockets do not have auto-tuning buffer sizes; so increasing those will
|
49
|
+
allow syscalls and task switches to be saved for larger requests
|
50
|
+
and responses. If your app only generates small responses or expects
|
51
|
+
small requests, you may shrink the buffer sizes to save memory, too.
|
52
|
+
|
53
|
+
* Having socket buffers too large can also be detrimental or have
|
54
|
+
little effect. Huge buffers can put more pressure on the allocator
|
55
|
+
and may also thrash CPU caches, cancelling out performance gains
|
56
|
+
one would normally expect.
|
57
|
+
|
58
|
+
* UNIX domain sockets are slightly faster than TCP sockets, but only
|
59
|
+
work if nginx is on the same machine.
|
60
|
+
|
61
|
+
== Other unicorn settings
|
62
|
+
|
63
|
+
* Setting "preload_app true" can allow copy-on-write-friendly GC to
|
64
|
+
be used to save memory. It will probably not work out of the box with
|
65
|
+
applications that open sockets or perform random I/O on files.
|
66
|
+
Databases like TokyoCabinet use concurrency-safe pread()/pwrite()
|
67
|
+
functions for safe sharing of database file descriptors across
|
68
|
+
processes.
|
69
|
+
|
70
|
+
* On POSIX-compliant filesystems, it is safe for multiple threads or
|
71
|
+
processes to append to one log file as long as all the processes are
|
72
|
+
have them unbuffered (File#sync = true) or they are
|
73
|
+
record(line)-buffered in userspace before any writes.
|
74
|
+
|
75
|
+
== Kernel Parameters (Linux sysctl and sysfs)
|
76
|
+
|
77
|
+
WARNING: Do not change system parameters unless you know what you're doing!
|
78
|
+
|
79
|
+
* Transparent hugepages (THP) improves performance in many cases,
|
80
|
+
but can also increase memory use when relying on a
|
81
|
+
copy-on-write(CoW)-friendly GC (Ruby 2.0+) with "preload_app true".
|
82
|
+
CoW operates at the page level, so writing to a huge page would
|
83
|
+
trigger a 2 MB copy (x86-64), as opposed to a 4 KB copy on a
|
84
|
+
regular (non-huge) page.
|
85
|
+
|
86
|
+
Consider only allowing THP to be used when it is requested via the
|
87
|
+
madvise(2) syscall:
|
88
|
+
|
89
|
+
echo madvise >/sys/kernel/mm/transparent_hugepage/enabled
|
90
|
+
|
91
|
+
Or disabling it system-wide, via "never".
|
92
|
+
|
93
|
+
n.b. "page" in this context only applies to the OS kernel,
|
94
|
+
Ruby GC implementations also use this term for the same concept
|
95
|
+
in a way that is agnostic to the OS.
|
96
|
+
|
97
|
+
* net.core.rmem_max and net.core.wmem_max can increase the allowed
|
98
|
+
size of :rcvbuf and :sndbuf respectively. This is mostly only useful
|
99
|
+
for UNIX domain sockets which do not have auto-tuning buffer sizes.
|
100
|
+
|
101
|
+
* For load testing/benchmarking with UNIX domain sockets, you should
|
102
|
+
consider increasing net.core.somaxconn or else nginx will start
|
103
|
+
failing to connect under heavy load. You may also consider setting
|
104
|
+
a higher :backlog to listen on as noted earlier.
|
105
|
+
|
106
|
+
* If you're running out of local ports, consider lowering
|
107
|
+
net.ipv4.tcp_fin_timeout to 20-30 (default: 60 seconds). Also
|
108
|
+
consider widening the usable port range by changing
|
109
|
+
net.ipv4.ip_local_port_range.
|
110
|
+
|
111
|
+
* Setting net.ipv4.tcp_timestamps=1 will also allow setting
|
112
|
+
net.ipv4.tcp_tw_reuse=1 and net.ipv4.tcp_tw_recycle=1, which along
|
113
|
+
with the above settings can slow down port exhaustion. Not all
|
114
|
+
networks are compatible with these settings, check with your friendly
|
115
|
+
network administrator before changing these.
|
116
|
+
|
117
|
+
* Increasing the MTU size can reduce framing overhead for larger
|
118
|
+
transfers. One often-overlooked detail is that the loopback
|
119
|
+
device (usually "lo") can have its MTU increased, too.
|
data/archive/.gitignore
ADDED