passenger 4.0.44 → 4.0.45
Sign up to get free protection for your applications and to get access to all the features.
Potentially problematic release.
This version of passenger might be problematic. Click here for more details.
- checksums.yaml +8 -8
- checksums.yaml.gz.asc +7 -7
- data.tar.gz.asc +7 -7
- data/.travis.yml +3 -0
- data/CHANGELOG +31 -0
- data/CONTRIBUTING.md +70 -10
- data/CONTRIBUTORS +4 -0
- data/README.md +1 -1
- data/Vagrantfile +50 -0
- data/bin/passenger-install-nginx-module +7 -2
- data/build/basics.rb +4 -1
- data/build/documentation.rb +6 -0
- data/build/node_tests.rb +7 -1
- data/build/packaging.rb +5 -0
- data/build/test_basics.rb +3 -3
- data/debian.template/copyright +1 -1
- data/debian.template/passenger.manpages +0 -1
- data/dev/rack.test/config.ru +5 -0
- data/dev/rack.test/public/asset.txt +1 -0
- data/dev/vagrant/apache_default_site.conf +35 -0
- data/dev/vagrant/apache_passenger.conf +5 -0
- data/dev/vagrant/apache_passenger.load +1 -0
- data/dev/vagrant/apache_ports.conf +24 -0
- data/dev/vagrant/apache_rack_test.conf +9 -0
- data/dev/vagrant/bashrc +21 -0
- data/dev/vagrant/nginx.conf +39 -0
- data/dev/vagrant/nginx_rakefile +34 -0
- data/dev/vagrant/nginx_start +32 -0
- data/dev/vagrant/provision.sh +115 -0
- data/dev/vagrant/sudoers.conf +5 -0
- data/doc/Design and Architecture.txt +515 -0
- data/doc/DeveloperQuickstart.md +70 -0
- data/doc/Users guide Apache.idmap.txt +24 -18
- data/doc/Users guide Apache.txt +200 -62
- data/doc/Users guide Nginx.idmap.txt +53 -45
- data/doc/Users guide Nginx.txt +501 -360
- data/doc/Users guide Standalone.txt +8 -0
- data/doc/images/direct_spawning.png +0 -0
- data/doc/images/direct_spawning.svg +16 -13
- data/doc/images/helper_agent_core_architecture.png +0 -0
- data/doc/images/passenger_architecture_overview.png +0 -0
- data/doc/images/smart_spawning.png +0 -0
- data/doc/images/{smart.svg → smart_spawning.svg} +23 -20
- data/doc/images/spawning_preparation_work.png +0 -0
- data/doc/images/startup_sequence.png +0 -0
- data/doc/users_guide_snippets/appendix_c_spawning_methods.txt +82 -121
- data/doc/users_guide_snippets/environment_variables.txt +1 -1
- data/doc/users_guide_snippets/support_information.txt +2 -0
- data/doc/users_guide_snippets/tips.txt +117 -9
- data/ext/apache2/Configuration.hpp +4 -2
- data/ext/apache2/ConfigurationCommands.cpp +14 -0
- data/ext/apache2/ConfigurationFields.hpp +4 -0
- data/ext/apache2/ConfigurationSetters.cpp +22 -0
- data/ext/apache2/CreateDirConfig.cpp +2 -0
- data/ext/apache2/Hooks.cpp +30 -14
- data/ext/apache2/MergeDirConfig.cpp +14 -0
- data/ext/apache2/SetHeaders.cpp +8 -0
- data/ext/common/ApplicationPool2/AppTypes.cpp +6 -1
- data/ext/common/ApplicationPool2/Implementation.cpp +1 -1
- data/ext/common/ApplicationPool2/Session.h +1 -1
- data/ext/common/Constants.h +9 -7
- data/ext/common/Utils/HttpHeaderBufferer.h +23 -4
- data/ext/common/Utils/StrIntUtils.h +35 -0
- data/ext/common/Utils/StringScanning.h +4 -10
- data/ext/common/agents/HelperAgent/RequestHandler.h +90 -49
- data/ext/nginx/CacheLocationConfig.c +40 -0
- data/ext/nginx/ConfigurationCommands.c +20 -0
- data/ext/nginx/ConfigurationFields.h +4 -0
- data/ext/nginx/ContentHandler.c +1 -1
- data/ext/nginx/CreateLocationConfig.c +9 -0
- data/ext/nginx/MergeLocationConfig.c +12 -0
- data/ext/nginx/config +2 -2
- data/ext/nginx/ngx_http_passenger_module.c +4 -4
- data/helper-scripts/node-loader.js +40 -27
- data/lib/phusion_passenger.rb +1 -1
- data/lib/phusion_passenger/apache2/config_options.rb +14 -2
- data/lib/phusion_passenger/constants.rb +7 -6
- data/lib/phusion_passenger/loader_shared_helpers.rb +11 -1
- data/lib/phusion_passenger/nginx/config_options.rb +8 -0
- data/lib/phusion_passenger/packaging.rb +8 -3
- data/lib/phusion_passenger/platform_info/apache.rb +3 -0
- data/lib/phusion_passenger/platform_info/ruby.rb +4 -1
- data/lib/phusion_passenger/standalone/command.rb +0 -1
- data/lib/phusion_passenger/standalone/package_runtime_command.rb +1 -0
- data/lib/phusion_passenger/standalone/start_command.rb +80 -62
- data/lib/phusion_passenger/standalone/status_command.rb +1 -0
- data/lib/phusion_passenger/standalone/stop_command.rb +1 -0
- data/man/passenger-config.1 +1 -1
- data/man/passenger-memory-stats.8 +1 -1
- data/man/passenger-status.8 +1 -1
- data/npm-shrinkwrap.json +229 -0
- data/package.json +28 -0
- data/resources/templates/standalone/config.erb +2 -0
- data/rpm/Vagrantfile +0 -3
- data/test/config.json.vagrant +30 -0
- data/test/cxx/HttpHeaderBuffererTest.cpp +64 -10
- data/test/cxx/RequestHandlerTest.cpp +35 -13
- data/test/integration_tests/apache2_tests.rb +1 -0
- data/test/stub/node/app.js +26 -18
- metadata +28 -13
- metadata.gz.asc +7 -7
- data/doc/Architectural overview.idmap.txt +0 -36
- data/doc/Architectural overview.txt +0 -410
- data/doc/images/smart.png +0 -0
- data/ext/common/ApplicationPool2/README.md +0 -56
- data/man/passenger-stress-test.1 +0 -43
- data/node_lib/phusion_passenger/httplib_emulation.js +0 -215
- data/node_lib/phusion_passenger/request_handler.js +0 -73
- data/node_lib/phusion_passenger/session_protocol_parser.js +0 -113
- data/test/node/httplib_emulation_spec.js +0 -623
@@ -50,6 +50,10 @@ Most configuration is done by customizing the arguments passed to the `passenger
|
|
50
50
|
+
|
51
51
|
You can use `--friendly-error-pages` and `--no-friendly-error-pages` to explicitly enable or disable this feature, respectively.
|
52
52
|
|
53
|
+
`--sticky-sessions`::
|
54
|
+
Enables link:Users%20guide%20Nginx.html#PassengerStickySessions[sticky sessions]. Required for SockJS, Socket.io, faye-websocket and Meteor JS applications.
|
55
|
+
`--sticky-sessions-cookie-name NAME`::
|
56
|
+
Allows customizing the sticky session cookie name.
|
53
57
|
`--ssl`::
|
54
58
|
Enables SSL support. If this is set, you must also set `--ssl-certificate` and `--ssl-certificate-key` to the SSL certificate and key files, respectively.
|
55
59
|
`--ssl-port`::
|
@@ -87,6 +91,10 @@ The following configuration options are supported:
|
|
87
91
|
Equivalent to the `--min-instances` command line option.
|
88
92
|
`spawn_method`::
|
89
93
|
Equivalent to the `--spawn-method` command line option.
|
94
|
+
`sticky_sessions`::
|
95
|
+
Equivalent to the `--sticky-sessions` command line option.
|
96
|
+
`sticky_sessions_cookie_name`::
|
97
|
+
Equivalent to the `--sticky-sessions-cookie-name` command line option.
|
90
98
|
`ssl`::
|
91
99
|
Equivalent to the `--ssl` command line option. When given, you must also set `ssl_certificate` and `ssl_certificate_key` in the configuration file.
|
92
100
|
`ssl_port`::
|
Binary file
|
@@ -1,5 +1,6 @@
|
|
1
1
|
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
|
2
2
|
<!-- Created with Inkscape (http://www.inkscape.org/) -->
|
3
|
+
|
3
4
|
<svg
|
4
5
|
xmlns:dc="http://purl.org/dc/elements/1.1/"
|
5
6
|
xmlns:cc="http://creativecommons.org/ns#"
|
@@ -13,11 +14,11 @@
|
|
13
14
|
height="275"
|
14
15
|
id="svg2"
|
15
16
|
sodipodi:version="0.32"
|
16
|
-
inkscape:version="0.
|
17
|
-
sodipodi:docname="
|
17
|
+
inkscape:version="0.48.2 r9819"
|
18
|
+
sodipodi:docname="direct_spawning.svg"
|
18
19
|
inkscape:output_extension="org.inkscape.output.svg.inkscape"
|
19
20
|
version="1.0"
|
20
|
-
inkscape:export-filename="/
|
21
|
+
inkscape:export-filename="/Users/hongli/Projects/passenger/doc/images/direct_spawning.png"
|
21
22
|
inkscape:export-xdpi="73.972603"
|
22
23
|
inkscape:export-ydpi="73.972603">
|
23
24
|
<defs
|
@@ -79,8 +80,9 @@
|
|
79
80
|
showgrid="false"
|
80
81
|
inkscape:window-width="1280"
|
81
82
|
inkscape:window-height="703"
|
82
|
-
inkscape:window-x="
|
83
|
-
inkscape:window-y="
|
83
|
+
inkscape:window-x="73"
|
84
|
+
inkscape:window-y="28"
|
85
|
+
inkscape:window-maximized="0" />
|
84
86
|
<metadata
|
85
87
|
id="metadata7">
|
86
88
|
<rdf:RDF>
|
@@ -89,6 +91,7 @@
|
|
89
91
|
<dc:format>image/svg+xml</dc:format>
|
90
92
|
<dc:type
|
91
93
|
rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
|
94
|
+
<dc:title></dc:title>
|
92
95
|
</cc:Work>
|
93
96
|
</rdf:RDF>
|
94
97
|
</metadata>
|
@@ -161,15 +164,15 @@
|
|
161
164
|
style="font-weight:normal;-inkscape-font-specification:Arial">Other memory</tspan></text>
|
162
165
|
<text
|
163
166
|
xml:space="preserve"
|
164
|
-
style="font-size:66.72337341px;font-style:normal;font-weight:normal;fill:#000000;fill-opacity:1;stroke:none;
|
165
|
-
x="
|
167
|
+
style="font-size:66.72337341px;font-style:normal;font-weight:normal;fill:#000000;fill-opacity:1;stroke:none;font-family:Bitstream Vera Sans"
|
168
|
+
x="285.86664"
|
166
169
|
y="45.084782"
|
167
170
|
id="text3185"><tspan
|
168
171
|
sodipodi:role="line"
|
169
172
|
id="tspan3187"
|
170
|
-
x="
|
173
|
+
x="285.86664"
|
171
174
|
y="45.084782"
|
172
|
-
style="font-size:23.35318184px;font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial">
|
175
|
+
style="font-size:23.35318184px;font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial">App process 1</tspan></text>
|
173
176
|
<rect
|
174
177
|
style="fill:url(#linearGradient3213);fill-opacity:1;stroke:none"
|
175
178
|
id="rect3189"
|
@@ -235,14 +238,14 @@
|
|
235
238
|
style="font-weight:normal;-inkscape-font-specification:Arial">Other memory</tspan></text>
|
236
239
|
<text
|
237
240
|
xml:space="preserve"
|
238
|
-
style="font-size:66.72337341px;font-style:normal;font-weight:normal;fill:#000000;fill-opacity:1;stroke:none;
|
239
|
-
x="
|
241
|
+
style="font-size:66.72337341px;font-style:normal;font-weight:normal;fill:#000000;fill-opacity:1;stroke:none;font-family:Bitstream Vera Sans"
|
242
|
+
x="285.86664"
|
240
243
|
y="175.08478"
|
241
244
|
id="text3209"><tspan
|
242
245
|
sodipodi:role="line"
|
243
246
|
id="tspan3211"
|
244
|
-
x="
|
247
|
+
x="285.86664"
|
245
248
|
y="175.08478"
|
246
|
-
style="font-size:23.35318184px;font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial">
|
249
|
+
style="font-size:23.35318184px;font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial">App process 2</tspan></text>
|
247
250
|
</g>
|
248
251
|
</svg>
|
Binary file
|
Binary file
|
Binary file
|
@@ -1,5 +1,6 @@
|
|
1
1
|
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
|
2
2
|
<!-- Created with Inkscape (http://www.inkscape.org/) -->
|
3
|
+
|
3
4
|
<svg
|
4
5
|
xmlns:dc="http://purl.org/dc/elements/1.1/"
|
5
6
|
xmlns:cc="http://creativecommons.org/ns#"
|
@@ -13,11 +14,11 @@
|
|
13
14
|
height="420"
|
14
15
|
id="svg3235"
|
15
16
|
sodipodi:version="0.32"
|
16
|
-
inkscape:version="0.
|
17
|
-
sodipodi:docname="
|
17
|
+
inkscape:version="0.48.2 r9819"
|
18
|
+
sodipodi:docname="smart_spawning.svg"
|
18
19
|
inkscape:output_extension="org.inkscape.output.svg.inkscape"
|
19
20
|
version="1.0"
|
20
|
-
inkscape:export-filename="/
|
21
|
+
inkscape:export-filename="/Users/hongli/Projects/passenger/doc/images/smart_spawning.png"
|
21
22
|
inkscape:export-xdpi="78.947365"
|
22
23
|
inkscape:export-ydpi="78.947365">
|
23
24
|
<defs
|
@@ -105,15 +106,16 @@
|
|
105
106
|
inkscape:pageopacity="1"
|
106
107
|
inkscape:pageshadow="2"
|
107
108
|
inkscape:zoom="1"
|
108
|
-
inkscape:cx="
|
109
|
-
inkscape:cy="
|
109
|
+
inkscape:cx="406.58261"
|
110
|
+
inkscape:cy="159.47704"
|
110
111
|
inkscape:document-units="px"
|
111
112
|
inkscape:current-layer="layer1"
|
112
113
|
showgrid="false"
|
113
114
|
inkscape:window-width="1280"
|
114
115
|
inkscape:window-height="703"
|
115
116
|
inkscape:window-x="0"
|
116
|
-
inkscape:window-y="72"
|
117
|
+
inkscape:window-y="72"
|
118
|
+
inkscape:window-maximized="0" />
|
117
119
|
<metadata
|
118
120
|
id="metadata3240">
|
119
121
|
<rdf:RDF>
|
@@ -122,6 +124,7 @@
|
|
122
124
|
<dc:format>image/svg+xml</dc:format>
|
123
125
|
<dc:type
|
124
126
|
rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
|
127
|
+
<dc:title></dc:title>
|
125
128
|
</cc:Work>
|
126
129
|
</rdf:RDF>
|
127
130
|
</metadata>
|
@@ -211,14 +214,14 @@
|
|
211
214
|
<text
|
212
215
|
id="text3185"
|
213
216
|
y="150.08478"
|
214
|
-
x="
|
215
|
-
style="font-size:66.72337341px;font-style:normal;font-weight:normal;fill:#000000;fill-opacity:1;stroke:none;
|
217
|
+
x="305.36664"
|
218
|
+
style="font-size:66.72337341px;font-style:normal;font-weight:normal;fill:#000000;fill-opacity:1;stroke:none;font-family:Bitstream Vera Sans"
|
216
219
|
xml:space="preserve"><tspan
|
217
220
|
style="font-size:23.35318184px;font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial"
|
218
221
|
y="150.08478"
|
219
|
-
x="
|
222
|
+
x="305.36664"
|
220
223
|
id="tspan3187"
|
221
|
-
sodipodi:role="line">
|
224
|
+
sodipodi:role="line">App process 1</tspan></text>
|
222
225
|
</g>
|
223
226
|
<g
|
224
227
|
id="g3835"
|
@@ -252,15 +255,15 @@
|
|
252
255
|
sodipodi:role="line">Other memory</tspan></text>
|
253
256
|
<text
|
254
257
|
id="text3807"
|
255
|
-
y="
|
256
|
-
x="
|
257
|
-
style="font-size:66.72337341px;font-style:normal;font-weight:normal;fill:#000000;fill-opacity:1;stroke:none;
|
258
|
+
y="302.58478"
|
259
|
+
x="303.36664"
|
260
|
+
style="font-size:66.72337341px;font-style:normal;font-weight:normal;fill:#000000;fill-opacity:1;stroke:none;font-family:Bitstream Vera Sans"
|
258
261
|
xml:space="preserve"><tspan
|
259
262
|
style="font-size:23.35318184px;font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial"
|
260
|
-
y="
|
261
|
-
x="
|
263
|
+
y="302.58478"
|
264
|
+
x="303.36664"
|
262
265
|
id="tspan3809"
|
263
|
-
sodipodi:role="line">
|
266
|
+
sodipodi:role="line">App process 2</tspan></text>
|
264
267
|
</g>
|
265
268
|
<g
|
266
269
|
id="g3843"
|
@@ -295,14 +298,14 @@
|
|
295
298
|
<text
|
296
299
|
id="text3821"
|
297
300
|
y="225.58478"
|
298
|
-
x="
|
299
|
-
style="font-size:66.72337341px;font-style:normal;font-weight:normal;fill:#000000;fill-opacity:1;stroke:none;
|
301
|
+
x="99.366638"
|
302
|
+
style="font-size:66.72337341px;font-style:normal;font-weight:normal;fill:#000000;fill-opacity:1;stroke:none;font-family:Bitstream Vera Sans"
|
300
303
|
xml:space="preserve"><tspan
|
301
304
|
style="font-size:23.35318184px;font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial"
|
302
305
|
y="225.58478"
|
303
|
-
x="
|
306
|
+
x="99.366638"
|
304
307
|
id="tspan3823"
|
305
|
-
sodipodi:role="line">
|
308
|
+
sodipodi:role="line">Preloader</tspan></text>
|
306
309
|
</g>
|
307
310
|
<path
|
308
311
|
style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:4;stroke-linecap:butt;stroke-linejoin:miter;marker-end:url(#Arrow1Mend);stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
|
Binary file
|
Binary file
|
@@ -1,121 +1,103 @@
|
|
1
1
|
[[spawning_methods_explained]]
|
2
|
-
== Appendix C: Spawning methods explained
|
2
|
+
== Appendix C: Spawning methods explained
|
3
3
|
|
4
4
|
At its core, Phusion Passenger is an HTTP proxy and process manager. It spawns
|
5
|
-
|
6
|
-
'backend processes'), and forwards incoming HTTP request to one of the worker
|
7
|
-
processes.
|
5
|
+
application processes and forwards incoming HTTP request to one of them.
|
8
6
|
|
9
|
-
While this may sound simple, there's not just one way to spawn
|
7
|
+
While this may sound simple, there's not just one way to spawn application processes.
|
10
8
|
Let's go over the different spawning methods. For simplicity's sake, let's
|
11
9
|
assume that we're only talking about Ruby on Rails applications.
|
12
10
|
|
13
|
-
=== The most straightforward and traditional way: direct spawning
|
11
|
+
=== The most straightforward and traditional way: direct spawning
|
14
12
|
|
15
13
|
Phusion Passenger could create a new Ruby process, which will then load the
|
16
14
|
Rails application along with the entire Rails framework. This process will then
|
17
15
|
enter an request handling main loop.
|
18
16
|
|
19
|
-
This is the most straightforward way to spawn
|
20
|
-
|
21
|
-
what mongrel_cluster performs: it creates N worker processes, each which loads
|
22
|
-
a full copy of the Rails application and the Rails framework in memory. The Thin
|
23
|
-
application server employs pretty much the same approach.
|
17
|
+
This is the most straightforward way to spawn processes, and each process contains
|
18
|
+
a full copy of the Rails application and the Rails framework in memory.
|
24
19
|
|
25
|
-
|
26
|
-
from mongrel_cluster. Mongrel_cluster creates entirely new Ruby processes. In
|
27
|
-
programmers jargon, mongrel_cluster creates new Ruby processes by forking the
|
28
|
-
current process and exec()-ing a new Ruby interpreter. Phusion Passenger on the
|
29
|
-
other hand creates processes that reuse the already loaded Ruby interpreter. In
|
30
|
-
programmers jargon, Phusion Passenger calls fork(), but not exec().
|
20
|
+
=== The smart spawning method
|
31
21
|
|
32
|
-
|
33
|
-
|
34
|
-
NOTE: Smart spawning is supported for all Ruby applications but not for WSGI applications.
|
22
|
+
NOTE: Smart spawning is only supported for Ruby applications. It's not supported for other languages.
|
35
23
|
|
36
24
|
While direct spawning works well, it's not as efficient as it could be
|
37
|
-
because each
|
25
|
+
because each process has its own private copy of the Rails application
|
38
26
|
as well as the Rails framework. This wastes memory as well as startup time.
|
39
27
|
|
40
|
-
image:images/direct_spawning.png[
|
41
|
-
'Figure:
|
28
|
+
image:images/direct_spawning.png[Application processes and direct spawning] +
|
29
|
+
'Figure: Application processes and direct spawning. Each process has its
|
42
30
|
own private copy of the application code and Rails framework code.'
|
43
31
|
|
44
|
-
It is possible to make the different
|
32
|
+
It is possible to make the different processes share the memory occupied
|
45
33
|
by application and Rails framework code, by utilizing so-called
|
46
34
|
copy-on-write semantics of the virtual memory system on modern operating
|
47
35
|
systems. As a side effect, the startup time is also reduced. This is technique
|
48
36
|
is exploited by Phusion Passenger's 'smart' spawn method.
|
49
37
|
|
50
|
-
|
38
|
+
The 'smart' spawn method is similar to Unicorn's `preload_app true` feature.
|
39
|
+
|
40
|
+
==== How it works
|
51
41
|
|
52
42
|
When the 'smart' spawn method is being used, Phusion Passenger will first
|
53
|
-
create a so-called '
|
54
|
-
entire Rails application along with the Rails framework, by
|
55
|
-
|
56
|
-
it will instruct the
|
57
|
-
|
58
|
-
|
59
|
-
|
60
|
-
10 times faster than loading the
|
61
|
-
|
62
|
-
|
63
|
-
|
64
|
-
|
65
|
-
|
66
|
-
|
67
|
-
|
68
|
-
'Figure: Worker processes and the smart spawn method. All worker processes,
|
69
|
-
as well as the ApplicationSpawner, share the same application code and Rails
|
43
|
+
create a so-called 'preloader' process. This process loads the
|
44
|
+
entire Rails application along with the Rails framework, by evaluating
|
45
|
+
`config.ru`. Then, whenever Phusion Passenger needs a new application process,
|
46
|
+
it will instruct the preloader to create one. The preloader then then spawns
|
47
|
+
a child process, which is an exact virtual copy of itself. This child process
|
48
|
+
therefore already has the application code and the Rails framework code in memory.
|
49
|
+
|
50
|
+
Creating a process like this is very fast, about 10 times faster than loading the
|
51
|
+
Rails application/framework from scratch. On top of that, the OS also applies an
|
52
|
+
optimization called 'copy-on-write'. This means that all memory that the child
|
53
|
+
process hasn't modified, is shared with the parent process.
|
54
|
+
|
55
|
+
image:images/smart_spawning.png[] +
|
56
|
+
'Figure: Application processes and the smart spawn method. All processes,
|
57
|
+
as well as the preloader, share the same application code and Rails
|
70
58
|
framework code.'
|
71
59
|
|
72
|
-
|
73
|
-
|
60
|
+
However, Ruby can only leverage this copy-on-write optimization if its garbage
|
61
|
+
collector is friendly. This is only the case starting from Ruby 2.0.0. Earlier
|
62
|
+
versions cannot leverage copy-on-write optimizations.
|
74
63
|
|
75
|
-
|
64
|
+
Note that preloader processes have an idle timeout just like application processes.
|
65
|
+
If a preloader hasn't been instructed to do anything for a while, it will be shutdown
|
66
|
+
in order to conserve memory. This idle timeout is configurable.
|
76
67
|
|
77
|
-
|
78
|
-
both vendored Rails frameworks are the same version. So for efficiency reasons
|
79
|
-
we don't recommend vendoring Rails.
|
80
|
-
- ApplicationSpawner servers have an idle timeout just like worker processes.
|
81
|
-
If an ApplicationSpawner/FrameworkSpawner server hasn't been instructed to do
|
82
|
-
anything for a while, it will be shutdown in order to conserve memory. This
|
83
|
-
idle timeout is configurable.
|
68
|
+
==== Summary of benefits
|
84
69
|
|
85
|
-
|
70
|
+
Suppose that Phusion Passenger needs a process for an application
|
71
|
+
that uses Rails 4.1.0.
|
86
72
|
|
87
|
-
|
88
|
-
|
89
|
-
|
90
|
-
|
91
|
-
|
92
|
-
about 10 times faster than direct spawning. This worker process will also
|
93
|
-
share application and Rails framework code memory with the ApplicationSpawner
|
94
|
-
server and the worker processes that had been spawned by this ApplicationSpawner
|
95
|
-
server.
|
73
|
+
If the 'smart' spawning method is used, and a preloader for this application is
|
74
|
+
already running, then process creation time is about 10 times faster than direct
|
75
|
+
spawning. This process will also share application and Rails framework code memory
|
76
|
+
with the preloader, as well as with other processes that have been spawned by the
|
77
|
+
same preloader.
|
96
78
|
|
97
79
|
In practice, the smart spawning method could mean a memory saving of about 33%,
|
98
|
-
assuming that your Ruby interpreter is
|
80
|
+
assuming that your Ruby interpreter is copy-on-write friendly.
|
99
81
|
|
100
|
-
Of course, smart spawning is not without
|
101
|
-
|
82
|
+
Of course, smart spawning is not without caveats. But if you understand the
|
83
|
+
caveats you can easily reap the benefits of smart spawning.
|
102
84
|
|
103
|
-
=== Smart spawning
|
85
|
+
=== Smart spawning caveat #1: unintentional file descriptor sharing
|
104
86
|
|
105
|
-
Because
|
106
|
-
|
107
|
-
|
87
|
+
Because application processes are created by forking from a preloader process,
|
88
|
+
it will share all file descriptors that are opened by the
|
89
|
+
preloader process. (This is part of the semantics of the Unix
|
108
90
|
'fork()' system call. You might want to Google it if you're not familiar with
|
109
91
|
it.) A file descriptor is a handle which can be an opened file, an opened socket
|
110
|
-
connection, a pipe, etc. If different
|
92
|
+
connection, a pipe, etc. If different application processes write to such a file
|
111
93
|
descriptor at the same time, then their write calls will be interleaved, which
|
112
94
|
may potentially cause problems.
|
113
95
|
|
114
96
|
The problem commonly involves socket connections that are unintentionally being
|
115
97
|
shared. You can fix it by closing and reestablishing the connection when Phusion
|
116
|
-
Passenger is creating a new
|
98
|
+
Passenger is creating a new application process. Phusion Passenger provides the API
|
117
99
|
call `PhusionPassenger.on_event(:starting_worker_process)` to do so. So you
|
118
|
-
could insert the following code in your '
|
100
|
+
could insert the following code in your 'config.ru':
|
119
101
|
|
120
102
|
[source, ruby]
|
121
103
|
-----------------------------------------
|
@@ -132,61 +114,60 @@ end
|
|
132
114
|
-----------------------------------------
|
133
115
|
|
134
116
|
Note that Phusion Passenger automatically reestablishes the connection to the
|
135
|
-
database upon creating a new
|
117
|
+
database upon creating a new application process, which is why you normally do not
|
136
118
|
encounter any database issues when using smart spawning mode.
|
137
119
|
|
138
|
-
==== Example 1: Memcached connection sharing (harmful)
|
120
|
+
==== Example 1: Memcached connection sharing (harmful)
|
139
121
|
|
140
122
|
Suppose we have a Rails application that connects to a Memcached server in
|
141
|
-
'environment.rb'. This causes the
|
123
|
+
'environment.rb'. This causes the preloader to have a socket connection
|
142
124
|
(file descriptor) to the Memcached server, as shown in the following figure:
|
143
125
|
|
144
126
|
+--------------------+
|
145
|
-
|
|
127
|
+
| Preloader |-----------[Memcached server]
|
146
128
|
+--------------------+
|
147
129
|
|
148
|
-
Phusion Passenger then proceeds with creating a new Rails
|
130
|
+
Phusion Passenger then proceeds with creating a new Rails application process, which
|
149
131
|
is to process incoming HTTP requests. The result will look like this:
|
150
132
|
|
151
133
|
+--------------------+
|
152
|
-
|
|
134
|
+
| Preloader |------+----[Memcached server]
|
153
135
|
+--------------------+ |
|
154
136
|
|
|
155
137
|
+--------------------+ |
|
156
|
-
|
|
138
|
+
| App process 1 |-----/
|
157
139
|
+--------------------+
|
158
140
|
|
159
141
|
Since a 'fork()' makes a (virtual) complete copy of a process, all its file
|
160
|
-
descriptors will be copied as well. What we see here is that
|
161
|
-
and
|
142
|
+
descriptors will be copied as well. What we see here is that Preloader
|
143
|
+
and App process 1 both share the same connection to Memcached.
|
162
144
|
|
163
|
-
Now supposed that your site gets
|
164
|
-
spawn another
|
165
|
-
result is now as follows:
|
145
|
+
Now supposed that your site gets a sudden large surge of traffic, and Phusion Passenger needs to
|
146
|
+
spawn another process. It does so by forking Preloader. The result is now as follows:
|
166
147
|
|
167
148
|
+--------------------+
|
168
|
-
|
|
149
|
+
| Preloader |------+----[Memcached server]
|
169
150
|
+--------------------+ |
|
170
151
|
|
|
171
152
|
+--------------------+ |
|
172
|
-
|
|
153
|
+
| App process 1 |-----/|
|
173
154
|
+--------------------+ |
|
174
155
|
|
|
175
156
|
+--------------------+ |
|
176
|
-
|
|
157
|
+
| App process 2 |-----/
|
177
158
|
+--------------------+
|
178
159
|
|
179
|
-
As you can see,
|
160
|
+
As you can see, App process 1 and App process 2 have the same Memcached
|
180
161
|
connection.
|
181
162
|
|
182
163
|
Suppose that users Joe and Jane visit your website at the same time. Joe's
|
183
|
-
request is handled by
|
184
|
-
process 2. Both
|
164
|
+
request is handled by App process 1, and Jane's request is handled by App
|
165
|
+
process 2. Both application processes want to fetch something from Memcached. Suppose
|
185
166
|
that in order to do that, both handlers need to send a "FETCH" command to Memcached.
|
186
167
|
|
187
|
-
But suppose that, after
|
188
|
-
occurs, and
|
189
|
-
well. If
|
168
|
+
But suppose that, after App process 1 having only sent "FE", a context switch
|
169
|
+
occurs, and App process 2 starts sending a "FETCH" command to Memcached as
|
170
|
+
well. If App process 2 succeeds in sending only one bye, 'F', then Memcached
|
190
171
|
will receive a command which begins with "FEF", a command that it does not
|
191
172
|
recognize. In other words: the data from both handlers get interleaved. And thus
|
192
173
|
Memcached is forced to handle this as an error.
|
@@ -194,22 +175,22 @@ Memcached is forced to handle this as an error.
|
|
194
175
|
This problem can be solved by reestablishing the connection to Memcached after forking:
|
195
176
|
|
196
177
|
+--------------------+
|
197
|
-
|
|
178
|
+
| Preloader |------+----[Memcached server]
|
198
179
|
+--------------------+ | |
|
199
180
|
| |
|
200
181
|
+--------------------+ | |
|
201
|
-
|
|
182
|
+
| App process 1 |-----/| |
|
202
183
|
+--------------------+ | | <--- created this
|
203
184
|
X | new
|
204
185
|
| connection
|
205
186
|
X <-- closed this |
|
206
187
|
+--------------------+ | old |
|
207
|
-
|
|
188
|
+
| App process 2 |-----/ connection |
|
208
189
|
+--------------------+ |
|
209
190
|
| |
|
210
191
|
+-------------------------------------+
|
211
192
|
|
212
|
-
|
193
|
+
App process 2 now has its own, separate communication channel with Memcached.
|
213
194
|
The code in 'environment.rb' looks like this:
|
214
195
|
|
215
196
|
[source, ruby]
|
@@ -226,7 +207,7 @@ if defined?(PhusionPassenger)
|
|
226
207
|
end
|
227
208
|
-----------------------------------------
|
228
209
|
|
229
|
-
==== Example 2: Log file sharing (not harmful)
|
210
|
+
==== Example 2: Log file sharing (not harmful)
|
230
211
|
|
231
212
|
There are also cases in which unintentional file descriptor sharing is not harmful.
|
232
213
|
One such case is log file file descriptor sharing. Even if two processes write
|
@@ -238,12 +219,12 @@ must synchronize write access via an inter-process synchronization mechanism,
|
|
238
219
|
such as file locks. Reopening the log file, like you would have done in the
|
239
220
|
Memcached example, doesn't help.
|
240
221
|
|
241
|
-
=== Smart spawning
|
222
|
+
=== Smart spawning caveat #2: the need to revive threads
|
242
223
|
|
243
224
|
Another part of the 'fork()' system call's semantics is the fact that threads
|
244
225
|
disappear after a fork call. So if you've created any threads in environment.rb,
|
245
|
-
then those threads will no longer be running in newly created
|
246
|
-
You need to revive them when a new
|
226
|
+
then those threads will no longer be running in newly created application process.
|
227
|
+
You need to revive them when a new process is created. Use the
|
247
228
|
`:starting_worker_process` event that Phusion Passenger provides, like this:
|
248
229
|
|
249
230
|
[source, ruby]
|
@@ -259,23 +240,3 @@ if defined?(PhusionPassenger)
|
|
259
240
|
end
|
260
241
|
end
|
261
242
|
-----------------------------------------
|
262
|
-
|
263
|
-
=== Smart spawning gotcha #3: code load order ===
|
264
|
-
|
265
|
-
This gotcha is only applicable to the 'smart' spawn method, not the 'smart-lv2'
|
266
|
-
spawn method.
|
267
|
-
|
268
|
-
If your application expects the Rails framework to be not loaded during the
|
269
|
-
beginning of 'environment.rb', then it can cause problems when an
|
270
|
-
ApplicationSpawner is created from a FrameworkSpawner, which already has the
|
271
|
-
Rails framework loaded. The most common case is when applications try to patch
|
272
|
-
Rails by dropping a modified file that has the same name as Rails's own file,
|
273
|
-
in a path that comes earlier in the Ruby search path.
|
274
|
-
|
275
|
-
For example, suppose that we have an application which has a patched version
|
276
|
-
of 'active_record/base.rb' located in 'RAILS_ROOT/lib/patches', and
|
277
|
-
'RAILS_ROOT/lib/patches' comes first in the Ruby load path. When conservative
|
278
|
-
spawning is used, the patched version of 'base.rb' is properly loaded. When
|
279
|
-
'smart' (not 'smart-lv2') spawning is used, the original 'base.rb' is used
|
280
|
-
because it was already loaded, so a subsequent `require "active_record/base"`
|
281
|
-
has no effect.
|