passenger 4.0.44 → 4.0.45

Sign up to get free protection for your applications and to get access to all the features.

Potentially problematic release.


This version of passenger might be problematic. Click here for more details.

Files changed (110) hide show
  1. checksums.yaml +8 -8
  2. checksums.yaml.gz.asc +7 -7
  3. data.tar.gz.asc +7 -7
  4. data/.travis.yml +3 -0
  5. data/CHANGELOG +31 -0
  6. data/CONTRIBUTING.md +70 -10
  7. data/CONTRIBUTORS +4 -0
  8. data/README.md +1 -1
  9. data/Vagrantfile +50 -0
  10. data/bin/passenger-install-nginx-module +7 -2
  11. data/build/basics.rb +4 -1
  12. data/build/documentation.rb +6 -0
  13. data/build/node_tests.rb +7 -1
  14. data/build/packaging.rb +5 -0
  15. data/build/test_basics.rb +3 -3
  16. data/debian.template/copyright +1 -1
  17. data/debian.template/passenger.manpages +0 -1
  18. data/dev/rack.test/config.ru +5 -0
  19. data/dev/rack.test/public/asset.txt +1 -0
  20. data/dev/vagrant/apache_default_site.conf +35 -0
  21. data/dev/vagrant/apache_passenger.conf +5 -0
  22. data/dev/vagrant/apache_passenger.load +1 -0
  23. data/dev/vagrant/apache_ports.conf +24 -0
  24. data/dev/vagrant/apache_rack_test.conf +9 -0
  25. data/dev/vagrant/bashrc +21 -0
  26. data/dev/vagrant/nginx.conf +39 -0
  27. data/dev/vagrant/nginx_rakefile +34 -0
  28. data/dev/vagrant/nginx_start +32 -0
  29. data/dev/vagrant/provision.sh +115 -0
  30. data/dev/vagrant/sudoers.conf +5 -0
  31. data/doc/Design and Architecture.txt +515 -0
  32. data/doc/DeveloperQuickstart.md +70 -0
  33. data/doc/Users guide Apache.idmap.txt +24 -18
  34. data/doc/Users guide Apache.txt +200 -62
  35. data/doc/Users guide Nginx.idmap.txt +53 -45
  36. data/doc/Users guide Nginx.txt +501 -360
  37. data/doc/Users guide Standalone.txt +8 -0
  38. data/doc/images/direct_spawning.png +0 -0
  39. data/doc/images/direct_spawning.svg +16 -13
  40. data/doc/images/helper_agent_core_architecture.png +0 -0
  41. data/doc/images/passenger_architecture_overview.png +0 -0
  42. data/doc/images/smart_spawning.png +0 -0
  43. data/doc/images/{smart.svg → smart_spawning.svg} +23 -20
  44. data/doc/images/spawning_preparation_work.png +0 -0
  45. data/doc/images/startup_sequence.png +0 -0
  46. data/doc/users_guide_snippets/appendix_c_spawning_methods.txt +82 -121
  47. data/doc/users_guide_snippets/environment_variables.txt +1 -1
  48. data/doc/users_guide_snippets/support_information.txt +2 -0
  49. data/doc/users_guide_snippets/tips.txt +117 -9
  50. data/ext/apache2/Configuration.hpp +4 -2
  51. data/ext/apache2/ConfigurationCommands.cpp +14 -0
  52. data/ext/apache2/ConfigurationFields.hpp +4 -0
  53. data/ext/apache2/ConfigurationSetters.cpp +22 -0
  54. data/ext/apache2/CreateDirConfig.cpp +2 -0
  55. data/ext/apache2/Hooks.cpp +30 -14
  56. data/ext/apache2/MergeDirConfig.cpp +14 -0
  57. data/ext/apache2/SetHeaders.cpp +8 -0
  58. data/ext/common/ApplicationPool2/AppTypes.cpp +6 -1
  59. data/ext/common/ApplicationPool2/Implementation.cpp +1 -1
  60. data/ext/common/ApplicationPool2/Session.h +1 -1
  61. data/ext/common/Constants.h +9 -7
  62. data/ext/common/Utils/HttpHeaderBufferer.h +23 -4
  63. data/ext/common/Utils/StrIntUtils.h +35 -0
  64. data/ext/common/Utils/StringScanning.h +4 -10
  65. data/ext/common/agents/HelperAgent/RequestHandler.h +90 -49
  66. data/ext/nginx/CacheLocationConfig.c +40 -0
  67. data/ext/nginx/ConfigurationCommands.c +20 -0
  68. data/ext/nginx/ConfigurationFields.h +4 -0
  69. data/ext/nginx/ContentHandler.c +1 -1
  70. data/ext/nginx/CreateLocationConfig.c +9 -0
  71. data/ext/nginx/MergeLocationConfig.c +12 -0
  72. data/ext/nginx/config +2 -2
  73. data/ext/nginx/ngx_http_passenger_module.c +4 -4
  74. data/helper-scripts/node-loader.js +40 -27
  75. data/lib/phusion_passenger.rb +1 -1
  76. data/lib/phusion_passenger/apache2/config_options.rb +14 -2
  77. data/lib/phusion_passenger/constants.rb +7 -6
  78. data/lib/phusion_passenger/loader_shared_helpers.rb +11 -1
  79. data/lib/phusion_passenger/nginx/config_options.rb +8 -0
  80. data/lib/phusion_passenger/packaging.rb +8 -3
  81. data/lib/phusion_passenger/platform_info/apache.rb +3 -0
  82. data/lib/phusion_passenger/platform_info/ruby.rb +4 -1
  83. data/lib/phusion_passenger/standalone/command.rb +0 -1
  84. data/lib/phusion_passenger/standalone/package_runtime_command.rb +1 -0
  85. data/lib/phusion_passenger/standalone/start_command.rb +80 -62
  86. data/lib/phusion_passenger/standalone/status_command.rb +1 -0
  87. data/lib/phusion_passenger/standalone/stop_command.rb +1 -0
  88. data/man/passenger-config.1 +1 -1
  89. data/man/passenger-memory-stats.8 +1 -1
  90. data/man/passenger-status.8 +1 -1
  91. data/npm-shrinkwrap.json +229 -0
  92. data/package.json +28 -0
  93. data/resources/templates/standalone/config.erb +2 -0
  94. data/rpm/Vagrantfile +0 -3
  95. data/test/config.json.vagrant +30 -0
  96. data/test/cxx/HttpHeaderBuffererTest.cpp +64 -10
  97. data/test/cxx/RequestHandlerTest.cpp +35 -13
  98. data/test/integration_tests/apache2_tests.rb +1 -0
  99. data/test/stub/node/app.js +26 -18
  100. metadata +28 -13
  101. metadata.gz.asc +7 -7
  102. data/doc/Architectural overview.idmap.txt +0 -36
  103. data/doc/Architectural overview.txt +0 -410
  104. data/doc/images/smart.png +0 -0
  105. data/ext/common/ApplicationPool2/README.md +0 -56
  106. data/man/passenger-stress-test.1 +0 -43
  107. data/node_lib/phusion_passenger/httplib_emulation.js +0 -215
  108. data/node_lib/phusion_passenger/request_handler.js +0 -73
  109. data/node_lib/phusion_passenger/session_protocol_parser.js +0 -113
  110. data/test/node/httplib_emulation_spec.js +0 -623
@@ -50,6 +50,10 @@ Most configuration is done by customizing the arguments passed to the `passenger
50
50
  +
51
51
  You can use `--friendly-error-pages` and `--no-friendly-error-pages` to explicitly enable or disable this feature, respectively.
52
52
 
53
+ `--sticky-sessions`::
54
+ Enables link:Users%20guide%20Nginx.html#PassengerStickySessions[sticky sessions]. Required for SockJS, Socket.io, faye-websocket and Meteor JS applications.
55
+ `--sticky-sessions-cookie-name NAME`::
56
+ Allows customizing the sticky session cookie name.
53
57
  `--ssl`::
54
58
  Enables SSL support. If this is set, you must also set `--ssl-certificate` and `--ssl-certificate-key` to the SSL certificate and key files, respectively.
55
59
  `--ssl-port`::
@@ -87,6 +91,10 @@ The following configuration options are supported:
87
91
  Equivalent to the `--min-instances` command line option.
88
92
  `spawn_method`::
89
93
  Equivalent to the `--spawn-method` command line option.
94
+ `sticky_sessions`::
95
+ Equivalent to the `--sticky-sessions` command line option.
96
+ `sticky_sessions_cookie_name`::
97
+ Equivalent to the `--sticky-sessions-cookie-name` command line option.
90
98
  `ssl`::
91
99
  Equivalent to the `--ssl` command line option. When given, you must also set `ssl_certificate` and `ssl_certificate_key` in the configuration file.
92
100
  `ssl_port`::
Binary file
@@ -1,5 +1,6 @@
1
1
  <?xml version="1.0" encoding="UTF-8" standalone="no"?>
2
2
  <!-- Created with Inkscape (http://www.inkscape.org/) -->
3
+
3
4
  <svg
4
5
  xmlns:dc="http://purl.org/dc/elements/1.1/"
5
6
  xmlns:cc="http://creativecommons.org/ns#"
@@ -13,11 +14,11 @@
13
14
  height="275"
14
15
  id="svg2"
15
16
  sodipodi:version="0.32"
16
- inkscape:version="0.46"
17
- sodipodi:docname="conservative_spawning.svg"
17
+ inkscape:version="0.48.2 r9819"
18
+ sodipodi:docname="direct_spawning.svg"
18
19
  inkscape:output_extension="org.inkscape.output.svg.inkscape"
19
20
  version="1.0"
20
- inkscape:export-filename="/home/hongli/Projects/mod_rails/doc/images/conservative_spawning.png"
21
+ inkscape:export-filename="/Users/hongli/Projects/passenger/doc/images/direct_spawning.png"
21
22
  inkscape:export-xdpi="73.972603"
22
23
  inkscape:export-ydpi="73.972603">
23
24
  <defs
@@ -79,8 +80,9 @@
79
80
  showgrid="false"
80
81
  inkscape:window-width="1280"
81
82
  inkscape:window-height="703"
82
- inkscape:window-x="0"
83
- inkscape:window-y="72" />
83
+ inkscape:window-x="73"
84
+ inkscape:window-y="28"
85
+ inkscape:window-maximized="0" />
84
86
  <metadata
85
87
  id="metadata7">
86
88
  <rdf:RDF>
@@ -89,6 +91,7 @@
89
91
  <dc:format>image/svg+xml</dc:format>
90
92
  <dc:type
91
93
  rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
94
+ <dc:title></dc:title>
92
95
  </cc:Work>
93
96
  </rdf:RDF>
94
97
  </metadata>
@@ -161,15 +164,15 @@
161
164
  style="font-weight:normal;-inkscape-font-specification:Arial">Other memory</tspan></text>
162
165
  <text
163
166
  xml:space="preserve"
164
- style="font-size:66.72337341px;font-style:normal;font-weight:normal;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;font-family:Bitstream Vera Sans"
165
- x="276.86664"
167
+ style="font-size:66.72337341px;font-style:normal;font-weight:normal;fill:#000000;fill-opacity:1;stroke:none;font-family:Bitstream Vera Sans"
168
+ x="285.86664"
166
169
  y="45.084782"
167
170
  id="text3185"><tspan
168
171
  sodipodi:role="line"
169
172
  id="tspan3187"
170
- x="276.86664"
173
+ x="285.86664"
171
174
  y="45.084782"
172
- style="font-size:23.35318184px;font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial">Worker process 1</tspan></text>
175
+ style="font-size:23.35318184px;font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial">App process 1</tspan></text>
173
176
  <rect
174
177
  style="fill:url(#linearGradient3213);fill-opacity:1;stroke:none"
175
178
  id="rect3189"
@@ -235,14 +238,14 @@
235
238
  style="font-weight:normal;-inkscape-font-specification:Arial">Other memory</tspan></text>
236
239
  <text
237
240
  xml:space="preserve"
238
- style="font-size:66.72337341px;font-style:normal;font-weight:normal;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;font-family:Bitstream Vera Sans"
239
- x="275.86664"
241
+ style="font-size:66.72337341px;font-style:normal;font-weight:normal;fill:#000000;fill-opacity:1;stroke:none;font-family:Bitstream Vera Sans"
242
+ x="285.86664"
240
243
  y="175.08478"
241
244
  id="text3209"><tspan
242
245
  sodipodi:role="line"
243
246
  id="tspan3211"
244
- x="275.86664"
247
+ x="285.86664"
245
248
  y="175.08478"
246
- style="font-size:23.35318184px;font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial">Worker process 2</tspan></text>
249
+ style="font-size:23.35318184px;font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial">App process 2</tspan></text>
247
250
  </g>
248
251
  </svg>
Binary file
@@ -1,5 +1,6 @@
1
1
  <?xml version="1.0" encoding="UTF-8" standalone="no"?>
2
2
  <!-- Created with Inkscape (http://www.inkscape.org/) -->
3
+
3
4
  <svg
4
5
  xmlns:dc="http://purl.org/dc/elements/1.1/"
5
6
  xmlns:cc="http://creativecommons.org/ns#"
@@ -13,11 +14,11 @@
13
14
  height="420"
14
15
  id="svg3235"
15
16
  sodipodi:version="0.32"
16
- inkscape:version="0.46"
17
- sodipodi:docname="smart-lv2.svg"
17
+ inkscape:version="0.48.2 r9819"
18
+ sodipodi:docname="smart_spawning.svg"
18
19
  inkscape:output_extension="org.inkscape.output.svg.inkscape"
19
20
  version="1.0"
20
- inkscape:export-filename="/home/hongli/Projects/mod_rails/doc/images/smart-lv2.png"
21
+ inkscape:export-filename="/Users/hongli/Projects/passenger/doc/images/smart_spawning.png"
21
22
  inkscape:export-xdpi="78.947365"
22
23
  inkscape:export-ydpi="78.947365">
23
24
  <defs
@@ -105,15 +106,16 @@
105
106
  inkscape:pageopacity="1"
106
107
  inkscape:pageshadow="2"
107
108
  inkscape:zoom="1"
108
- inkscape:cx="276.76087"
109
- inkscape:cy="173.93662"
109
+ inkscape:cx="406.58261"
110
+ inkscape:cy="159.47704"
110
111
  inkscape:document-units="px"
111
112
  inkscape:current-layer="layer1"
112
113
  showgrid="false"
113
114
  inkscape:window-width="1280"
114
115
  inkscape:window-height="703"
115
116
  inkscape:window-x="0"
116
- inkscape:window-y="72" />
117
+ inkscape:window-y="72"
118
+ inkscape:window-maximized="0" />
117
119
  <metadata
118
120
  id="metadata3240">
119
121
  <rdf:RDF>
@@ -122,6 +124,7 @@
122
124
  <dc:format>image/svg+xml</dc:format>
123
125
  <dc:type
124
126
  rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
127
+ <dc:title></dc:title>
125
128
  </cc:Work>
126
129
  </rdf:RDF>
127
130
  </metadata>
@@ -211,14 +214,14 @@
211
214
  <text
212
215
  id="text3185"
213
216
  y="150.08478"
214
- x="287.36664"
215
- style="font-size:66.72337341px;font-style:normal;font-weight:normal;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;font-family:Bitstream Vera Sans"
217
+ x="305.36664"
218
+ style="font-size:66.72337341px;font-style:normal;font-weight:normal;fill:#000000;fill-opacity:1;stroke:none;font-family:Bitstream Vera Sans"
216
219
  xml:space="preserve"><tspan
217
220
  style="font-size:23.35318184px;font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial"
218
221
  y="150.08478"
219
- x="287.36664"
222
+ x="305.36664"
220
223
  id="tspan3187"
221
- sodipodi:role="line">Worker process 1</tspan></text>
224
+ sodipodi:role="line">App process 1</tspan></text>
222
225
  </g>
223
226
  <g
224
227
  id="g3835"
@@ -252,15 +255,15 @@
252
255
  sodipodi:role="line">Other memory</tspan></text>
253
256
  <text
254
257
  id="text3807"
255
- y="304.58478"
256
- x="288.36664"
257
- style="font-size:66.72337341px;font-style:normal;font-weight:normal;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;font-family:Bitstream Vera Sans"
258
+ y="302.58478"
259
+ x="303.36664"
260
+ style="font-size:66.72337341px;font-style:normal;font-weight:normal;fill:#000000;fill-opacity:1;stroke:none;font-family:Bitstream Vera Sans"
258
261
  xml:space="preserve"><tspan
259
262
  style="font-size:23.35318184px;font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial"
260
- y="304.58478"
261
- x="288.36664"
263
+ y="302.58478"
264
+ x="303.36664"
262
265
  id="tspan3809"
263
- sodipodi:role="line">Worker process 2</tspan></text>
266
+ sodipodi:role="line">App process 2</tspan></text>
264
267
  </g>
265
268
  <g
266
269
  id="g3843"
@@ -295,14 +298,14 @@
295
298
  <text
296
299
  id="text3821"
297
300
  y="225.58478"
298
- x="43.366638"
299
- style="font-size:66.72337341px;font-style:normal;font-weight:normal;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;font-family:Bitstream Vera Sans"
301
+ x="99.366638"
302
+ style="font-size:66.72337341px;font-style:normal;font-weight:normal;fill:#000000;fill-opacity:1;stroke:none;font-family:Bitstream Vera Sans"
300
303
  xml:space="preserve"><tspan
301
304
  style="font-size:23.35318184px;font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-family:Arial;-inkscape-font-specification:Arial"
302
305
  y="225.58478"
303
- x="43.366638"
306
+ x="99.366638"
304
307
  id="tspan3823"
305
- sodipodi:role="line">ApplicationSpawner</tspan></text>
308
+ sodipodi:role="line">Preloader</tspan></text>
306
309
  </g>
307
310
  <path
308
311
  style="fill:none;fill-rule:evenodd;stroke:#000000;stroke-width:4;stroke-linecap:butt;stroke-linejoin:miter;marker-end:url(#Arrow1Mend);stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
Binary file
@@ -1,121 +1,103 @@
1
1
  [[spawning_methods_explained]]
2
- == Appendix C: Spawning methods explained ==
2
+ == Appendix C: Spawning methods explained
3
3
 
4
4
  At its core, Phusion Passenger is an HTTP proxy and process manager. It spawns
5
- Ruby on Rails/Rack/WSGI worker processes (which may also be referred to as
6
- 'backend processes'), and forwards incoming HTTP request to one of the worker
7
- processes.
5
+ application processes and forwards incoming HTTP request to one of them.
8
6
 
9
- While this may sound simple, there's not just one way to spawn worker processes.
7
+ While this may sound simple, there's not just one way to spawn application processes.
10
8
  Let's go over the different spawning methods. For simplicity's sake, let's
11
9
  assume that we're only talking about Ruby on Rails applications.
12
10
 
13
- === The most straightforward and traditional way: direct spawning ===
11
+ === The most straightforward and traditional way: direct spawning
14
12
 
15
13
  Phusion Passenger could create a new Ruby process, which will then load the
16
14
  Rails application along with the entire Rails framework. This process will then
17
15
  enter an request handling main loop.
18
16
 
19
- This is the most straightforward way to spawn worker processes. If you're
20
- familiar with the Mongrel application server, then this approach is exactly
21
- what mongrel_cluster performs: it creates N worker processes, each which loads
22
- a full copy of the Rails application and the Rails framework in memory. The Thin
23
- application server employs pretty much the same approach.
17
+ This is the most straightforward way to spawn processes, and each process contains
18
+ a full copy of the Rails application and the Rails framework in memory.
24
19
 
25
- Note that Phusion Passenger's version of direct spawning differs slightly
26
- from mongrel_cluster. Mongrel_cluster creates entirely new Ruby processes. In
27
- programmers jargon, mongrel_cluster creates new Ruby processes by forking the
28
- current process and exec()-ing a new Ruby interpreter. Phusion Passenger on the
29
- other hand creates processes that reuse the already loaded Ruby interpreter. In
30
- programmers jargon, Phusion Passenger calls fork(), but not exec().
20
+ === The smart spawning method
31
21
 
32
- === The smart spawning method ===
33
-
34
- NOTE: Smart spawning is supported for all Ruby applications but not for WSGI applications.
22
+ NOTE: Smart spawning is only supported for Ruby applications. It's not supported for other languages.
35
23
 
36
24
  While direct spawning works well, it's not as efficient as it could be
37
- because each worker process has its own private copy of the Rails application
25
+ because each process has its own private copy of the Rails application
38
26
  as well as the Rails framework. This wastes memory as well as startup time.
39
27
 
40
- image:images/direct_spawning.png[Worker processes and direct spawning] +
41
- 'Figure: Worker processes and direct spawning. Each worker process has its
28
+ image:images/direct_spawning.png[Application processes and direct spawning] +
29
+ 'Figure: Application processes and direct spawning. Each process has its
42
30
  own private copy of the application code and Rails framework code.'
43
31
 
44
- It is possible to make the different worker processes share the memory occupied
32
+ It is possible to make the different processes share the memory occupied
45
33
  by application and Rails framework code, by utilizing so-called
46
34
  copy-on-write semantics of the virtual memory system on modern operating
47
35
  systems. As a side effect, the startup time is also reduced. This is technique
48
36
  is exploited by Phusion Passenger's 'smart' spawn method.
49
37
 
50
- ==== How it works ====
38
+ The 'smart' spawn method is similar to Unicorn's `preload_app true` feature.
39
+
40
+ ==== How it works
51
41
 
52
42
  When the 'smart' spawn method is being used, Phusion Passenger will first
53
- create a so-called 'ApplicationSpawner server' process. This process loads the
54
- entire Rails application along with the Rails framework, by loading
55
- 'environment.rb'. Then, whenever Phusion Passenger needs a new worker process,
56
- it will instruct the ApplicationSpawner server to do so. The ApplicationSpawner
57
- server will create a worker new process
58
- that reuses the already loaded Rails application/framework. Creating a worker
59
- process through an already running ApplicationSpawner server is very fast, about
60
- 10 times faster than loading the Rails application/framework from scratch. If
61
- the Ruby interpreter is copy-on-write friendly (that is, if you're running
62
- <<reducing_memory_usage,Ruby Enterprise Edition>>) then all created worker
63
- processes will share as much common
64
- memory as possible. That is, they will all share the same application and Rails
65
- framework code.
66
-
67
- image:images/smart.png[] +
68
- 'Figure: Worker processes and the smart spawn method. All worker processes,
69
- as well as the ApplicationSpawner, share the same application code and Rails
43
+ create a so-called 'preloader' process. This process loads the
44
+ entire Rails application along with the Rails framework, by evaluating
45
+ `config.ru`. Then, whenever Phusion Passenger needs a new application process,
46
+ it will instruct the preloader to create one. The preloader then then spawns
47
+ a child process, which is an exact virtual copy of itself. This child process
48
+ therefore already has the application code and the Rails framework code in memory.
49
+
50
+ Creating a process like this is very fast, about 10 times faster than loading the
51
+ Rails application/framework from scratch. On top of that, the OS also applies an
52
+ optimization called 'copy-on-write'. This means that all memory that the child
53
+ process hasn't modified, is shared with the parent process.
54
+
55
+ image:images/smart_spawning.png[] +
56
+ 'Figure: Application processes and the smart spawn method. All processes,
57
+ as well as the preloader, share the same application code and Rails
70
58
  framework code.'
71
59
 
72
- The 'smart' method allows different worker processes that belong to the same
73
- application to share memory.
60
+ However, Ruby can only leverage this copy-on-write optimization if its garbage
61
+ collector is friendly. This is only the case starting from Ruby 2.0.0. Earlier
62
+ versions cannot leverage copy-on-write optimizations.
74
63
 
75
- Notes:
64
+ Note that preloader processes have an idle timeout just like application processes.
65
+ If a preloader hasn't been instructed to do anything for a while, it will be shutdown
66
+ in order to conserve memory. This idle timeout is configurable.
76
67
 
77
- - Vendored Rails frameworks cannot be shared by different applications, even if
78
- both vendored Rails frameworks are the same version. So for efficiency reasons
79
- we don't recommend vendoring Rails.
80
- - ApplicationSpawner servers have an idle timeout just like worker processes.
81
- If an ApplicationSpawner/FrameworkSpawner server hasn't been instructed to do
82
- anything for a while, it will be shutdown in order to conserve memory. This
83
- idle timeout is configurable.
68
+ ==== Summary of benefits
84
69
 
85
- ==== Summary of benefits ====
70
+ Suppose that Phusion Passenger needs a process for an application
71
+ that uses Rails 4.1.0.
86
72
 
87
- Suppose that Phusion Passenger needs a new worker process for an application
88
- that uses Rails 2.2.1.
89
-
90
- If the 'smart' spawning method is used, and an ApplicationSpawner server
91
- for this application is already running, then worker process creation time is
92
- about 10 times faster than direct spawning. This worker process will also
93
- share application and Rails framework code memory with the ApplicationSpawner
94
- server and the worker processes that had been spawned by this ApplicationSpawner
95
- server.
73
+ If the 'smart' spawning method is used, and a preloader for this application is
74
+ already running, then process creation time is about 10 times faster than direct
75
+ spawning. This process will also share application and Rails framework code memory
76
+ with the preloader, as well as with other processes that have been spawned by the
77
+ same preloader.
96
78
 
97
79
  In practice, the smart spawning method could mean a memory saving of about 33%,
98
- assuming that your Ruby interpreter is <<reducing_memory_usage,copy-on-write friendly>>.
80
+ assuming that your Ruby interpreter is copy-on-write friendly.
99
81
 
100
- Of course, smart spawning is not without gotchas. But if you understand the
101
- gotchas you can easily reap the benefits of smart spawning.
82
+ Of course, smart spawning is not without caveats. But if you understand the
83
+ caveats you can easily reap the benefits of smart spawning.
102
84
 
103
- === Smart spawning gotcha #1: unintentional file descriptor sharing ===
85
+ === Smart spawning caveat #1: unintentional file descriptor sharing
104
86
 
105
- Because worker processes are created by forking from an ApplicationSpawner
106
- server, it will share all file descriptors that are opened by the
107
- ApplicationSpawner server. (This is part of the semantics of the Unix
87
+ Because application processes are created by forking from a preloader process,
88
+ it will share all file descriptors that are opened by the
89
+ preloader process. (This is part of the semantics of the Unix
108
90
  'fork()' system call. You might want to Google it if you're not familiar with
109
91
  it.) A file descriptor is a handle which can be an opened file, an opened socket
110
- connection, a pipe, etc. If different worker processes write to such a file
92
+ connection, a pipe, etc. If different application processes write to such a file
111
93
  descriptor at the same time, then their write calls will be interleaved, which
112
94
  may potentially cause problems.
113
95
 
114
96
  The problem commonly involves socket connections that are unintentionally being
115
97
  shared. You can fix it by closing and reestablishing the connection when Phusion
116
- Passenger is creating a new worker process. Phusion Passenger provides the API
98
+ Passenger is creating a new application process. Phusion Passenger provides the API
117
99
  call `PhusionPassenger.on_event(:starting_worker_process)` to do so. So you
118
- could insert the following code in your 'environment.rb':
100
+ could insert the following code in your 'config.ru':
119
101
 
120
102
  [source, ruby]
121
103
  -----------------------------------------
@@ -132,61 +114,60 @@ end
132
114
  -----------------------------------------
133
115
 
134
116
  Note that Phusion Passenger automatically reestablishes the connection to the
135
- database upon creating a new worker process, which is why you normally do not
117
+ database upon creating a new application process, which is why you normally do not
136
118
  encounter any database issues when using smart spawning mode.
137
119
 
138
- ==== Example 1: Memcached connection sharing (harmful) ====
120
+ ==== Example 1: Memcached connection sharing (harmful)
139
121
 
140
122
  Suppose we have a Rails application that connects to a Memcached server in
141
- 'environment.rb'. This causes the ApplicationSpawner to have a socket connection
123
+ 'environment.rb'. This causes the preloader to have a socket connection
142
124
  (file descriptor) to the Memcached server, as shown in the following figure:
143
125
 
144
126
  +--------------------+
145
- | ApplicationSpawner |-----------[Memcached server]
127
+ | Preloader |-----------[Memcached server]
146
128
  +--------------------+
147
129
 
148
- Phusion Passenger then proceeds with creating a new Rails worker process, which
130
+ Phusion Passenger then proceeds with creating a new Rails application process, which
149
131
  is to process incoming HTTP requests. The result will look like this:
150
132
 
151
133
  +--------------------+
152
- | ApplicationSpawner |------+----[Memcached server]
134
+ | Preloader |------+----[Memcached server]
153
135
  +--------------------+ |
154
136
  |
155
137
  +--------------------+ |
156
- | Worker process 1 |-----/
138
+ | App process 1 |-----/
157
139
  +--------------------+
158
140
 
159
141
  Since a 'fork()' makes a (virtual) complete copy of a process, all its file
160
- descriptors will be copied as well. What we see here is that ApplicationSpawner
161
- and Worker process 1 both share the same connection to Memcached.
142
+ descriptors will be copied as well. What we see here is that Preloader
143
+ and App process 1 both share the same connection to Memcached.
162
144
 
163
- Now supposed that your site gets Slashdotted and Phusion Passenger needs to
164
- spawn another worker process. It does so by forking ApplicationSpawner. The
165
- result is now as follows:
145
+ Now supposed that your site gets a sudden large surge of traffic, and Phusion Passenger needs to
146
+ spawn another process. It does so by forking Preloader. The result is now as follows:
166
147
 
167
148
  +--------------------+
168
- | ApplicationSpawner |------+----[Memcached server]
149
+ | Preloader |------+----[Memcached server]
169
150
  +--------------------+ |
170
151
  |
171
152
  +--------------------+ |
172
- | Worker process 1 |-----/|
153
+ | App process 1 |-----/|
173
154
  +--------------------+ |
174
155
  |
175
156
  +--------------------+ |
176
- | Worker process 2 |-----/
157
+ | App process 2 |-----/
177
158
  +--------------------+
178
159
 
179
- As you can see, Worker process 1 and Worker process 2 have the same Memcached
160
+ As you can see, App process 1 and App process 2 have the same Memcached
180
161
  connection.
181
162
 
182
163
  Suppose that users Joe and Jane visit your website at the same time. Joe's
183
- request is handled by Worker process 1, and Jane's request is handled by Worker
184
- process 2. Both worker processes want to fetch something from Memcached. Suppose
164
+ request is handled by App process 1, and Jane's request is handled by App
165
+ process 2. Both application processes want to fetch something from Memcached. Suppose
185
166
  that in order to do that, both handlers need to send a "FETCH" command to Memcached.
186
167
 
187
- But suppose that, after worker process 1 having only sent "FE", a context switch
188
- occurs, and worker process 2 starts sending a "FETCH" command to Memcached as
189
- well. If worker process 2 succeeds in sending only one bye, 'F', then Memcached
168
+ But suppose that, after App process 1 having only sent "FE", a context switch
169
+ occurs, and App process 2 starts sending a "FETCH" command to Memcached as
170
+ well. If App process 2 succeeds in sending only one bye, 'F', then Memcached
190
171
  will receive a command which begins with "FEF", a command that it does not
191
172
  recognize. In other words: the data from both handlers get interleaved. And thus
192
173
  Memcached is forced to handle this as an error.
@@ -194,22 +175,22 @@ Memcached is forced to handle this as an error.
194
175
  This problem can be solved by reestablishing the connection to Memcached after forking:
195
176
 
196
177
  +--------------------+
197
- | ApplicationSpawner |------+----[Memcached server]
178
+ | Preloader |------+----[Memcached server]
198
179
  +--------------------+ | |
199
180
  | |
200
181
  +--------------------+ | |
201
- | Worker process 1 |-----/| |
182
+ | App process 1 |-----/| |
202
183
  +--------------------+ | | <--- created this
203
184
  X | new
204
185
  | connection
205
186
  X <-- closed this |
206
187
  +--------------------+ | old |
207
- | Worker process 2 |-----/ connection |
188
+ | App process 2 |-----/ connection |
208
189
  +--------------------+ |
209
190
  | |
210
191
  +-------------------------------------+
211
192
 
212
- Worker process 2 now has its own, separate communication channel with Memcached.
193
+ App process 2 now has its own, separate communication channel with Memcached.
213
194
  The code in 'environment.rb' looks like this:
214
195
 
215
196
  [source, ruby]
@@ -226,7 +207,7 @@ if defined?(PhusionPassenger)
226
207
  end
227
208
  -----------------------------------------
228
209
 
229
- ==== Example 2: Log file sharing (not harmful) ====
210
+ ==== Example 2: Log file sharing (not harmful)
230
211
 
231
212
  There are also cases in which unintentional file descriptor sharing is not harmful.
232
213
  One such case is log file file descriptor sharing. Even if two processes write
@@ -238,12 +219,12 @@ must synchronize write access via an inter-process synchronization mechanism,
238
219
  such as file locks. Reopening the log file, like you would have done in the
239
220
  Memcached example, doesn't help.
240
221
 
241
- === Smart spawning gotcha #2: the need to revive threads ===
222
+ === Smart spawning caveat #2: the need to revive threads
242
223
 
243
224
  Another part of the 'fork()' system call's semantics is the fact that threads
244
225
  disappear after a fork call. So if you've created any threads in environment.rb,
245
- then those threads will no longer be running in newly created worker process.
246
- You need to revive them when a new worker process is created. Use the
226
+ then those threads will no longer be running in newly created application process.
227
+ You need to revive them when a new process is created. Use the
247
228
  `:starting_worker_process` event that Phusion Passenger provides, like this:
248
229
 
249
230
  [source, ruby]
@@ -259,23 +240,3 @@ if defined?(PhusionPassenger)
259
240
  end
260
241
  end
261
242
  -----------------------------------------
262
-
263
- === Smart spawning gotcha #3: code load order ===
264
-
265
- This gotcha is only applicable to the 'smart' spawn method, not the 'smart-lv2'
266
- spawn method.
267
-
268
- If your application expects the Rails framework to be not loaded during the
269
- beginning of 'environment.rb', then it can cause problems when an
270
- ApplicationSpawner is created from a FrameworkSpawner, which already has the
271
- Rails framework loaded. The most common case is when applications try to patch
272
- Rails by dropping a modified file that has the same name as Rails's own file,
273
- in a path that comes earlier in the Ruby search path.
274
-
275
- For example, suppose that we have an application which has a patched version
276
- of 'active_record/base.rb' located in 'RAILS_ROOT/lib/patches', and
277
- 'RAILS_ROOT/lib/patches' comes first in the Ruby load path. When conservative
278
- spawning is used, the patched version of 'base.rb' is properly loaded. When
279
- 'smart' (not 'smart-lv2') spawning is used, the original 'base.rb' is used
280
- because it was already loaded, so a subsequent `require "active_record/base"`
281
- has no effect.