spawnling 2.1

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ data.tar.gz: 22c752d3c7f7e6f38838941833c5b3e05cd376e1
4
+ metadata.gz: 3ed0947f67a6a6937970268bcb2988dc56184dda
5
+ SHA512:
6
+ data.tar.gz: 858a4f1a283da28138a84b801f2943901a86bf49250804bd46477f01b5e667927bab24de0350a5f8887f210af57ae233b3e2cb45647404ff0677bb877eb3ca8c
7
+ metadata.gz: 949044c486e08b519984b6734c009568143d7d64325838f8df47f6cee75d0e8903a831f3552cf4373d25ca8a749381067fae25c327357609c917e11182f7e3a0
data/CHANGELOG ADDED
@@ -0,0 +1,55 @@
1
+ v0.1 - 2007/09/13
2
+
3
+ initial version
4
+
5
+ --------------------------------------------------
6
+ v0.2 - 2007/09/28
7
+
8
+ * return PID of the child process
9
+ * added ":detach => false" option
10
+
11
+ --------------------------------------------------
12
+ v0.3 - 2007/10/15
13
+
14
+ * added ':method => :thread' for threaded spawns
15
+ * removed ':detach => false' option in favor of more generic implementation
16
+ * added ability to set configuration of the form 'Spawn::method :thread'
17
+ * added patch to ActiveRecord::Base to allow for more efficient reconnect in child processes
18
+ * added monkey patch for http://dev.rubyonrails.org/ticket/7579
19
+ * added wait() method to wait for spawned code blocks
20
+ * don't allow threading if allow_concurrency=false
21
+
22
+ --------------------------------------------------
23
+ v0.4 - 2008/1/26
24
+
25
+ * default to :thread on windows, still :fork on all other platforms
26
+ * raise exception when used with :method=>:true and allow_concurrency != true
27
+
28
+ --------------------------------------------------
29
+ v0.5 - 2008/3/1
30
+ * also default to :thread on JRuby (java)
31
+ * added new :method => :yield which doesn't fork or thread, this is useful for testing
32
+ * fixed problem with connections piling up on PostgreSQL
33
+
34
+ --------------------------------------------------
35
+ v0.6 - 2008/04/21
36
+ * only apply clear_reloadable_connections patch on Rails 1.x (7579 fixed in Rails 2.x)
37
+ * made it more responsive in more environments by disconnecting from the listener socket in the forked process
38
+
39
+ --------------------------------------------------
40
+ v0.7 - 2008/04/24
41
+ * more generic mechanism for closing resources after fork
42
+ * check for existence of Mongrel before patching it
43
+
44
+ --------------------------------------------------
45
+ v0.8 - 2008/05/02
46
+ * call exit! within the ensure block so that at_exit handlers aren't called on exceptions
47
+ * set logger from RAILS_DEFAULT_LOGGER if available, else STDERR
48
+
49
+ --------------------------------------------------
50
+ v0.9 - 2008/05/11
51
+ * added ability to set nice level for child process
52
+
53
+ --------------------------------------------------
54
+ v1.0 - 2010/10/09
55
+ * merged edged to master, let's call this version 1.0
data/LICENSE ADDED
@@ -0,0 +1,22 @@
1
+ Copyright (c) 2007 Tom Anderson (tom@squeat.com)
2
+
3
+ Permission is hereby granted, free of charge, to any person
4
+ obtaining a copy of this software and associated documentation
5
+ files (the "Software"), to deal in the Software without
6
+ restriction, including without limitation the rights to use,
7
+ copy, modify, merge, publish, distribute, sublicense, and/or sell
8
+ copies of the Software, and to permit persons to whom the
9
+ Software is furnished to do so, subject to the following
10
+ conditions:
11
+
12
+ The above copyright notice and this permission notice shall be
13
+ included in all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
16
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
17
+ OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
18
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
19
+ HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
20
+ WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
21
+ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
22
+ OTHER DEALINGS IN THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,221 @@
1
+ #News
2
+
3
+ Now runs with ruby 1.9 (and later). Because ruby "stole" the name "spawn", this gem
4
+ now has been redefined to use "Spawn.new(&block)" instead of "spawn(&block)". Other
5
+ than that nothing has changed in the basic usage. Read below for detailed usage.
6
+
7
+ # Spawn
8
+
9
+ This gem provides a 'Spawn' class to easily fork OR thread long-running sections of
10
+ code so that your application can return results to your users more quickly.
11
+ It works by creating new database connections in ActiveRecord::Base for the
12
+ spawned block so that you don't have to worry about database connections working,
13
+ they just do.
14
+
15
+ The gem also patches ActiveRecord::Base to handle some known bugs when using
16
+ threads if you prefer using the threaded model over forking.
17
+
18
+ ## Installation
19
+
20
+ The name of the gem is "spawn-block" (unfortunately somebody took the name "spawn" before
21
+ I was able to convert this to a gem).
22
+
23
+ ### git
24
+
25
+ If you want to live on the latest master branch, add this to your Gemfile,
26
+
27
+ gem 'spawn-block', :git => 'git://github.com/tra/spawn'
28
+
29
+ and use bundler to manage it (bundle install, bundle update).
30
+
31
+ ### rubygem
32
+
33
+ If you'd rather install from the latest gem,
34
+
35
+ gem 'spawn-block', '~>2.0'
36
+
37
+ ### configure
38
+
39
+ Make sure that ActiveRecord reconnects to your database automatically when needed,
40
+ for instance put
41
+
42
+ production/development:
43
+ ...
44
+ reconnect: true
45
+
46
+ into your config/database.yml.
47
+
48
+ ## Usage
49
+
50
+ Here's a simple example of how to demonstrate the spawn plugin.
51
+ In one of your controllers, insert this code (after installing the plugin of course):
52
+ ```ruby
53
+ Spawn.new do
54
+ logger.info("I feel sleepy...")
55
+ sleep 11
56
+ logger.info("Time to wake up!")
57
+ end
58
+ ```
59
+ If everything is working correctly, your controller should finish quickly then you'll see
60
+ the last log message several seconds later.
61
+
62
+ If you need to wait for the spawned processes/threads, then pass the objects returned by
63
+ spawn to Spawn.wait(), like this:
64
+ ```ruby
65
+ spawns = []
66
+ N.times do |i|
67
+ # spawn N blocks of code
68
+ spawns << Spawn.new do
69
+ something(i)
70
+ end
71
+ end
72
+ # wait for all N blocks of code to finish running
73
+ Spawn.wait(spawns)
74
+ ```
75
+ ## Options
76
+
77
+ The options you can pass to spawn are:
78
+
79
+ <table>
80
+ <tr><th>Option</th><th>Values</th></tr>
81
+ <tr><td>:method</td><td>:fork, :thread, :yield</td></tr>
82
+ <tr><td>:nice</td><td>integer value 0-19, 19 = really nice</td></tr>
83
+ <tr><td>:kill</td><td>boolean value indicating whether the parent should kill the spawned process
84
+ when it exits (only valid when :method => :fork)</td></tr>
85
+ <tr><td>:argv</td><td>string to override the process name</td></tr>
86
+ </table>
87
+
88
+ Any option to spawn can be set as a default so that you don't have to pass them in
89
+ to every call of spawn. To configure the spawn default options, add a line to
90
+ your configuration file(s) like this:
91
+ ```ruby
92
+ Spawn::default_options {:method => :thread}
93
+ ```
94
+ If you don't set any default options, the :method will default to :fork. To
95
+ specify different values for different environments, add the default_options call to
96
+ he appropriate environment file (development.rb, test.rb). For testing you can set
97
+ the default :method to :yield so that the code is run inline.
98
+ ```ruby
99
+ # in environment.rb
100
+ Spawn.method :method => :fork, :nice => 7
101
+ # in test.rb, will override the environment.rb setting
102
+ Spawn.method :method => :yield
103
+ ```
104
+ This allows you to set your production and development environments to use different
105
+ methods according to your needs.
106
+
107
+ ### be nice
108
+
109
+ If you want your forked child to run at a lower priority than the parent process, pass in
110
+ the :nice option like this:
111
+ ```ruby
112
+ Spawn.new(:nice => 7) do
113
+ do_something_nicely
114
+ end
115
+ ```
116
+ ### fork me
117
+
118
+ By default, spawn will use the fork to spawn child processes. You can configure it to
119
+ do threading either by telling the spawn method when you call it or by configuring your
120
+ environment.
121
+ For example, this is how you can tell spawn to use threading on the call,
122
+ ```ruby
123
+ Spawn.new(:method => :thread) do
124
+ something
125
+ end
126
+ ```
127
+ When you use threaded spawning, make sure that your application is thread-safe. Rails
128
+ can be switched to thread-safe mode with (not sure if this is needed anymore)
129
+ ```ruby
130
+ # Enable threaded mode
131
+ config.threadsafe!
132
+ ```
133
+ in environments/your_environment.rb
134
+
135
+ ### kill or be killed
136
+
137
+ Depending on your application, you may want the children processes to go away when
138
+ the parent process exits. By default spawn lets the children live after the
139
+ parent dies. But you can tell it to kill the children by setting the :kill option
140
+ to true.
141
+
142
+ ### a process by any other name
143
+
144
+ If you'd like to be able to identify which processes are spawned by looking at the
145
+ output of ps then set the :argv option with a string of your choice.
146
+ You should then be able to see this string as the process name when
147
+ listing the running processes (ps).
148
+
149
+ For example, if you do something like this,
150
+ ```ruby
151
+ 3.times do |i|
152
+ Spawn.new(:argv => "spawn -#{i}-") do
153
+ something(i)
154
+ end
155
+ end
156
+ ```
157
+ then in the shell,
158
+ ```shell
159
+ $ ps -ef | grep spawn
160
+ 502 2645 2642 0 0:00.01 ttys002 0:00.02 spawn -0-
161
+ 502 2646 2642 0 0:00.02 ttys002 0:00.02 spawn -1-
162
+ 502 2647 2642 0 0:00.02 ttys002 0:00.03 spawn -2-
163
+ ```
164
+ The length of the process name may be limited by your OS so you might want to experiment
165
+ to see how long it can be (it may be limited by the length of the original process name).
166
+
167
+ ## Forking vs. Threading
168
+
169
+ There are several tradeoffs for using threading vs. forking. Forking was chosen as the
170
+ default primarily because it requires no configuration to get it working out of the box.
171
+
172
+ Forking advantages:
173
+
174
+ - more reliable? - the ActiveRecord code is generally not deemed to be thread-safe.
175
+ Even though spawn attempts to patch known problems with the threaded implementation,
176
+ there are no guarantees. Forking is heavier but should be fairly reliable.
177
+ - keep running - this could also be a disadvantage, but you may find you want to fork
178
+ off a process that could have a life longer than its parent. For example, maybe you
179
+ want to restart your server without killing the spawned processes.
180
+ We don't necessarily condone this (i.e. haven't tried it) but it's technically possible.
181
+ - easier - forking works out of the box with spawn, threading requires you set
182
+ allow_concurrency=true (for older versions of Rails).
183
+ Also, beware of automatic reloading of classes in development
184
+ mode (config.cache_classes = false).
185
+
186
+ Threading advantages:
187
+ - less filling - threads take less resources... how much less? it depends. Some
188
+ flavors of Unix are pretty efficient at forking so the threading advantage may not
189
+ be as big as you think... but then again, maybe it's more than you think. :wink:
190
+ - debugging - you can set breakpoints in your threads
191
+
192
+ ## Acknowledgements
193
+
194
+ This plugin was initially inspired by Scott Persinger's blog post on how to use fork
195
+ in rails for background processing (link no longer available).
196
+
197
+ Further inspiration for the threading implementation came from [Jonathon Rochkind's
198
+ blog post](http://bibwild.wordpress.com/2007/08/28/threading-in-rails/) on threading in rails.
199
+
200
+ Also thanks to all who have helped debug problems and suggest improvements
201
+ including:
202
+
203
+ - Ahmed Adam, Tristan Schneiter, Scott Haug, Andrew Garfield, Eugene Otto, Dan Sharp,
204
+ Olivier Ruffin, Adrian Duyzer, Cyrille Labesse
205
+
206
+ - Garry Tan, Matt Jankowski (Rails 2.2.x fixes), Mina Naguib (Rails 2.3.6 fix)
207
+
208
+ - Tim Kadom, Mauricio Marcon Zaffari, Danial Pearce, Hongli Lai, Scott Wadden
209
+ (passenger fixes)
210
+
211
+ - Will Bryant, James Sanders (memcache fix)
212
+
213
+ - David Kelso, Richard Hirner, Luke van der Hoeven (gemification and Rails 3 support)
214
+
215
+ - Jay Caines-Gooby, Eric Stewart (Unicorn fix)
216
+
217
+ - Dan Sharp for the changes that allow it to work with Ruby 1.9 and later
218
+
219
+ - &lt;your name here&gt;
220
+
221
+ Copyright (c) 2007-present Tom Anderson (tom@squeat.com), see LICENSE
data/init.rb ADDED
@@ -0,0 +1 @@
1
+ require 'spawn'
data/lib/patches.rb ADDED
@@ -0,0 +1,135 @@
1
+ if defined?(ActiveRecord)
2
+ # see activerecord/lib/active_record/connection_adaptors/abstract/connection_specification.rb
3
+ class ActiveRecord::Base
4
+ # reconnect without disconnecting
5
+ if ::Spawn::RAILS_3_x || ::Spawn::RAILS_2_2
6
+ def self.spawn_reconnect(klass=self)
7
+ # keep ancestors' connection_handlers around to avoid them being garbage collected in the forked child
8
+ @@ancestor_connection_handlers ||= []
9
+ @@ancestor_connection_handlers << self.connection_handler
10
+ self.connection_handler = ActiveRecord::ConnectionAdapters::ConnectionHandler.new
11
+
12
+ establish_connection
13
+ end
14
+ else
15
+ def self.spawn_reconnect(klass=self)
16
+ spec = @@defined_connections[klass.name]
17
+ konn = active_connections[klass.name]
18
+ # remove from internal arrays before calling establish_connection so that
19
+ # the connection isn't disconnected when it calls AR::Base.remove_connection
20
+ @@defined_connections.delete_if { |key, value| value == spec }
21
+ active_connections.delete_if { |key, value| value == konn }
22
+ establish_connection(spec ? spec.config : nil)
23
+ end
24
+ end
25
+
26
+ # this patch not needed on Rails 2.x and later
27
+ if ::Spawn::RAILS_1_x
28
+ # monkey patch to fix threading problems,
29
+ # see: http://dev.rubyonrails.org/ticket/7579
30
+ def self.clear_reloadable_connections!
31
+ if @@allow_concurrency
32
+ # Hash keyed by thread_id in @@active_connections. Hash of hashes.
33
+ @@active_connections.each do |thread_id, conns|
34
+ conns.each do |name, conn|
35
+ if conn.requires_reloading?
36
+ conn.disconnect!
37
+ @@active_connections[thread_id].delete(name)
38
+ end
39
+ end
40
+ end
41
+ else
42
+ # Just one level hash, no concurrency.
43
+ @@active_connections.each do |name, conn|
44
+ if conn.requires_reloading?
45
+ conn.disconnect!
46
+ @@active_connections.delete(name)
47
+ end
48
+ end
49
+ end
50
+ end
51
+ end
52
+ end
53
+ end
54
+
55
+ # just borrowing from the mongrel & passenger patches:
56
+ if defined?(Unicorn::HttpServer) && defined?(Unicorn::HttpRequest::REQ)
57
+ class Unicorn::HttpServer
58
+ REQ = Unicorn::HttpRequest::REQ
59
+ alias_method :orig_process_client, :process_client
60
+ def process_client(client)
61
+ ::Spawn.resources_to_close(client, REQ)
62
+ orig_process_client(client)
63
+ end
64
+ end
65
+ elsif defined? Unicorn::HttpServer
66
+ class Unicorn::HttpServer
67
+ alias_method :orig_process_client, :process_client
68
+ def process_client(client)
69
+ ::Spawn.resources_to_close(client, @request)
70
+ orig_process_client(client)
71
+ end
72
+ end
73
+ end
74
+
75
+ # see mongrel/lib/mongrel.rb
76
+ # it's possible that this is not defined if you're running outside of mongrel
77
+ # examples: ./script/runner or ./script/console
78
+ if defined? Mongrel::HttpServer
79
+ class Mongrel::HttpServer
80
+ # redefine Montrel::HttpServer::process_client so that we can intercept
81
+ # the socket that is being used so Spawn can close it upon forking
82
+ alias_method :orig_process_client, :process_client
83
+ def process_client(client)
84
+ ::Spawn.resources_to_close(client, @socket)
85
+ orig_process_client(client)
86
+ end
87
+ end
88
+ end
89
+
90
+ need_passenger_patch = true
91
+ if defined? PhusionPassenger::VERSION_STRING
92
+ # The VERSION_STRING variable was defined sometime after 2.1.0.
93
+ # We don't need passenger patch for 2.2.2 or later.
94
+ pv = PhusionPassenger::VERSION_STRING.split('.').collect{|s| s.to_i}
95
+ need_passenger_patch = pv[0] < 2 || (pv[0] == 2 && (pv[1] < 2 || (pv[1] == 2 && pv[2] < 2)))
96
+ end
97
+
98
+ if need_passenger_patch
99
+ # Patch for work with passenger < 2.1.0
100
+ if defined? Passenger::Railz::RequestHandler
101
+ class Passenger::Railz::RequestHandler
102
+ alias_method :orig_process_request, :process_request
103
+ def process_request(headers, input, output)
104
+ ::Spawn.resources_to_close(input, output)
105
+ orig_process_request(headers, input, output)
106
+ end
107
+ end
108
+ end
109
+
110
+ # Patch for work with passenger >= 2.1.0
111
+ if defined? PhusionPassenger::Railz::RequestHandler
112
+ class PhusionPassenger::Railz::RequestHandler
113
+ alias_method :orig_process_request, :process_request
114
+ def process_request(headers, input, output)
115
+ ::Spawn.resources_to_close(input, output)
116
+ orig_process_request(headers, input, output)
117
+ end
118
+ end
119
+ end
120
+
121
+ # Patch for passenger with Rails >= 2.3.0 (uses rack)
122
+ if defined? PhusionPassenger::Rack::RequestHandler
123
+ class PhusionPassenger::Rack::RequestHandler
124
+ alias_method :orig_process_request, :process_request
125
+ def process_request(headers, input, output)
126
+ ::Spawn.resources_to_close(input, output)
127
+ orig_process_request(headers, input, output)
128
+ end
129
+ end
130
+ end
131
+ end
132
+
133
+ if defined?(::ActiveSupport::Cache::MemCacheStore) && ::ActiveSupport::Cache.autoload?(:MemCacheStore).nil?
134
+ ::ActiveSupport::Cache::MemCacheStore.delegate :reset, :to => :@data
135
+ end
data/lib/spawn.rb ADDED
@@ -0,0 +1,223 @@
1
+ require 'logger'
2
+
3
+ class Spawn
4
+ if defined? ::Rails
5
+ RAILS_1_x = (::Rails::VERSION::MAJOR == 1) unless defined?(RAILS_1_x)
6
+ RAILS_2_2 = ((::Rails::VERSION::MAJOR == 2 && ::Rails::VERSION::MINOR >= 2)) unless defined?(RAILS_2_2)
7
+ RAILS_3_x = (::Rails::VERSION::MAJOR > 2) unless defined?(RAILS_3_x)
8
+ else
9
+ RAILS_1_x = nil
10
+ RAILS_2_2 = nil
11
+ RAILS_3_x = nil
12
+ end
13
+
14
+ @@default_options = {
15
+ # default to forking (unless windows or jruby)
16
+ :method => ((RUBY_PLATFORM =~ /(win32|mingw32|java)/) ? :thread : :fork),
17
+ :nice => nil,
18
+ :kill => false,
19
+ :argv => nil
20
+ }
21
+
22
+ # things to close in child process
23
+ @@resources = []
24
+
25
+ # forked children to kill on exit
26
+ @@punks = []
27
+
28
+ # in some environments, logger isn't defined
29
+ @@logger = defined?(::Rails) ? ::Rails.logger : ::Logger.new(STDERR)
30
+
31
+ attr_accessor :type
32
+ attr_accessor :handle
33
+
34
+ # Set the options to use every time spawn is called unless specified
35
+ # otherwise. For example, in your environment, do something like
36
+ # this:
37
+ # Spawn::default_options = {:nice => 5}
38
+ # to default to using the :nice option with a value of 5 on every call.
39
+ # Valid options are:
40
+ # :method => (:thread | :fork | :yield)
41
+ # :nice => nice value of the forked process
42
+ # :kill => whether or not the parent process will kill the
43
+ # spawned child process when the parent exits
44
+ # :argv => changes name of the spawned process as seen in ps
45
+ def self.default_options(options = {})
46
+ @@default_options.merge!(options)
47
+ @@logger.info "spawn> default options = #{options.inspect}" if @@logger
48
+ end
49
+
50
+ # set the resources to disconnect from in the child process (when forking)
51
+ def self.resources_to_close(*resources)
52
+ @@resources = resources
53
+ end
54
+
55
+ # close all the resources added by calls to resource_to_close
56
+ def self.close_resources
57
+ @@resources.each do |resource|
58
+ resource.close if resource && resource.respond_to?(:close) && !resource.closed?
59
+ end
60
+ # in case somebody spawns recursively
61
+ @@resources.clear
62
+ end
63
+
64
+ def self.alive?(pid)
65
+ begin
66
+ Process::kill 0, pid
67
+ # if the process is alive then kill won't throw an exception
68
+ true
69
+ rescue Errno::ESRCH
70
+ false
71
+ end
72
+ end
73
+
74
+ def self.kill_punks
75
+ @@punks.each do |punk|
76
+ if alive?(punk)
77
+ @@logger.info "spawn> parent(#{Process.pid}) killing child(#{punk})" if @@logger
78
+ begin
79
+ Process.kill("TERM", punk)
80
+ rescue
81
+ end
82
+ end
83
+ end
84
+ @@punks = []
85
+ end
86
+ # register to kill marked children when parent exits
87
+ at_exit { Spawn.kill_punks }
88
+
89
+ # Spawns a long-running section of code and returns the ID of the spawned process.
90
+ # By default the process will be a forked process. To use threading, pass
91
+ # :method => :thread or override the default behavior in the environment by setting
92
+ # 'Spawn::method :thread'.
93
+ def initialize(opts = {})
94
+ raise "Must give block of code to be spawned" unless block_given?
95
+ options = @@default_options.merge(symbolize_options(opts))
96
+ # setting options[:method] will override configured value in default_options[:method]
97
+ if options[:method] == :yield
98
+ yield
99
+ elsif options[:method].respond_to?(:call)
100
+ options[:method].call(proc { yield })
101
+ elsif options[:method] == :thread
102
+ # for versions before 2.2, check for allow_concurrency
103
+ if RAILS_2_2 || ActiveRecord::Base.respond_to?(:allow_concurrency) ?
104
+ ActiveRecord::Base.allow_concurrency : Rails.application.config.allow_concurrency
105
+ @type = :thread
106
+ @handle = thread_it(options) { yield }
107
+ else
108
+ @@logger.error("spawn(:method=>:thread) only allowed when allow_concurrency=true")
109
+ raise "spawn requires config.active_record.allow_concurrency=true when used with :method=>:thread"
110
+ end
111
+ else
112
+ @type = :fork
113
+ @handle = fork_it(options) { yield }
114
+ end
115
+ end
116
+
117
+ def self.wait(sids = [])
118
+ # wait for all threads and/or forks (if a single sid passed in, convert to array first)
119
+ Array(sids).each do |sid|
120
+ if sid.type == :thread
121
+ sid.handle.join()
122
+ else
123
+ begin
124
+ Process.wait(sid.handle)
125
+ rescue
126
+ # if the process is already done, ignore the error
127
+ end
128
+ end
129
+ end
130
+ # clean up connections from expired threads
131
+ ActiveRecord::Base.verify_active_connections!() if defined?(ActiveRecord)
132
+ end
133
+
134
+ protected
135
+
136
+ def fork_it(options)
137
+ # The problem with rails is that it only has one connection (per class),
138
+ # so when we fork a new process, we need to reconnect.
139
+ @@logger.debug "spawn> parent PID = #{Process.pid}" if @@logger
140
+ child = fork do
141
+ begin
142
+ start = Time.now
143
+ @@logger.debug "spawn> child PID = #{Process.pid}" if @@logger
144
+
145
+ # this child has no children of it's own to kill (yet)
146
+ @@punks = []
147
+
148
+ # set the nice priority if needed
149
+ Process.setpriority(Process::PRIO_PROCESS, 0, options[:nice]) if options[:nice]
150
+
151
+ # disconnect from the listening socket, et al
152
+ Spawn.close_resources
153
+ if defined?(Rails)
154
+ # get a new database connection so the parent can keep the original one
155
+ ActiveRecord::Base.spawn_reconnect
156
+ # close the memcache connection so the parent can keep the original one
157
+ Rails.cache.reset if Rails.cache.respond_to?(:reset)
158
+ end
159
+
160
+ # set the process name
161
+ $0 = options[:argv] if options[:argv]
162
+
163
+ # run the block of code that takes so long
164
+ yield
165
+
166
+ rescue => ex
167
+ @@logger.error "spawn> Exception in child[#{Process.pid}] - #{ex.class}: #{ex.message}" if @@logger
168
+ @@logger.error "spawn> " + ex.backtrace.join("\n") if @@logger
169
+ ensure
170
+ begin
171
+ # to be safe, catch errors on closing the connnections too
172
+ ActiveRecord::Base.connection_handler.clear_all_connections! if defined?(ActiveRecord)
173
+ ensure
174
+ @@logger.info "spawn> child[#{Process.pid}] took #{Time.now - start} sec" if @@logger
175
+ # ensure log is flushed since we are using exit!
176
+ @@logger.flush if @@logger && @@logger.respond_to?(:flush)
177
+ # this child might also have children to kill if it called spawn
178
+ Spawn.kill_punks
179
+ # this form of exit doesn't call at_exit handlers
180
+ exit!(0)
181
+ end
182
+ end
183
+ end
184
+
185
+ # detach from child process (parent may still wait for detached process if they wish)
186
+ Process.detach(child)
187
+
188
+ # remove dead children from the target list to avoid memory leaks
189
+ @@punks.delete_if {|punk| !alive?(punk)}
190
+
191
+ # mark this child for death when this process dies
192
+ if options[:kill]
193
+ @@punks << child
194
+ @@logger.debug "spawn> death row = #{@@punks.inspect}" if @@logger
195
+ end
196
+
197
+ # return Spawn::Id.new(:fork, child)
198
+ return child
199
+ end
200
+
201
+ def thread_it(options)
202
+ # clean up stale connections from previous threads
203
+ ActiveRecord::Base.verify_active_connections!() if defined?(ActiveRecord)
204
+ thr = Thread.new do
205
+ # run the long-running code block
206
+ ActiveRecord::Base.connection_pool.with_connection { yield }
207
+ end
208
+ thr.priority = -options[:nice] if options[:nice]
209
+ return thr
210
+ end
211
+
212
+ # In case we don't have rails, can't call opts.symbolize_keys
213
+ def symbolize_options(hash)
214
+ hash.inject({}) do |new_hash, (key, value)|
215
+ new_hash[key.to_sym] = value
216
+ new_hash
217
+ end
218
+ end
219
+ end
220
+ Spawnling = Spawn
221
+
222
+ # patches depends on Spawn so require it after the class
223
+ require 'patches'
@@ -0,0 +1,32 @@
1
+ module SpawnExtensions
2
+ # FIXME don't know how to tell Spawn to use #add_spawn_proc without extended
3
+ # using extended forces to make methods class methods while this is not very clean
4
+ def self.extended(base)
5
+ Spawn::method proc{ |block| add_spawn_proc(block) }
6
+ end
7
+
8
+ # Calls the spawn that was created
9
+ #
10
+ # Can be used to keep control over forked processes in your tests
11
+ def call_last_spawn_proc
12
+ spawns = SpawnExtensions.spawn_procs
13
+
14
+ raise "No spawn procs left" if spawns.empty?
15
+
16
+ spawns.pop.call
17
+ end
18
+
19
+ private
20
+
21
+ def self.spawn_procs
22
+ @@spawn_procs ||= []
23
+ end
24
+
25
+ def self.add_spawn_proc(block)
26
+ spawn_procs << block
27
+ end
28
+
29
+ end
30
+
31
+ # Extend cucumber to take control over spawns
32
+ World(SpawnExtensions)
@@ -0,0 +1,63 @@
1
+ require File.dirname(__FILE__) + '/../spec_helper'
2
+
3
+ describe Spawn do
4
+
5
+ describe "yields" do
6
+ before(:each) do
7
+ Spawn::method :yield
8
+ define_spawned
9
+ end
10
+
11
+ it "should be able to yield directly" do
12
+ Spawned.hello.should == "hello"
13
+ end
14
+ end
15
+
16
+ describe "override" do
17
+ before(:each) do
18
+ Spawn::method(proc{ "foo" })
19
+ define_spawned
20
+ end
21
+
22
+ it "should be able to return a proc" do
23
+ Spawned.hello.should == "foo"
24
+ end
25
+
26
+ end
27
+
28
+ describe "delegate to a proc" do
29
+ before(:each) do
30
+ Spawn::method( proc{ |block| block })
31
+ define_spawned
32
+ end
33
+
34
+ it "should be able to return a proc" do
35
+ Spawned.hello.should be_kind_of(Proc)
36
+ end
37
+
38
+ it "should be able to return a proc" do
39
+ Spawned.hello.call.should == "hello"
40
+ end
41
+
42
+ end
43
+
44
+ after(:each) do
45
+ Object.send(:remove_const, :Spawned)
46
+ end
47
+
48
+ def define_spawned
49
+ cls = Class.new do
50
+ extend Spawn
51
+
52
+ def self.hello
53
+ spawn do
54
+ "hello"
55
+ end
56
+ end
57
+
58
+ end
59
+
60
+ Object.const_set :Spawned, cls
61
+ end
62
+
63
+ end
@@ -0,0 +1,13 @@
1
+ require 'rubygems'
2
+ gem 'rspec'
3
+ require 'spec'
4
+ require 'active_record'
5
+ require 'action_controller'
6
+ require 'rails/version'
7
+
8
+ $:.unshift(File.join(File.dirname(__FILE__), %w[.. lib]))
9
+
10
+ require 'spawn'
11
+ Spec::Runner.configure do |config|
12
+
13
+ end
metadata ADDED
@@ -0,0 +1,69 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: spawnling
3
+ version: !ruby/object:Gem::Version
4
+ version: "2.1"
5
+ platform: ruby
6
+ authors:
7
+ - Tom Anderson
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+
12
+ date: 2010-08-08 00:00:00 Z
13
+ dependencies: []
14
+
15
+ description: |-
16
+ This plugin provides a 'Spawn' class to easily fork OR
17
+ thread long-running sections of code so that your application can return
18
+ results to your users more quickly. This plugin works by creating new database
19
+ connections in ActiveRecord::Base for the spawned block.
20
+
21
+ The plugin also patches ActiveRecord::Base to handle some known bugs when using
22
+ threads (see lib/patches.rb).
23
+ email:
24
+ - tom@squeat.com
25
+ executables: []
26
+
27
+ extensions: []
28
+
29
+ extra_rdoc_files: []
30
+
31
+ files:
32
+ - lib/patches.rb
33
+ - lib/spawn/cucumber.rb
34
+ - lib/spawn.rb
35
+ - spec/spawn/spawn_spec.rb
36
+ - spec/spec_helper.rb
37
+ - CHANGELOG
38
+ - LICENSE
39
+ - README.md
40
+ - init.rb
41
+ homepage: http://github.com/tra/spawn
42
+ licenses: []
43
+
44
+ metadata: {}
45
+
46
+ post_install_message:
47
+ rdoc_options: []
48
+
49
+ require_paths:
50
+ - lib
51
+ required_ruby_version: !ruby/object:Gem::Requirement
52
+ requirements:
53
+ - - ">="
54
+ - !ruby/object:Gem::Version
55
+ version: "0"
56
+ required_rubygems_version: !ruby/object:Gem::Requirement
57
+ requirements:
58
+ - - ">="
59
+ - !ruby/object:Gem::Version
60
+ version: 1.3.6
61
+ requirements: []
62
+
63
+ rubyforge_project:
64
+ rubygems_version: 2.0.3
65
+ signing_key:
66
+ specification_version: 4
67
+ summary: Easily fork OR thread long-running sections of code in Ruby
68
+ test_files: []
69
+