preforker 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/.document ADDED
@@ -0,0 +1,5 @@
1
+ README.rdoc
2
+ lib/**/*.rb
3
+ bin/*
4
+ features/**/*.feature
5
+ LICENSE
data/.gitignore ADDED
@@ -0,0 +1,24 @@
1
+ ## MAC OS
2
+ .DS_Store
3
+
4
+ ## TEXTMATE
5
+ *.tmproj
6
+ tmtags
7
+
8
+ ## EMACS
9
+ *~
10
+ \#*
11
+ .\#*
12
+
13
+ ## VIM
14
+ *.swp
15
+
16
+ ## PROJECT::GENERAL
17
+ coverage
18
+ rdoc
19
+ pkg
20
+
21
+ ## PROJECT::SPECIFIC
22
+ *.log
23
+ *.pid
24
+
data/LICENSE ADDED
@@ -0,0 +1,20 @@
1
+ Copyright (c) 2009 Daniel Cadenas
2
+
3
+ Permission is hereby granted, free of charge, to any person obtaining
4
+ a copy of this software and associated documentation files (the
5
+ "Software"), to deal in the Software without restriction, including
6
+ without limitation the rights to use, copy, modify, merge, publish,
7
+ distribute, sublicense, and/or sell copies of the Software, and to
8
+ permit persons to whom the Software is furnished to do so, subject to
9
+ the following conditions:
10
+
11
+ The above copyright notice and this permission notice shall be
12
+ included in all copies or substantial portions of the Software.
13
+
14
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
15
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
16
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
17
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
18
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
19
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
20
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.rdoc ADDED
@@ -0,0 +1,77 @@
1
+ = preforker
2
+
3
+ A gem to easily create prefork servers.
4
+
5
+ == Example
6
+
7
+ Let's see an example using the Mac 'say' command.
8
+
9
+ require 'preforker'
10
+
11
+ #you can open some socket here or reserve any other resource you want to share with your workers, this is master space
12
+ Preforker.new(:timeout => 10) do |master|
13
+ #this block is only run from each worker (10 by default)
14
+
15
+ #here you should write the code that is needed to be ran each time a fork is created, initializations, etc.
16
+ `say hi`
17
+
18
+ #here you could IO.select a socket, run an EventMachine service (see example), or just run worker loop
19
+ #you need to ask master if it wants you alive periodically or else it will kill you after the timeout elapses. Respect your master!
20
+ while master.wants_me_alive? do
21
+ sleep 1
22
+ `say ping pong`
23
+ end
24
+
25
+ #here we can do whatever we want when exiting gracefully
26
+ `say bye`
27
+ end.start
28
+
29
+ To kill the server just run:
30
+
31
+ kill -s QUIT `cat preforker.pid`
32
+
33
+ See the examples directory and the specs for more examples.
34
+
35
+ == Configuration options
36
+
37
+ [:timeout] The timeout in seconds, 5 by default. If a worker takes more than this it will be killed and respawned.
38
+ [:workers] Number of workers, 10 by default.
39
+ [:stdout_path] Path to redirect stdout to. By default it's /dev/null
40
+ [:stderr_path] Path to redirect stderr to. By default it's /dev/null
41
+ [:app_name] The app name, 'preforker' by default. Used for some ps message, log messages messages and pid file name.
42
+ [:pid_path] The path to the pid file for this server. By default it's './preforker.pid'.
43
+ [:logger] This is Logger.new('./preforker.log') by default
44
+
45
+ == Signals
46
+
47
+ You can send some signals to master to control the way it handles the workers lifetime.
48
+
49
+ [WINCH] Gracefully kill all workers but keep master alive
50
+ [TTIN] Increase number of workers
51
+ [TTOU] Decrease number of workers
52
+ [QUIT] Kill workers and master in a graceful way
53
+ [TERM, INT] Kill workers and master immediately
54
+
55
+ == Acknowledgments
56
+
57
+ Most of the preforking operating system tricks come from Unicorn. Check out its source code for a hands on tutorial on Unix internals!
58
+
59
+ == To do list
60
+
61
+ * More tests
62
+ * Log rotation through the USR1 signal
63
+ * Have something like min_spare_workers, max_workers
64
+
65
+ == Note on Patches/Pull Requests
66
+
67
+ * Fork the project.
68
+ * Make your feature addition or bug fix.
69
+ * Add tests for it. This is important so I don't break it in a
70
+ future version unintentionally.
71
+ * Commit, do not mess with rakefile, version, or history.
72
+ (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
73
+ * Send me a pull request. Bonus points for topic branches.
74
+
75
+ == Copyright
76
+
77
+ Copyright (c) 2010 Daniel Cadenas. See LICENSE for details.
data/Rakefile ADDED
@@ -0,0 +1,46 @@
1
+ require 'rubygems'
2
+ require 'rake'
3
+
4
+ begin
5
+ require 'jeweler'
6
+ Jeweler::Tasks.new do |gem|
7
+ gem.name = "preforker"
8
+ gem.summary = %Q{A gem to easily create prefork servers.}
9
+ gem.description = %Q{A gem to easily create prefork servers.}
10
+ gem.email = "dcadenas@gmail.com"
11
+ gem.homepage = "http://github.com/dcadenas/preforker"
12
+ gem.authors = ["Daniel Cadenas"]
13
+ gem.add_development_dependency "rspec", ">= 1.3.0"
14
+ gem.add_development_dependency "filetesthelper", ">= 0.10.1"
15
+ # gem is a Gem::Specification... see http://www.rubygems.org/read/chapter/20 for additional settings
16
+ end
17
+ Jeweler::GemcutterTasks.new
18
+ rescue LoadError
19
+ puts "Jeweler (or a dependency) not available. Install it with: gem install jeweler"
20
+ end
21
+
22
+ require 'spec/rake/spectask'
23
+ Spec::Rake::SpecTask.new(:spec) do |spec|
24
+ spec.libs << 'lib' << 'spec'
25
+ spec.spec_files = FileList['spec/**/*_spec.rb']
26
+ end
27
+
28
+ Spec::Rake::SpecTask.new(:rcov) do |spec|
29
+ spec.libs << 'lib' << 'spec'
30
+ spec.pattern = 'spec/**/*_spec.rb'
31
+ spec.rcov = true
32
+ end
33
+
34
+ task :spec => :check_dependencies
35
+
36
+ task :default => :spec
37
+
38
+ require 'rake/rdoctask'
39
+ Rake::RDocTask.new do |rdoc|
40
+ version = File.exist?('VERSION') ? File.read('VERSION') : ""
41
+
42
+ rdoc.rdoc_dir = 'rdoc'
43
+ rdoc.title = "preforker #{version}"
44
+ rdoc.rdoc_files.include('README*')
45
+ rdoc.rdoc_files.include('lib/**/*.rb')
46
+ end
data/VERSION ADDED
@@ -0,0 +1 @@
1
+ 0.1.0
data/examples/amqp.rb ADDED
@@ -0,0 +1,24 @@
1
+ require 'rubygems'
2
+ require 'preforker'
3
+ require 'mq'
4
+
5
+ #be sure to run your AMQP server first
6
+
7
+ Preforker.new(:timeout => 5, :workers => 4, :app_name => "Amqp example") do |master|
8
+ AMQP.start(:host => 'dcadenas-laptop.local') do
9
+ EM.add_periodic_timer(1) do
10
+ AMQP.stop{ EM.stop } unless master.wants_me_alive?
11
+ end
12
+
13
+ MQ.prefetch(1)
14
+ channel = MQ.new
15
+ test_exchange = channel.fanout('test_exchange')
16
+ channel.queue('test_queue').bind(test_exchange).subscribe(:ack => true) do |h, msg|
17
+ $stdout.puts "#{$$} received #{msg.inspect}"
18
+ h.ack
19
+ end
20
+ end
21
+ end.start
22
+
23
+ #run examples/amqp_client.rb to see the output
24
+ #kill server with: kill -s TERM `cat 'amqp example.pid'`
@@ -0,0 +1,11 @@
1
+ require 'rubygems'
2
+ require 'bunny'
3
+
4
+ Bunny.run(:host => 'dcadenas-laptop.local', :port => 5672, :user => 'guest', :password => 'guest') do |b|
5
+ e = b.exchange('test_exchange', :type => :fanout)
6
+
7
+ 25.times do |i|
8
+ e.publish("number #{i}")
9
+ end
10
+ end
11
+
@@ -0,0 +1,21 @@
1
+ require 'preforker'
2
+
3
+ #you can open some socket here or reserve any other resource you want to share with your workers, this is master space
4
+ Preforker.new(:timeout => 10) do |master|
5
+ #this block is only run from each worker (10 by default)
6
+
7
+ #here you should write the code that is needed to be ran each time a fork is created, initializations, etc.
8
+ `say hi`
9
+
10
+ #here you could IO.select a socket, run an EventMachine service (see example), or just run worker loop
11
+ #you need to ask master if it wants you alive periodically or else it will kill you after the timeout elapses. Respect your master!
12
+ while master.wants_me_alive? do
13
+ sleep 1
14
+ `say ping pong`
15
+ end
16
+
17
+ #here we can do whatever we want when exiting gracefully
18
+ `say bye`
19
+ end.start
20
+
21
+ #to kill the server just run: kill -s QUIT `cat preforker.pid`
@@ -0,0 +1,59 @@
1
+ class Preforker
2
+ class PidManager
3
+ attr_reader :pid_path, :pid
4
+
5
+ def initialize(pid_path)
6
+ set_pid_path(pid_path, $$)
7
+ end
8
+
9
+ def set_pid_path(new_pid_path, new_pid)
10
+ if new_pid_path
11
+ if read_pid = read_path_pid(new_pid_path)
12
+ return new_pid_path if @pid_path && new_pid_path == @pid_path && read_pid == @pid
13
+ raise ArgumentError, "#{$$} Already running on PID:#{read_pid} (or #{new_pid_path} is stale)"
14
+ end
15
+ end
16
+ unlink_pid_safe(@pid_path) if @pid_path
17
+
18
+ if new_pid_path
19
+ fp = begin
20
+ tmp = "#{File.dirname(new_pid_path)}/#{rand}.#{pid}"
21
+ File.open(tmp, File::RDWR|File::CREAT|File::EXCL, 0644)
22
+ rescue Errno::EEXIST
23
+ retry
24
+ end
25
+ fp.syswrite("#{new_pid}\n")
26
+ File.rename(fp.path, new_pid_path)
27
+ fp.close
28
+ end
29
+
30
+ @pid = new_pid
31
+ @pid_path = new_pid_path
32
+ end
33
+
34
+ # unlinks a PID file at given +path+ if it contains the current PID
35
+ # still potentially racy without locking the directory (which is
36
+ # non-portable and may interact badly with other programs), but the
37
+ # window for hitting the race condition is small
38
+ def unlink
39
+ File.unlink(@pid_path) if @pid_path && File.read(@pid_path).to_i == @pid
40
+ rescue
41
+ end
42
+
43
+ private
44
+
45
+ # returns a PID if a given path contains a non-stale PID file,
46
+ # nil otherwise.
47
+ def read_path_pid(path)
48
+ pid = File.read(path).to_i
49
+ return nil if pid <= 0
50
+ begin
51
+ Process.kill(0, pid)
52
+ pid
53
+ rescue Errno::ESRCH
54
+ # don't unlink stale pid files, racy without non-portable locking...
55
+ end
56
+ rescue Errno::ENOENT
57
+ end
58
+ end
59
+ end
@@ -0,0 +1,87 @@
1
+ class Preforker
2
+ class SignalProcessor
3
+ attr_reader :interesting_signals, :signal_queue
4
+ def initialize(master)
5
+ @read_pipe, @write_pipe = IO.pipe
6
+ @master = master
7
+ @signal_queue = []
8
+ @interesting_signals = [:WINCH, :QUIT, :INT, :TERM, :USR1, :USR2, :HUP, :TTIN, :TTOU]
9
+ @interesting_signals.each { |sig| trap_deferred(sig) }
10
+ end
11
+
12
+ def trap_deferred(signal)
13
+ trap(signal) do
14
+ if @signal_queue.size < 5
15
+ @signal_queue << signal
16
+ wake_up_master
17
+ else
18
+ @master.logger.error "ignoring SIG#{signal}, queue=#{@signal_queue.inspect}"
19
+ end
20
+ end
21
+ end
22
+
23
+ def start_signal_loop
24
+ last_check = Time.now
25
+ begin
26
+ loop do
27
+ @master.reap_all_workers
28
+ case @signal_queue.shift
29
+ when nil
30
+ # avoid murdering workers after our master process (or the
31
+ # machine) comes out of suspend/hibernation
32
+ @master.murder_lazy_workers if (last_check + @master.timeout) >= (last_check = Time.now)
33
+ @master.maintain_worker_count
34
+ sleep_master
35
+ when :WINCH
36
+ @master.logger.info "Gracefully stopping all workers"
37
+ @master.number_of_workers = 0
38
+ when :TTIN
39
+ @master.number_of_workers += 1
40
+ when :TTOU
41
+ @master.number_of_workers -= 1 if @master.number_of_workers > 0
42
+ when :QUIT # graceful shutdown
43
+ break
44
+ when :TERM, :INT # immediate shutdown
45
+ @master.stop(false)
46
+ break
47
+ end
48
+ end
49
+ rescue Errno::EINTR
50
+ retry
51
+ rescue => e
52
+ @master.logger.error "Unhandled master loop exception #{e.inspect}.\n#{e.backtrace.join("\n")}"
53
+ retry
54
+ ensure
55
+ @master.logger.info "Master quitting"
56
+ @master.quit
57
+ end
58
+ end
59
+
60
+ # wait for a signal hander to wake us up and then consume the pipe
61
+ # Wake up every second anyways to run murder_lazy_workers
62
+ def sleep_master
63
+ begin
64
+ maximum_sleep = @master.timeout > 1 ? 1 : @master.timeout / 2
65
+ ready = IO.select([@read_pipe], nil, nil, maximum_sleep) or return
66
+ ready.first && ready.first.first or return
67
+ chunk_size = 16 * 1024
68
+ loop { @read_pipe.read_nonblock(chunk_size) }
69
+ rescue Errno::EAGAIN, Errno::EINTR
70
+ end
71
+ end
72
+
73
+ def wake_up_master
74
+ begin
75
+ @write_pipe.write_nonblock('.') # wakeup master process from select
76
+ rescue Errno::EAGAIN, Errno::EINTR
77
+ # pipe is full, master should wake up anyways
78
+ retry
79
+ end
80
+ end
81
+
82
+ def reset
83
+ @interesting_signals.each { |sig| trap(sig, nil) }
84
+ @signal_queue.clear
85
+ end
86
+ end
87
+ end
@@ -0,0 +1,42 @@
1
+ # -*- encoding: binary -*-
2
+
3
+ require 'fcntl'
4
+ require 'tmpdir'
5
+
6
+ class Preforker
7
+ class TmpIO < ::File
8
+
9
+ # for easier env["rack.input"] compatibility
10
+ def size
11
+ # flush if sync
12
+ stat.size
13
+ end
14
+ end
15
+
16
+ class Util
17
+ def self.is_log?(fp)
18
+ append_flags = File::WRONLY | File::APPEND
19
+
20
+ ! fp.closed? &&
21
+ fp.sync &&
22
+ fp.path[0] == ?/ &&
23
+ (fp.fcntl(Fcntl::F_GETFL) & append_flags) == append_flags
24
+ end
25
+
26
+ # creates and returns a new File object. The File is unlinked
27
+ # immediately, switched to binary mode, and userspace output
28
+ # buffering is disabled
29
+ def self.tmpio
30
+ fp = begin
31
+ TmpIO.open("#{Dir::tmpdir}/#{rand}",
32
+ File::RDWR|File::CREAT|File::EXCL, 0600)
33
+ rescue Errno::EEXIST
34
+ retry
35
+ end
36
+ File.unlink(fp.path)
37
+ fp.binmode
38
+ fp.sync = true
39
+ fp
40
+ end
41
+ end
42
+ end
@@ -0,0 +1,86 @@
1
+ #all this code is called inside a forked block
2
+ require 'preforker/util'
3
+
4
+ class Preforker
5
+ class Worker
6
+ class WorkerAPI
7
+ attr_reader :logger
8
+
9
+ def initialize(worker, master)
10
+ @master = master
11
+ @logger = master.logger
12
+ @worker = worker
13
+ @alive = @worker.tmp
14
+ @last_file_bit = 0
15
+ end
16
+
17
+ def wants_me_alive?
18
+ if @alive
19
+ #we are alive, let's be thankful and tell master we are alive and happy
20
+ @last_file_bit = 0 == @last_file_bit ? 1 : 0
21
+ @alive.chmod(@last_file_bit)
22
+ end
23
+ @alive
24
+ end
25
+
26
+ def kill
27
+ if @alive
28
+ @worker.log "Exiting gracefully"
29
+ @alive = false
30
+ end
31
+ end
32
+ end
33
+
34
+ attr_reader :tmp
35
+ attr_accessor :pid
36
+
37
+ def initialize(worker_block, master)
38
+ @worker_block = worker_block
39
+ @master = master
40
+ @tmp = Util.tmpio
41
+ end
42
+
43
+ def init_self_pipe!
44
+ @read_pipe, @write_pipe = IO.pipe
45
+ [@read_pipe, @write_pipe].each { |io| io.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC) }
46
+ end
47
+
48
+ def work
49
+ log "Created"
50
+ init
51
+
52
+ @worker_block.call(@master_api) if @master_api.wants_me_alive?
53
+ end
54
+
55
+ def init
56
+ init_self_pipe!
57
+ handle_signals
58
+ tmp.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC)
59
+ @master_api = WorkerAPI.new(self, @master)
60
+ end
61
+
62
+ def handle_signals
63
+ [:TERM, :INT].each do |sig|
64
+ trap(sig) { exit!(0) }
65
+ end
66
+
67
+ [:EXIT, :QUIT].each do |sig|
68
+ trap(sig){ @master_api.kill }
69
+ end
70
+
71
+ #what usr1 does
72
+ trap(:USR1) { @read_pipe.close rescue nil }
73
+ trap(:CHLD, 'DEFAULT')
74
+ end
75
+
76
+ def proc_message(message)
77
+ full_message = ["#{@master.app_name} Child #{$$}", message].join(": ")
78
+ $0 = full_message
79
+ end
80
+
81
+ def log(message)
82
+ full_message = proc_message(message)
83
+ @master.logger.info full_message
84
+ end
85
+ end
86
+ end
data/lib/preforker.rb ADDED
@@ -0,0 +1,189 @@
1
+ require 'socket'
2
+ require 'fcntl'
3
+ require 'preforker/pid_manager'
4
+ require 'preforker/worker'
5
+ require 'preforker/signal_processor'
6
+ require 'logger'
7
+
8
+ class Preforker
9
+ attr_reader :timeout, :app_name, :logger
10
+ attr_accessor :number_of_workers
11
+
12
+ def initialize(options = {}, &worker_block)
13
+ @app_name = options[:app_name] || "Preforker"
14
+ default_log_file = "#{@app_name.downcase}.log"
15
+ @options = {
16
+ :timeout => 5,
17
+ :workers => 10,
18
+ :app_name => "Preforker",
19
+ :stderr_path => default_log_file,
20
+ :stderr_path => default_log_file
21
+ }.merge(options)
22
+
23
+ @logger = @options[:logger] || Logger.new(default_log_file)
24
+
25
+ @timeout = @options[:timeout]
26
+ @number_of_workers = @options[:workers]
27
+ @worker_block = worker_block || lambda {}
28
+
29
+ @workers = {}
30
+ $0 = "#@app_name Master"
31
+ end
32
+
33
+ def start
34
+ launch do |ready_write|
35
+ $stdin.reopen("/dev/null")
36
+ set_stdout_path(@options[:stdout_path])
37
+ set_stderr_path(@options[:stderr_path])
38
+
39
+ logger.info "Master started"
40
+
41
+ pid_path = @options[:pid_path] || "./#{@app_name.downcase}.pid"
42
+ @pid_manager = PidManager.new(pid_path)
43
+ @signal_processor = SignalProcessor.new(self)
44
+
45
+ spawn_missing_workers do
46
+ ready_write.close
47
+ end
48
+
49
+ #tell parent we are ready
50
+ ready_write.syswrite($$.to_s)
51
+ ready_write.close rescue nil
52
+
53
+ @signal_processor.start_signal_loop
54
+ end
55
+ end
56
+
57
+ def launch(&block)
58
+ puts "Starting server"
59
+
60
+ ready_read, ready_write = IO.pipe
61
+ [ready_read, ready_write].each { |io| io.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC) }
62
+
63
+ fork do
64
+ ready_read.close
65
+
66
+ Process.setsid
67
+ fork do
68
+ block.call(ready_write)
69
+ end
70
+ end
71
+
72
+ ready_write.close
73
+ master_pid = (ready_read.readpartial(16) rescue nil).to_i
74
+ ready_read.close
75
+ if master_pid <= 1
76
+ warn "Master failed to start, check stderr log for details"
77
+ exit!(1)
78
+ else
79
+ puts "Server started successfuly"
80
+ exit(0)
81
+ end
82
+ end
83
+
84
+ def close_resources_worker_wont_use
85
+ @signal_processor.reset
86
+ @workers.values.each { |other| other.tmp.close rescue nil }
87
+ @workers.clear
88
+ end
89
+
90
+ # Terminates all workers, but does not exit master process
91
+ def stop(graceful = true)
92
+ limit = Time.now + @timeout
93
+ until @workers.empty? || Time.now > limit
94
+ signal_each_worker(graceful ? :QUIT : :TERM)
95
+ sleep(0.1)
96
+ reap_all_workers
97
+ end
98
+ signal_each_worker(:KILL)
99
+ end
100
+
101
+ def quit(graceful = true)
102
+ stop(graceful)
103
+ @pid_manager.unlink
104
+ end
105
+
106
+ def reap_all_workers
107
+ begin
108
+ loop do
109
+ worker_pid, status = Process.waitpid2(-1, Process::WNOHANG)
110
+ break unless worker_pid
111
+ worker = @workers.delete(worker_pid) and worker.tmp.close rescue nil
112
+ logger.info "reaped #{status.inspect}"
113
+ end
114
+ rescue Errno::ECHILD
115
+ end
116
+ end
117
+
118
+ def spawn_missing_workers(new_workers_count = @number_of_workers, &init_block)
119
+ new_workers_count.times do
120
+ worker = Worker.new(@worker_block, self)
121
+ worker_pid = fork do
122
+ close_resources_worker_wont_use
123
+ init_block.call if init_block
124
+ worker.work
125
+ end
126
+
127
+ worker.pid = worker_pid
128
+ @workers[worker_pid] = worker
129
+ end
130
+ end
131
+
132
+ def maintain_worker_count
133
+ number_of_missing_workers = @number_of_workers - @workers.size
134
+ return if number_of_missing_workers == 0
135
+ return spawn_missing_workers(number_of_missing_workers) if number_of_missing_workers > 0
136
+ @workers.values[0..(-number_of_missing_workers - 1)].each do |unneeded_worker|
137
+ signal_worker(:QUIT, unneeded_worker.pid) rescue nil
138
+ end
139
+ end
140
+
141
+ # forcibly terminate all workers that haven't checked in in timeout
142
+ # seconds. The timeout is implemented using an unlinked File
143
+ # shared between the parent process and each worker. The worker
144
+ # runs File#chmod to modify the ctime of the File. If the ctime
145
+ # is stale for >timeout seconds, then we'll kill the corresponding
146
+ # worker.
147
+ def murder_lazy_workers
148
+ @workers.dup.each_pair do |worker_pid, worker|
149
+ stat = worker.tmp.stat
150
+ # skip workers that disable fchmod or have never fchmod-ed
151
+ next if stat.mode == 0100600
152
+ next if (diff = (Time.now - stat.ctime)) <= @timeout
153
+ logger.error "Worker=#{worker_pid} timeout (#{diff}s > #{@timeout}s), killing"
154
+ signal_worker(:KILL, worker_pid) # take no prisoners for timeout violations
155
+ end
156
+ end
157
+
158
+ # delivers a signal to a worker and fails gracefully if the worker
159
+ # is no longer running.
160
+ def signal_worker(signal, worker_pid)
161
+ begin
162
+ Process.kill(signal, worker_pid)
163
+ rescue Errno::ESRCH
164
+ worker = @workers.delete(worker_pid) and worker.tmp.close rescue nil
165
+ end
166
+ end
167
+
168
+ # delivers a signal to each worker
169
+ def signal_each_worker(signal)
170
+ @workers.keys.each { |worker_pid| signal_worker(signal, worker_pid) }
171
+ end
172
+
173
+ def signal_quit
174
+ signal_worker(:QUIT, @pid_manager.pid)
175
+ end
176
+
177
+ def set_stdout_path(path)
178
+ redirect_io($stdout, path)
179
+ end
180
+
181
+ def set_stderr_path(path)
182
+ redirect_io($stderr, path)
183
+ end
184
+
185
+ def redirect_io(io, path)
186
+ File.open(path, 'ab') { |fp| io.reopen(fp) } if path
187
+ io.sync = true
188
+ end
189
+ end
data/preforker.gemspec ADDED
@@ -0,0 +1,74 @@
1
+ # Generated by jeweler
2
+ # DO NOT EDIT THIS FILE DIRECTLY
3
+ # Instead, edit Jeweler::Tasks in Rakefile, and run the gemspec command
4
+ # -*- encoding: utf-8 -*-
5
+
6
+ Gem::Specification.new do |s|
7
+ s.name = %q{preforker}
8
+ s.version = "0.1.0"
9
+
10
+ s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
11
+ s.authors = ["Daniel Cadenas"]
12
+ s.date = %q{2010-04-02}
13
+ s.description = %q{A gem to easily create prefork servers.}
14
+ s.email = %q{dcadenas@gmail.com}
15
+ s.extra_rdoc_files = [
16
+ "LICENSE",
17
+ "README.rdoc"
18
+ ]
19
+ s.files = [
20
+ ".document",
21
+ ".gitignore",
22
+ "LICENSE",
23
+ "README.rdoc",
24
+ "Rakefile",
25
+ "VERSION",
26
+ "examples/amqp.rb",
27
+ "examples/amqp_client.rb",
28
+ "examples/ping_pong.rb",
29
+ "lib/preforker.rb",
30
+ "lib/preforker/pid_manager.rb",
31
+ "lib/preforker/signal_processor.rb",
32
+ "lib/preforker/util.rb",
33
+ "lib/preforker/worker.rb",
34
+ "preforker.gemspec",
35
+ "spec/integration/logging_spec.rb",
36
+ "spec/integration/timeout_spec.rb",
37
+ "spec/integration/worker_control_spec.rb",
38
+ "spec/spec.opts",
39
+ "spec/spec_helper.rb",
40
+ "spec/support/integration.rb"
41
+ ]
42
+ s.homepage = %q{http://github.com/dcadenas/preforker}
43
+ s.rdoc_options = ["--charset=UTF-8"]
44
+ s.require_paths = ["lib"]
45
+ s.rubygems_version = %q{1.3.6}
46
+ s.summary = %q{A gem to easily create prefork servers.}
47
+ s.test_files = [
48
+ "spec/integration/logging_spec.rb",
49
+ "spec/integration/timeout_spec.rb",
50
+ "spec/integration/worker_control_spec.rb",
51
+ "spec/spec_helper.rb",
52
+ "spec/support/integration.rb",
53
+ "examples/amqp.rb",
54
+ "examples/amqp_client.rb",
55
+ "examples/ping_pong.rb"
56
+ ]
57
+
58
+ if s.respond_to? :specification_version then
59
+ current_version = Gem::Specification::CURRENT_SPECIFICATION_VERSION
60
+ s.specification_version = 3
61
+
62
+ if Gem::Version.new(Gem::RubyGemsVersion) >= Gem::Version.new('1.2.0') then
63
+ s.add_development_dependency(%q<rspec>, [">= 1.3.0"])
64
+ s.add_development_dependency(%q<filetesthelper>, [">= 0.10.1"])
65
+ else
66
+ s.add_dependency(%q<rspec>, [">= 1.3.0"])
67
+ s.add_dependency(%q<filetesthelper>, [">= 0.10.1"])
68
+ end
69
+ else
70
+ s.add_dependency(%q<rspec>, [">= 1.3.0"])
71
+ s.add_dependency(%q<filetesthelper>, [">= 0.10.1"])
72
+ end
73
+ end
74
+
@@ -0,0 +1,67 @@
1
+ require 'spec_helper'
2
+
3
+ include FileTestHelper
4
+ describe "Preforker" do
5
+ sandboxed_it "should redirect stdout" do
6
+ run_preforker <<-CODE
7
+ Preforker.new(:workers => 1, :stdout_path => 'test.log') do
8
+ puts "hello"
9
+ sleep 0.3 while master.wants_me_alive?
10
+ end.start
11
+ CODE
12
+
13
+ term_server
14
+ File.read("test.log").should == "hello\n"
15
+ end
16
+
17
+ sandboxed_it "should redirect stdout to the null device" do
18
+ run_preforker <<-CODE
19
+ Preforker.new(:workers => 1, :stdout_path => '/dev/null') do
20
+ puts "hello"
21
+ sleep 0.3 while master.wants_me_alive?
22
+ end.start
23
+ CODE
24
+
25
+ term_server
26
+ File.exists?("test.log").should == false
27
+ end
28
+
29
+ sandboxed_it "should redirect stderr" do
30
+ run_preforker <<-CODE
31
+ Preforker.new(:workers => 1, :stderr_path => 'test.log') do |master|
32
+ warn "hello"
33
+ sleep 0.3 while master.wants_me_alive?
34
+ end.start
35
+ CODE
36
+
37
+ term_server
38
+ File.read("test.log").should == "hello\n"
39
+ end
40
+
41
+ sandboxed_it "should have a default logger file" do
42
+ run_preforker <<-CODE
43
+ Preforker.new(:workers => 1) do
44
+ sleep 0.3 while master.wants_me_alive?
45
+ end.start
46
+ CODE
47
+
48
+ term_server
49
+ File.read("preforker.log").should =~ /Logfile created on/
50
+ end
51
+
52
+ sandboxed_it "should be possible to use the same file for logging, stdout and stderr" do
53
+ run_preforker <<-CODE
54
+ Preforker.new(:workers => 1, :stdout_path => 'test.log', :stderr_path => 'test.log', :logger => Logger.new('test.log')) do
55
+ puts "stdout string"
56
+ warn "stderr string"
57
+ sleep 0.3 while master.wants_me_alive?
58
+ end.start
59
+ CODE
60
+
61
+ term_server
62
+ log = File.read("test.log")
63
+ log.should =~ /stdout string/
64
+ log.should =~ /stderr string/
65
+ log.should =~ /Logfile created on/
66
+ end
67
+ end
@@ -0,0 +1,32 @@
1
+ require 'spec_helper'
2
+
3
+ include FileTestHelper
4
+ describe "Preforker" do
5
+ sandboxed_it "should not respawn workers when there's not a timeout" do
6
+ run_preforker <<-CODE
7
+ Preforker.new(:timeout => 2, :workers => 1) do |master|
8
+ sleep 0.1 while master.wants_me_alive?
9
+ end.start
10
+ CODE
11
+
12
+ sleep 1
13
+ term_server
14
+ log = File.read("preforker.log")
15
+ log.should_not =~ /ERROR.*timeout/
16
+ log.scan(/Child.*Created/).size.should == 1
17
+ end
18
+
19
+ sandboxed_it "should respawn workers when there's a timeout (master checks once a second max)" do
20
+ run_preforker <<-CODE
21
+ Preforker.new(:timeout => 1, :workers => 1) do
22
+ sleep 1000
23
+ end.start
24
+ CODE
25
+
26
+ sleep 1
27
+ term_server
28
+ log = File.read("preforker.log")
29
+ log.should =~ /ERROR.*timeout/
30
+ log.scan(/Child.*Created/).size.should > 1
31
+ end
32
+ end
@@ -0,0 +1,96 @@
1
+ require 'spec_helper'
2
+
3
+ include FileTestHelper
4
+ describe "Preforker" do
5
+ sandboxed_it "should quit gracefully" do
6
+ run_preforker <<-CODE
7
+ Preforker.new(:workers => 1) do |master|
8
+ sleep 0.1 while master.wants_me_alive?
9
+
10
+ master.logger.info("Main loop ended. Dying")
11
+ end.start
12
+ CODE
13
+
14
+ quit_server
15
+ log = File.read("preforker.log")
16
+ log.should =~ /Main loop ended. Dying/
17
+ end
18
+
19
+ sandboxed_it "shouldn't quit gracefully on term signal" do
20
+ run_preforker <<-CODE
21
+ Preforker.new(:workers => 1) do |master|
22
+ sleep 0.1 while master.wants_me_alive?
23
+
24
+ master.logger.info("Main loop ended. Dying")
25
+ end.start
26
+ CODE
27
+
28
+ term_server
29
+ log = File.read("preforker.log")
30
+ log.should_not =~ /Main loop ended. Dying/
31
+ end
32
+
33
+ sandboxed_it "shouldn't quit gracefully on int signal" do
34
+ run_preforker <<-CODE
35
+ Preforker.new(:workers => 1) do |master|
36
+ sleep 0.1 while master.wants_me_alive?
37
+
38
+ master.logger.info("Main loop ended. Dying")
39
+ end.start
40
+ CODE
41
+
42
+ int_server
43
+ log = File.read("preforker.log")
44
+ log.should_not =~ /Main loop ended. Dying/
45
+ end
46
+
47
+ sandboxed_it "should add a worker on ttin" do
48
+ run_preforker <<-CODE
49
+ Preforker.new(:workers => 2) do |master|
50
+ sleep 0.1 while master.wants_me_alive?
51
+ end.start
52
+ CODE
53
+
54
+ signal_server(:TTIN)
55
+ sleep 0.5
56
+ log = File.read("preforker.log")
57
+ log.scan(/Child.*Created/).size.should == 3
58
+ end
59
+
60
+ sandboxed_it "should remove a worker on ttou" do
61
+ run_preforker <<-CODE
62
+ Preforker.new(:workers => 2) do |master|
63
+ sleep 0.1 while master.wants_me_alive?
64
+ end.start
65
+ CODE
66
+
67
+ signal_server(:TTOU)
68
+ sleep 0.5
69
+ log = File.read("preforker.log")
70
+ log.scan(/Child.*Exiting/).size.should == 1
71
+ end
72
+
73
+ sandboxed_it "should remove all workers on winch" do
74
+ run_preforker <<-CODE
75
+ Preforker.new(:workers => 2) do |master|
76
+ sleep 0.1 while master.wants_me_alive?
77
+ end.start
78
+ CODE
79
+
80
+ signal_server(:WINCH)
81
+ sleep 0.5
82
+ log = File.read("preforker.log")
83
+ log.scan(/Child.*Exiting/).size.should == 2
84
+ end
85
+
86
+ sandboxed_it "should keep creating workers when they die" do
87
+ run_preforker <<-CODE
88
+ Preforker.new(:workers => 1, :timeout => 0.2) do |master|
89
+ end.start
90
+ CODE
91
+
92
+ sleep 0.3
93
+ log = File.read("preforker.log")
94
+ log.scan(/Child.*Created/).size.should > 1
95
+ end
96
+ end
data/spec/spec.opts ADDED
@@ -0,0 +1 @@
1
+ --color
@@ -0,0 +1,12 @@
1
+ $LOAD_PATH.unshift(File.dirname(__FILE__))
2
+ $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
3
+ require 'preforker'
4
+ require 'spec'
5
+ require 'spec/autorun'
6
+ require 'rubygems'
7
+ require 'filetesthelper'
8
+ Dir[File.dirname(__FILE__) + "/support/**/*.rb"].each {|f| require f}
9
+
10
+ include Integration
11
+ Spec::Runner.configure do |config|
12
+ end
@@ -0,0 +1,52 @@
1
+ module Integration
2
+ PREFORKER_LIB_PATH = File.expand_path(File.join(File.dirname(__FILE__), '..', '..', 'lib'))
3
+ def run_preforker(code)
4
+ File.open("test.rb", 'w') do |f|
5
+ f.write <<-TOP
6
+ $LOAD_PATH.unshift('#{PREFORKER_LIB_PATH}')
7
+ require 'preforker'
8
+ TOP
9
+ f.write(code)
10
+ end
11
+
12
+ system("ruby test.rb")
13
+ end
14
+
15
+ def sandboxed_it(desc)
16
+ it desc do
17
+ with_files do
18
+ yield
19
+ term_server
20
+ end
21
+ end
22
+ end
23
+
24
+ def signal_server(signal)
25
+ Dir['*.pid'].each do |pid_file_path|
26
+ Process.kill(signal, File.read(pid_file_path).to_i)
27
+ end
28
+ end
29
+
30
+ def term_server
31
+ signal_server(:TERM)
32
+ wait_till_server_ends
33
+ end
34
+
35
+ def quit_server
36
+ signal_server(:QUIT)
37
+ wait_till_server_ends
38
+ end
39
+
40
+ def int_server
41
+ signal_server(:INT)
42
+ wait_till_server_ends
43
+ end
44
+
45
+ def wait_till_server_ends
46
+ Dir['*.pid'].each do |pid_file_path|
47
+ while File.exists?(pid_file_path) do
48
+ sleep 0.2
49
+ end
50
+ end
51
+ end
52
+ end
metadata ADDED
@@ -0,0 +1,117 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: preforker
3
+ version: !ruby/object:Gem::Version
4
+ prerelease: false
5
+ segments:
6
+ - 0
7
+ - 1
8
+ - 0
9
+ version: 0.1.0
10
+ platform: ruby
11
+ authors:
12
+ - Daniel Cadenas
13
+ autorequire:
14
+ bindir: bin
15
+ cert_chain: []
16
+
17
+ date: 2010-04-02 00:00:00 -03:00
18
+ default_executable:
19
+ dependencies:
20
+ - !ruby/object:Gem::Dependency
21
+ name: rspec
22
+ prerelease: false
23
+ requirement: &id001 !ruby/object:Gem::Requirement
24
+ requirements:
25
+ - - ">="
26
+ - !ruby/object:Gem::Version
27
+ segments:
28
+ - 1
29
+ - 3
30
+ - 0
31
+ version: 1.3.0
32
+ type: :development
33
+ version_requirements: *id001
34
+ - !ruby/object:Gem::Dependency
35
+ name: filetesthelper
36
+ prerelease: false
37
+ requirement: &id002 !ruby/object:Gem::Requirement
38
+ requirements:
39
+ - - ">="
40
+ - !ruby/object:Gem::Version
41
+ segments:
42
+ - 0
43
+ - 10
44
+ - 1
45
+ version: 0.10.1
46
+ type: :development
47
+ version_requirements: *id002
48
+ description: A gem to easily create prefork servers.
49
+ email: dcadenas@gmail.com
50
+ executables: []
51
+
52
+ extensions: []
53
+
54
+ extra_rdoc_files:
55
+ - LICENSE
56
+ - README.rdoc
57
+ files:
58
+ - .document
59
+ - .gitignore
60
+ - LICENSE
61
+ - README.rdoc
62
+ - Rakefile
63
+ - VERSION
64
+ - examples/amqp.rb
65
+ - examples/amqp_client.rb
66
+ - examples/ping_pong.rb
67
+ - lib/preforker.rb
68
+ - lib/preforker/pid_manager.rb
69
+ - lib/preforker/signal_processor.rb
70
+ - lib/preforker/util.rb
71
+ - lib/preforker/worker.rb
72
+ - preforker.gemspec
73
+ - spec/integration/logging_spec.rb
74
+ - spec/integration/timeout_spec.rb
75
+ - spec/integration/worker_control_spec.rb
76
+ - spec/spec.opts
77
+ - spec/spec_helper.rb
78
+ - spec/support/integration.rb
79
+ has_rdoc: true
80
+ homepage: http://github.com/dcadenas/preforker
81
+ licenses: []
82
+
83
+ post_install_message:
84
+ rdoc_options:
85
+ - --charset=UTF-8
86
+ require_paths:
87
+ - lib
88
+ required_ruby_version: !ruby/object:Gem::Requirement
89
+ requirements:
90
+ - - ">="
91
+ - !ruby/object:Gem::Version
92
+ segments:
93
+ - 0
94
+ version: "0"
95
+ required_rubygems_version: !ruby/object:Gem::Requirement
96
+ requirements:
97
+ - - ">="
98
+ - !ruby/object:Gem::Version
99
+ segments:
100
+ - 0
101
+ version: "0"
102
+ requirements: []
103
+
104
+ rubyforge_project:
105
+ rubygems_version: 1.3.6
106
+ signing_key:
107
+ specification_version: 3
108
+ summary: A gem to easily create prefork servers.
109
+ test_files:
110
+ - spec/integration/logging_spec.rb
111
+ - spec/integration/timeout_spec.rb
112
+ - spec/integration/worker_control_spec.rb
113
+ - spec/spec_helper.rb
114
+ - spec/support/integration.rb
115
+ - examples/amqp.rb
116
+ - examples/amqp_client.rb
117
+ - examples/ping_pong.rb