preforker 0.1.0 → 0.1.1

data/README.rdoc

@@ -1,43 +1,67 @@
 = preforker
 
-A gem to easily create prefork servers.
+A gem to easily create protocol agnostic prefork servers.
 
 == Example
 
 Let's see an example using the Mac 'say' command.
 
+  require 'rubygems'
   require 'preforker'
 
-  #you can open some socket here or reserve any other resource you want to share with your workers, this is master space
+  #At this point we can open some socket or reserve any other resource you want to share with your workers.
+  #Whatever we do before Preforker.new is ran only in the master process.
+  say `hi, I\\'m the master`
+
   Preforker.new(:timeout => 10) do |master|
-    #this block is only run from each worker (10 by default)
+    #this block is only run by each worker (10 by default)
 
-    #here you should write the code that is needed to be ran each time a fork is created, initializations, etc.
-    `say hi`
+    #first you should write the code that is needed to be ran
+    #once when a new fork is created, initializations, etc.
+    `say hi, I\\'m a worker`
 
-    #here you could IO.select a socket, run an EventMachine service (see example), or just run worker loop
-    #you need to ask master if it wants you alive periodically or else it will kill you after the timeout elapses. Respect your master!
+    #now you could IO.select a socket, run an EventMachine service (see example), or as we do here,
+    #just run a worker loop. Notice that you need to ask master if it wants you alive periodically or
+    #else it will kill you after the timeout elapses. Respect your master!
     while master.wants_me_alive? do
       sleep 1
       `say ping pong`
     end
 
-    #here we can do whatever we want when exiting gracefully
+    #here we can do whatever we want to exit gracefully
     `say bye`
+
+
+  #here we are using #start to daemonize. We could have used #run to just block and then send the INT signal with ctrl-c to stop
   end.start
 
-To kill the server just run:
+  puts "I'm the launcher that forked off master, bye bye"
+  puts "To kill the server just do: kill -s QUIT `cat preforker.pid`
+
+Remember that to kill the server you need to:
 
   kill -s QUIT `cat preforker.pid`
 
 See the examples directory and the specs for more examples.
 
+== Why? I can always use threads!
+
+As always, it's a matter of trade offs, so it depends on how you ponder the advantages and disadvantages of a preforking architecture for your particular case.
+Still notice that you could have a mix of threads and processes, they are independent concepts.
+
+=== Advantages
+[Reliability] You may be using a third party library or a C extension with memory leaks or segfaults that could break your entire ruby process. If you use a preforking architecture you can just kill the misbehaving process without affecting the rest of your healthy workers.
+[Simplicity] When creating servers you can reduce the amount of extra code (thread pools, triggering, callbacks, etc) needed to deal with concurrency. Still you should always be aware that your app will be concurrent and correctly deal with shared resources, locking and possible race conditions. The concurrency aspect is very decoupled from your app (although in a coarse grained way) so in some cases you can add concurrency to a legacy library without changing its code, just by adding processes.
+
+=== Disadvantages
+[Efficiency] Threads are more fine grained so you have more control over which parts of your app should be concurrent. Efficiency is also better because context switching is cheap in threaded systems. But note that AFAIK this is still not the case in Ruby, specially if not using REE, see {here}[http://timetobleed.com/ruby-hoedown-slides/].
+
 == Configuration options
 
 [:timeout] The timeout in seconds, 5 by default. If a worker takes more than this it will be killed and respawned.
 [:workers] Number of workers, 10 by default.
-[:stdout_path] Path to redirect stdout to. By default it's /dev/null
-[:stderr_path] Path to redirect stderr to. By default it's /dev/null
+[:stdout_path] Path to redirect stdout to. By default it's the log file. You may prefer to use /dev/null or /dev/stdout
+[:stderr_path] Path to redirect stderr to. By default it's the log file. You may prefer to use /dev/null or /dev/stderr
 [:app_name] The app name, 'preforker' by default. Used for some ps message, log messages messages and pid file name.
 [:pid_path] The path to the pid file for this server. By default it's './preforker.pid'.
 [:logger] This is Logger.new('./preforker.log') by default
@@ -52,15 +76,19 @@ You can send some signals to master to control the way it handles the workers li
 [QUIT] Kill workers and master in a graceful way
 [TERM, INT] Kill workers and master immediately
 
+== Installation
+
+  gem install preforker
+
 == Acknowledgments
 
-Most of the preforking operating system tricks come from Unicorn. Check out its source code for a hands on tutorial on Unix internals!
+Most of the preforking operating system tricks come from {Unicorn}[http://unicorn.bogomips.org/]. I checked out its source code and read {this}[http://tomayko.com/writings/unicorn-is-unix] great introduction by {rtomayko}[http://github.com/rtomayko].
 
 == To do list
 
 * More tests
 * Log rotation through the USR1 signal
-* Have something like min_spare_workers, max_workers
+* Have something like min_spare_workers, max_workers, max_request_per_worker
 
 == Note on Patches/Pull Requests
 

data/VERSION

@@ -1 +1 @@
-0.1.0
+0.1.1

data/examples/em_echo_server.rb

@@ -0,0 +1,59 @@
+require 'rubygems'
+require 'preforker'
+require 'eventmachine'
+
+class EchoServer < EM::Connection
+  def notify_readable
+    while socket = @io.accept_nonblock
+      message = socket.gets
+      socket.write message
+      #let's fake some more work is done
+      sleep 0.3
+      socket.close
+    end
+  rescue Errno::EAGAIN, Errno::ECONNABORTED
+  end
+
+  def unbind
+    detach
+    @io.close
+  end
+end
+
+socket = TCPServer.new("0.0.0.0", 8081)
+socket.listen(100)
+EventMachine.epoll
+
+Preforker.new(:timeout => 5, :workers => 4, :app_name => "EM example") do |master|
+  EventMachine::run do
+    EM.add_periodic_timer(4) do
+      EM.stop_event_loop unless master.wants_me_alive?
+    end
+
+    EM.watch(socket, EchoServer, self){ |c| c.notify_readable = true}
+    puts "Listening..."
+  end
+end.start
+
+__END__
+ab -c 4 -n 100 "http://127.0.0.1:8081/"
+
+Concurrency Level:      4
+Time taken for tests:   8.202 seconds
+Complete requests:      100
+
+
+##################################
+Compare it with this other implementation that doesn't use preforker (or EM.defer)
+
+EventMachine::run do
+  EM.watch(socket, EchoServer, self){ |c| c.notify_readable = true}
+  puts "Listening..."
+end
+
+ab -c 4 -n 100 "http://127.0.0.1:8081/"
+
+Concurrency Level:      4
+Time taken for tests:   29.729 seconds
+Complete requests:      100
+

data/examples/ping_pong.rb

@@ -1,11 +1,14 @@
+require 'rubygems'
 require 'preforker'
 
 #you can open some socket here or reserve any other resource you want to share with your workers, this is master space
+`say hi, I\\'m the master`
+
 Preforker.new(:timeout => 10) do |master|
   #this block is only run from each worker (10 by default)
 
-  #here you should write the code that is needed to be ran each time a fork is created, initializations, etc.
-  `say hi`
+  #here you should write the code that is needed to be ran once each time a fork is created, initializations, etc.
+  `say hi, I\\'m a worker`
 
   #here you could IO.select a socket, run an EventMachine service (see example), or just run worker loop
   #you need to ask master if it wants you alive periodically or else it will kill you after the timeout elapses. Respect your master!
@@ -16,6 +19,9 @@ Preforker.new(:timeout => 10) do |master|
 
   #here we can do whatever we want when exiting gracefully
   `say bye`
+
+#we can use run instead of start to run the server without daemonizing
 end.start
 
-#to kill the server just run: kill -s QUIT `cat preforker.pid`
+puts "I'm the launching process that forked to create master, I did my job. Enjoy the noisy ping pong championship, bye bye!"
+puts "To kill the server just do: kill -s QUIT `cat preforker.pid`

data/lib/preforker.rb

@@ -30,27 +30,33 @@ class Preforker
     $0 = "#@app_name Master"
   end
 
-  def start
-    launch do |ready_write|
-      $stdin.reopen("/dev/null")
-      set_stdout_path(@options[:stdout_path])
-      set_stderr_path(@options[:stderr_path])
+  def run(ready_write = nil)
+    $stdin.reopen("/dev/null")
+    set_stdout_path(@options[:stdout_path])
+    set_stderr_path(@options[:stderr_path])
 
-      logger.info "Master started"
+    logger.info "Master started"
 
-      pid_path = @options[:pid_path] || "./#{@app_name.downcase}.pid"
-      @pid_manager = PidManager.new(pid_path)
-      @signal_processor = SignalProcessor.new(self)
+    pid_path = @options[:pid_path] || "./#{@app_name.downcase}.pid"
+    @pid_manager = PidManager.new(pid_path)
+    @signal_processor = SignalProcessor.new(self)
 
-      spawn_missing_workers do
-        ready_write.close
-      end
+    spawn_missing_workers do
+      ready_write.close if ready_write
+    end
 
-      #tell parent we are ready
+    #tell parent we are ready
+    if ready_write
       ready_write.syswrite($$.to_s)
       ready_write.close rescue nil
+    end
 
-      @signal_processor.start_signal_loop
+    @signal_processor.start_signal_loop
+  end
+
+  def start
+    launch do |ready_write|
+      run(ready_write)
     end
   end
 
@@ -77,7 +83,6 @@ class Preforker
        exit!(1)
      else
        puts "Server started successfuly"
-       exit(0)
      end
   end
 

data/preforker.gemspec

@@ -5,11 +5,11 @@
 
 Gem::Specification.new do |s|
   s.name = %q{preforker}
-  s.version = "0.1.0"
+  s.version = "0.1.1"
 
   s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
   s.authors = ["Daniel Cadenas"]
-  s.date = %q{2010-04-02}
+  s.date = %q{2010-04-11}
   s.description = %q{A gem to easily create prefork servers.}
   s.email = %q{dcadenas@gmail.com}
   s.extra_rdoc_files = [
@@ -25,6 +25,7 @@ Gem::Specification.new do |s|
      "VERSION",
      "examples/amqp.rb",
      "examples/amqp_client.rb",
+     "examples/em_echo_server.rb",
      "examples/ping_pong.rb",
      "lib/preforker.rb",
      "lib/preforker/pid_manager.rb",
@@ -52,6 +53,7 @@ Gem::Specification.new do |s|
      "spec/support/integration.rb",
      "examples/amqp.rb",
      "examples/amqp_client.rb",
+     "examples/em_echo_server.rb",
      "examples/ping_pong.rb"
   ]
 

metadata

@@ -5,8 +5,8 @@ version: !ruby/object:Gem::Version
   segments: 
   - 0
   - 1
-  - 0
-  version: 0.1.0
+  - 1
+  version: 0.1.1
 platform: ruby
 authors: 
 - Daniel Cadenas
@@ -14,7 +14,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2010-04-02 00:00:00 -03:00
+date: 2010-04-11 00:00:00 -03:00
 default_executable: 
 dependencies: 
 - !ruby/object:Gem::Dependency 
@@ -63,6 +63,7 @@ files:
 - VERSION
 - examples/amqp.rb
 - examples/amqp_client.rb
+- examples/em_echo_server.rb
 - examples/ping_pong.rb
 - lib/preforker.rb
 - lib/preforker/pid_manager.rb
@@ -114,4 +115,5 @@ test_files:
 - spec/support/integration.rb
 - examples/amqp.rb
 - examples/amqp_client.rb
+- examples/em_echo_server.rb
 - examples/ping_pong.rb