FooBarWidget-daemon_controller 0.1.0 → 0.2.0

data/LICENSE.txt

@@ -0,0 +1,20 @@
+Copyright (c) 2008 Phusion
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
+

data/README.markdown

@@ -0,0 +1,387 @@
+Introduction
+============
+
+There is a lot of software (both Rails related and unrelated) which rely on
+servers or daemons. To name a few, in no particular order:
+
+ * [Ultrasphinx](http://blog.evanweaver.com/files/doc/fauna/ultrasphinx/), a
+   Rails library for full-text searching. It makes use the [Sphinx search
+   software](http://www.sphinxsearch.com/) for indexing and searching. Indexing
+   is done by running a command, while searching is done by querying the Sphinx
+   search server.
+ * [acts_as_ferret](http://projects.jkraemer.net/acts_as_ferret/wiki), another
+   Rails library for full-text searching. It uses the Ferret search software.
+   On production environments, it relies on the Ferret DRB server for both
+   searching and indexing.
+ * [BackgrounDRb](http://backgroundrb.rubyforge.org/), a Ruby job server and
+   scheduler. Scheduling is done by contacting the BackgrounDRb daemon.
+ * [mongrel_cluster](http://mongrel.rubyforge.org/wiki/MongrelCluster), which
+   starts and stops multiple Mongrel daemons.
+
+Relying on daemons is quite common, but not without problems. Let's go over
+some of them.
+
+### Starting daemons is a hassle
+
+If you've used similar software, then you might agree that managing these
+daemons are a hassle. If you're using BackgrounDRb, then the daemon must be
+running. Starting the daemon is not hard, but it is annoying. It's also
+possible that the system administrator forgets to start the daemon. While
+configuring the system to automatically start a daemon at startup is not hard,
+it is an extra thing to do, and thus a hassle. We thought, why can't such
+daemons be automatically started? Indeed, this won't be possible if the daemon
+is to be run on a remote machine. But in by far the majority of use cases, the
+daemon runs on the same host as the Rails application. If a Rails application -
+or indeed, <em>any</em> application - is configured to contact a daemon on the
+local host, then why not start the daemon automatically on demand?
+
+### Daemon starting code may not be robust or efficient
+
+We've also observed that people write daemon controlling code over and over
+again. Consider for example UltraSphinx, which provides a
+`rake sphinx:daemon:start` Rake task to start the daemon. The time that a
+daemon needs to initialize is variable, and depends on things such as the
+current system load. The Sphinx daemon usually needs less than a second before
+we can connect to it. However, the way different software handles starting of a
+daemon varies. We've observed that waiting a fixed amount of time is by far the
+most common way. For example, UltraSphinx's daemon starting code looks like
+this:
+
+    system "searchd --config '#{Ultrasphinx::CONF_PATH}'"
+    sleep(4) # give daemon a chance to write the pid file
+    if ultrasphinx_daemon_running?
+       say "started successfully"
+    else
+       say "failed to start"
+    end
+
+This is in no way a slam against UltraSphinx. However, if the daemon starts in
+200 miliseconds, then the user who issued the start command will be waiting for
+3.8 seconds for no good reason. This is not good for usability or for the
+user's patience.
+
+### Startup error handling
+
+Different software handles daemon startup errors in different ways. Some might
+not even handle errors at all. For example, consider 'mongrel_cluster'. If
+there's a typo in one of your application source files, then 'mongrel_cluster'
+will not report the error. Instead, you have to check its log files to see what
+happened. This is not good for usability: many people will be wondering why
+they can't connect to their Mongrel ports after issuing a
+`mongrel_rails cluster::start` - until they realize that they should read the
+log file. But the thing is, not everybody realizes this. And typing in an extra
+command to read the log file to check whether Mongrel started correctly, is
+just a big hassle. Why can't the daemon startup code report such errors
+immediately?
+
+### Stale or corrupt Pid files
+
+Suppose that you're running a Mongrel cluster, and your server suddenly powers
+off because of a power outage. When the server is online again, it fails to
+start your Mongrel cluster because the PID file that it had written still
+exists, and wasn't cleaned up properly (it's supposed to be cleaned up when
+Mongrel exits). mongrel_cluster provides the `--clean` option to check whether
+the PID file is *stale*, and will automatically clean it up if it is. But not
+all daemon controlling software supports this. Why can't all software check for
+stale PID files automatically?
+
+
+Implementation problems
+=======================
+
+From the problem descriptions, it would become apparent that our wishlist is as
+follows. Why is this wishlist often not implemented? Let's go over them.
+
+ -  **A daemon should be automatically started on demand, instead of requiring the user to manually start it.**
+    
+    The most obvious problems are related to concurrency. Suppose that your web
+    application has a search box, and you want to start the search daemon if it
+    isn't already started, then connect to. Two problems will arise:
+    
+     *  Suppose that Rails process A is still starting the daemon. At the same
+        time, another visitor tries to search something, and Rails process B
+        notices that the daemon is not running. If B tries to start the daemon
+        while it's already being started by A, then things can go wrong.
+        *A robust daemon starter must ensure that only one process at the same time may start the daemon.*
+     *  It's not a good idea to wait a fixed amount of time for the daemon to
+        start, because you don't know in advance how long it will take for it to
+        start. For example, if you wait 2 seconds, then try to connect to the
+        daemon, and the daemon isn't done initializing yet, then it will seem as
+        if the daemon failed to start.
+    
+    These are the most probable reasons why people don't try to write
+    auto-starting code, and instead require the user to start the daemon
+    manually.
+    
+    These problems, as well as several less obvious problems, are closely
+    related to the next few points.
+    
+ -  **The daemon starter must wait until the daemon is done initializing, no longer and no shorter**
+    
+    Because only after the daemon is fully initialized, is it safe to connect
+    to it. And because the user should not have to wait longer than he really
+    has to. During startup, the daemon will have to be continuously checked
+    whether it's done initializing or whether an error occured. Writing this
+    code can be quite a hassle, which is why most people don't do it.
+    
+ -  **The daemon starter must report any startup errors**
+    
+    If the daemon starting command - e.g. `sphinx -c config_file.conf`,
+    `apachectl start` or `mongrel_rails cluster::start` - reports startup
+    errors, then all is fine as long as the user is starting the command from a
+    terminal. A problem occurs when the error occurs after the daemon has
+    already gone into the background. Such errors are only reported to the log
+    file.
+    *The daemon starter should also check the log file for any startup errors.*
+    
+    Furthermore, it should be able to raise startup errors as exceptions. This
+    allows the the application to decide what to do with the error. For less
+    experienced system administrators, the error might be displayed in the
+    browser, allowing the administrators to become aware of the problem without
+    forcing them to manually check the log files. Or the error might be emailed
+    to a system administrator's email address.
+    
+ -  **The daemon starter must be able to correct stale or corrupted PID files**
+    
+    If the PID file is stale, or for some reason has been corrupted, then the
+    daemon starter must be able to cope with that.
+    *It should check whether the PID file contains a valid PID, and whether the PID exists.*
+
+
+Introducing daemon_controller
+=============================
+
+*daemon_controller* is a library for managing daemons in a robust manner. It is
+not a tool for managing daemons. Rather, it is a library which lets you write
+applications that manage daemons in a robust manner. For example,
+'mongrel_cluster' or UltraSphinx may be adapted to utilize this library, for
+more robust daemon management.
+
+*daemon_controller* implements all items in the aforementioned wishlist. It
+provides the following functionalities:
+
+### Starting a daemon
+
+This ensures that no two processes can start the same daemon at the same time.
+It will also reports any startup errors, even errors that occur after the
+daemon has already gone into the background but before it has fully initialized
+yet. It also allows you to set a timeout, and will try to abort the daemon if
+it takes too long to initialize.
+
+The start function won't return until the daemon has been fully initialized,
+and is responding to connections. So if the start function has returned, then
+the daemon is guaranteed to be usable.
+
+### Stopping a daemon
+
+It will stop the daemon, but only if it's already running. Any errors
+are reported. If the daemon isn't already running, then it will silently
+succeed. Just like starting a daemon, you can set a timeout for stopping the
+daemon.
+
+Like the start function, the stop function won't return until the daemon is no
+longer running. This makes it save to immediately start the same daemon again
+after having stopped it, without worrying that the previous daemon instance
+hasn't exited yet and might conflict with the newly started daemon instance.
+
+### Connecting to a daemon, starting it if it isn't running
+
+Every daemon has to be connected to using a different way. As a developer, you
+tell 'daemon_controller' how to connect to the daemon. It will then attempt to
+do that, and if that fails, it will check whether the daemon is running. If it
+isn't running, then it will automatically start the daemon, and attempt to
+connect to the daemon again. Failures are reported.
+
+### Checking whether a daemon is running
+
+This information is retrieved from the PID file. It also checks whether the PID
+file is stale.
+
+### All failures are reported via exceptions
+
+So that you can exactly determine how you want to handle errors.
+
+### Lots and lots of error checking
+
+So that there are very few ways in which the system can screw up.
+
+daemon_controller's goal is to make daemon management less of a hassle, and as
+automatic and straightforward as possible.
+
+
+Tutorial #1: controlling Apache
+===============================
+
+Suppose that you're a [Phusion Passenger](http://www.modrails.com/) developer,
+and you need to write tests for the Apache module. In particular, you want to
+test whether the different Phusion Passenger configuration directives are
+working as expected. Obviously, to test the Apache module, the Apache web
+server must be running. For every test, you will want the unit test suite to:
+
+ 1. Write an Apache configuration file, with the relevant configuration
+    directive set to a specific value.
+ 2. Start Apache.
+ 3. Send an HTTP request to Apache and check whether the HTTP response matches
+    your expectations.
+ 4. Stop Apache.
+
+That can be done with the following code:
+
+    require 'daemon_controller'
+    
+    File.open("apache.conf", "w") do |f|
+       f.write("PidFile apache.pid\n")
+       f.write("LogFile apache.log\n")
+       f.write("Listen 1234\n")
+       f.write(... other relevant configuration options ...)
+    end
+    
+    controller = DaemonController.new(
+       :identifier    => 'Apache web server',
+       :start_command => 'apachectl -f apache.conf -k start',
+       :ping_command  => lambda { TCPSocket.new('localhost', 1234) },
+       :pid_file      => 'apache.pid',
+       :log_file      => 'apache.log',
+       :timeout       => 25
+    )
+    controller.start
+    
+    .... apache is now started ....
+    .... some test code here ....
+    
+    controller.stop
+
+The `File.open` line is obvious: it writes the relevant Apache configuration
+file.
+
+The next line is for creating a new DaemonController object. We pass a
+human-readable identifier for this daemon ("Apache web server") to the
+constructor. This is used for generating friendlier error messages.
+We also tell it how Apache is supposed to be started (`:start_command`), how to
+check whether it can be connected to (`:ping_command`), and where its PID file
+and log file is. If Apache failed with an error during startup, then it will be
+reported. If Apache failed with an error after it has gone into the background,
+then that will be reported too: the given log file is monitored for new error
+messages.
+Finally, a timeout of 25 seconds is given. If Apache doesn't start within 25
+seconds, then an exception will be raised.
+
+The ping command is just a `Proc` which returns true or false. If the Proc
+raises `Errno::ECONNREFUSED`, then that's also interpreted by DaemonController
+as meaning that the daemon isn't responding yet.
+
+After `controller.start` has returned, we can continue with the test case. At
+this point, we know that Apache has done initializing.
+When we're done with Apache, we stop it with `controller.stop`. This does not
+return until Apache has fully stopped.
+
+The cautious reader might notice that the socket returned by the ping command
+is never closed. That's true, because DaemonController will close it
+automatically for us, if it notices that the ping command proc's return value
+responds to `#close`.
+
+From this example, it becomes apparent that for daemon_controller to work, you
+must know how to start the daemon, how to contact the daemon, and you must know
+where it will put its PID file and log file.
+
+
+Tutorial #2: Sphinx indexing and search server management
+=========================================================
+
+We at Phusion are currently developing a web application with full-text search
+capabilities, and we're using Sphinx for this purpose. We want to make the
+lives of our developers and our system administrators as easy as possible, so
+that there's little room for human screw-up, and so we've developed this
+library. Our Sphinx search daemon is completely managed through this library
+and is automatically started on demand.
+
+Our Sphinx config file is generated from an ERB template. This ERB templates
+writes different values in the config file, depending on whether we're in
+development, test or production mode. We will want to regenerate this config
+file every time, just before we start the search daemon.
+But there's more. The search daemon will fail if there is no search index. If a
+new developer has just checked out the application's source code, then there is
+no search index yet. We don't want him to go through the pain of having to
+generate the index manually. (That said, it isn't that much of a pain, but it's
+just yet-another-thing to do, which can and should be automated.) So before
+starting the daemon, we will also want to check whether the index exists. If
+not, then we'll generate it, and then start the daemon. Of course, no two Rails
+processes may generate the config file or the index at the same time.
+
+When querying the search server, we will want to automatically start it if it
+isn't running.
+
+This can be achieved with the following code:
+
+    require 'daemon_controller'
+    
+    class SearchServer
+       SEARCH_SERVER_PORT = 1234
+    
+       def initialize
+          @controller = DaemonController.new(
+             :identifier => 'Sphinx search server',
+             :start_command => "searchd -c config/sphinx.conf",
+             :before_start => method(:before_start),
+             :ping_command => lambda { TCPSocket.new('localhost', SEARCH_SERVER_PORT) },
+             :pid_file => 'tmp/pids/sphinx.pid',
+             :log_file => 'log/sphinx.log',
+       end
+       
+       def query(search_terms)
+          socket = @controller.connect do
+             TCPSocket.new('localhost', SEARCH_SERVER_PORT)
+          end
+          send_query(socket, search_terms)
+          return retrieve_results(socket)
+       end
+       
+    private
+       def before_start
+          generate_configuration_file
+          if !index_exists?
+             generate_index
+          end
+       end
+       
+       ...
+    end
+
+Notice the `:before_start` option. We pass a block of code which is to be run,
+just before the daemon is started. This block, along with starting the daemon,
+is completely serialized. That is, if you're inside the block, then it's
+guaranteed that no other process is running this block at the same time as well.
+
+The `#query` method is the method for querying the search server with search
+terms. It returns a list of result. It uses `DaemonController#connect`: one
+passes a block of that method, which contains code for connecting to the
+daemon. If the block returns nil, or if it raises `Errno::ECONNREFUSED`, then
+`DaemonController#connect` will automatically take care of auto-starting the
+Sphinx daemon for us.
+
+
+A little bit of history
+=======================
+
+The issue of managing daemons has been a thorn in our eyes for quite some time
+now. Until now, we've solved this problem by equipping any daemons that we
+write with the ability to gracefully handle being concurrently started, the
+ability to initialize as much as possible *before* forking into the background,
+etc. However, equipping all this robustness into our code over and over is a
+lot of work. We've considered documenting a standard behavior for daemons so
+that they can properly support auto-starting and such.
+
+However, we've recently realized that that's probably a futile effort.
+Convincing everybody to write a lot of code for a bit more robustness is
+probably not realistic. So we took the pragmatic approach and developed a
+library which adds more robustness on top of daemons' existing behavior. And
+thus, daemon_controller was born. It is a little bit less efficient compared to
+when the daemon is designed from the beginning with such abilities in mind, but
+it's compatible with virtually all daemons, and is easy to use.
+
+
+API documentation
+=================
+
+Detailed API documentation is available in the form of inline comments in
+`lib/daemon_controller.rb`.

data/daemon_controller.gemspec

@@ -1,17 +1,18 @@
 Gem::Specification.new do |s|
   s.name = "daemon_controller"
-  s.version = "0.1.0"
+  s.version = "0.2.0"
   s.date = "2008-08-21"
   s.summary = "A library for implementing daemon management capabilities"
   s.email = "hongli@phusion.nl"
   s.homepage = "http://github.com/FooBarWidget/daemon_controller/tree/master"
-  s.description = "A library for implementing daemon management capabilities."
-  s.has_rdoc = false
+  s.description = "A library for robust daemon management."
+  s.has_rdoc = true
   s.authors = ["Hongli Lai"]
   
   s.files = [
-      "README.rdoc", "LICENSE.txt", "daemon_controller.gemspec",
+      "README.markdown", "LICENSE.txt", "daemon_controller.gemspec",
       "lib/daemon_controller.rb",
+      "lib/daemon_controller/lock_file.rb",
       "spec/daemon_controller_spec.rb",
       "spec/echo_server.rb"
   ]

data/lib/daemon_controller/lock_file.rb

@@ -0,0 +1,48 @@
+# daemon_controller, library for robust daemon management
+# Copyright (c) 2008 Phusion
+# 
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+# 
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+# 
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+# THE SOFTWARE.
+
+class DaemonController
+class LockFile
+	def initialize(filename)
+		@filename = filename
+	end
+	
+	def exclusive_lock
+		File.open(@filename, 'w') do |f|
+			if Fcntl.const_defined? :F_SETFD
+				f.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC)
+			end
+			f.flock(File::LOCK_EX)
+			yield
+		end
+	end
+	
+	def shared_lock
+		File.open(@filename, 'w') do |f|
+			if Fcntl.const_defined? :F_SETFD
+				f.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC)
+			end
+			f.flock(File::LOCK_SH)
+			yield
+		end
+	end
+end # class PidFile
+end # class DaemonController

data/lib/daemon_controller.rb

@@ -1,20 +1,33 @@
-# Basic functionality for a single, local, external daemon:
-# - starting daemon
-#   * must be concurrency-safe!
-#   * must be able to report startup errors!
-#   * returns when daemon is fully operational
-# - stopping daemon
-#   * must be concurrency-safe!
-#   * returns when daemon has exited
-# - querying the status of a daemon
-#   * querying the status of a daemon (i.e. whether it's running)
-# - connect to a daemon, and start it if it isn't already running
-#   * must be a single atomic action
+# daemon_controller, library for robust daemon management
+# Copyright (c) 2008 Phusion
+# 
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+# 
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+# 
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+# THE SOFTWARE.
 
 require 'tempfile'
 require 'fcntl'
+require File.expand_path(File.dirname(__FILE__) << '/daemon_controller/lock_file')
 
+# Main daemon controller object. See the README for an introduction and tutorial.
 class DaemonController
+	ALLOWED_CONNECT_EXCEPTIONS = [Errno::ECONNREFUSED, Errno::ENETUNREACH,
+		Errno::ETIMEDOUT, Errno::ECONNRESET]
+	
 	class Error < StandardError
 	end
 	class TimeoutError < Error
@@ -57,6 +70,15 @@ class DaemonController
 	#  
 	#  The value may also be a Proc, which returns an expression that evaluates to
 	#  true (indicating that the daemon can be connected to) or false (failure).
+	#  If the Proc raises Errno::ECONNREFUSED, Errno::ENETUNREACH, Errno::ETIMEDOUT
+	#  or Errno::ECONNRESET, then that also means that the daemon cannot be connected
+	#  to.
+	#  <b>NOTE:</b> if the ping command returns an object which responds to
+	#  <tt>#close</tt>, then that method will be called on the return value.
+	#  This makes it possible to specify a ping command such as
+	#  <tt>lambda { TCPSocket.new('localhost', 1234) }</tt>, without having to worry
+	#  about closing it afterwards.
+	#  Any exceptions raised by #close are ignored.
 	#  
 	# [:pid_file]
 	#  The PID file that the daemon will write to. Used to check whether the daemon
@@ -73,6 +95,10 @@ class DaemonController
 	#  by killing the PID written in the PID file.
 	#  
 	#  The default value is +nil+.
+	#
+	# [:before_start]
+	#  This may be a Proc. It will be called just before running the start command.
+	#  The before_start proc is not subject to the start timeout.
 	#  
 	# [:start_timeout]
 	#  The maximum amount of time, in seconds, that #start may take to start
@@ -118,6 +144,7 @@ class DaemonController
 		@ping_interval = options[:ping_interval] || 0.1
 		@pid_file = options[:pid_file]
 		@log_file = options[:log_file]
+		@before_start = options[:before_start]
 		@start_timeout = options[:start_timeout] || 15
 		@stop_timeout = options[:stop_timeout] || 15
 		@log_file_activity_timeout = options[:log_file_activity_timeout] || 7
@@ -132,7 +159,7 @@ class DaemonController
 	# - StartTimeout - the daemon did not start in time. This could also
 	#   mean that the daemon failed after it has gone into the background.
 	def start
-		exclusive_lock do
+		@lock_file.exclusive_lock do
 			start_without_locking
 		end
 	end
@@ -142,8 +169,8 @@ class DaemonController
 	# started.
 	#
 	# The block must return nil or raise Errno::ECONNREFUSED, Errno::ENETUNREACH,
-	# or Errno::ETIMEDOUT to indicate that the daemon cannot be connected to.
-	# It must return non-nil if the daemon can be connected to.
+	# Errno::ETIMEDOUT, Errno::ECONNRESET to indicate that the daemon cannot be
+	# connected to. It must return non-nil if the daemon can be connected to.
 	# Upon successful connection, the return value of the block will
 	# be returned by #connect.
 	#
@@ -158,21 +185,21 @@ class DaemonController
 	#   to the daemon even after starting it.
 	def connect
 		connection = nil
-		shared_lock do
+		@lock_file.shared_lock do
 			begin
 				connection = yield
-			rescue Errno::ECONNREFUSED, Errno::ENETUNREACH, Errno::ETIMEDOUT
+			rescue *ALLOWED_CONNECT_EXCEPTIONS
 				connection = nil
 			end
 		end
 		if connection.nil?
-			exclusive_lock do
+			@lock_file.exclusive_lock do
 				if !daemon_is_running?
 					start_without_locking
 				end
 				begin
 					connection = yield
-				rescue Errno::ECONNREFUSED, Errno::ENETUNREACH, Errno::ETIMEDOUT
+				rescue *ALLOWED_CONNECT_EXCEPTIONS
 					connection = nil
 				end
 				if connection.nil?
@@ -197,7 +224,7 @@ class DaemonController
 	# - StopError - the stop command failed.
 	# - StopTimeout - the daemon didn't stop in time.
 	def stop
-		exclusive_lock do
+		@lock_file.exclusive_lock do
 			begin
 				Timeout.timeout(@stop_timeout) do
 					kill_daemon
@@ -211,14 +238,16 @@ class DaemonController
 		end
 	end
 	
-	# Returns the daemon's PID, as reported by its PID file.
+	# Returns the daemon's PID, as reported by its PID file. Returns the PID
+	# as an integer, or nil there is no valid PID in the PID file.
+	#
 	# This method doesn't check whether the daemon's actually running.
 	# Use #running? if you want to check whether it's actually running.
 	#
 	# Raises SystemCallError or IOError if something went wrong during
 	# reading of the PID file.
 	def pid
-		shared_lock do
+		@lock_file.shared_lock do
 			return read_pid_file
 		end
 	end
@@ -230,32 +259,12 @@ class DaemonController
 	# Raises SystemCallError or IOError if something went wrong during
 	# reading of the PID file.
 	def running?
-		shared_lock do
+		@lock_file.shared_lock do
 			return daemon_is_running?
 		end
 	end
 
 private
-	def exclusive_lock
-		File.open(@lock_file, 'w') do |f|
-			if Fcntl.const_defined? :F_SETFD
-				f.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC)
-			end
-			f.flock(File::LOCK_EX)
-			yield
-		end
-	end
-	
-	def shared_lock
-		File.open(@lock_file, 'w') do |f|
-			if Fcntl.const_defined? :F_SETFD
-				f.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC)
-			end
-			f.flock(File::LOCK_SH)
-			yield
-		end
-	end
-	
 	def start_without_locking
 		if daemon_is_running?
 			raise AlreadyStarted, "Daemon '#{@identifier}' is already started"
@@ -264,6 +273,7 @@ private
 		delete_pid_file
 		begin
 			started = false
+			before_start
 			Timeout.timeout(@start_timeout) do
 				done = false
 				spawn_daemon
@@ -305,14 +315,22 @@ private
 			result = :timeout
 		end
 		if !result
-			raise StartError, differences_in_log_file
+			raise(StartError, differences_in_log_file ||
+				"Daemon '#{@identifier}' failed to start.")
 		elsif result == :timeout
-			raise StartTimeout, differences_in_log_file
+			raise(StartTimeout, differences_in_log_file ||
+				"Daemon '#{@identifier}' failed to start in time.")
 		else
 			return true
 		end
 	end
 	
+	def before_start
+		if @before_start
+			@before_start.call
+		end
+	end
+	
 	def spawn_daemon
 		run_command(@start_command)
 	end
@@ -330,7 +348,10 @@ private
 	end
 	
 	def kill_daemon_with_signal
-		Process.kill('SIGTERM', read_pid_file)
+		pid = read_pid_file
+		if pid
+			Process.kill('SIGTERM', pid)
+		end
 	rescue Errno::ESRCH, Errno::ENOENT
 	end
 	
@@ -354,7 +375,12 @@ private
 	end
 	
 	def read_pid_file
-		return File.read(@pid_file).strip.to_i
+		pid = File.read(@pid_file).strip
+		if pid =~ /\A\d+\Z/
+			return pid.to_i
+		else
+			return nil
+		end
 	end
 	
 	def delete_pid_file
@@ -437,7 +463,12 @@ private
 		if @original_log_file_stat
 			File.open(@log_file, 'r') do |f|
 				f.seek(@original_log_file_stat.size, IO::SEEK_SET)
-				return f.read.strip
+				diff = f.read.strip
+				if diff.empty?
+					return nil
+				else
+					return diff
+				end
 			end
 		else
 			return nil
@@ -447,7 +478,7 @@ private
 	end
 	
 	def determine_lock_file(identifier, pid_file)
-		return File.expand_path(pid_file + ".lock")
+		return LockFile.new(File.expand_path(pid_file + ".lock"))
 	end
 	
 	def self.fork_supported?
@@ -498,7 +529,15 @@ private
 	
 	def run_ping_command
 		if @ping_command.respond_to?(:call)
-			return @ping_command.call
+			begin
+				value = @ping_command.call
+				if value.respond_to?(:close)
+					value.close rescue nil
+				end
+				return value
+			rescue *ALLOWED_CONNECT_EXCEPTIONS
+				return false
+			end
 		else
 			return system(@ping_command)
 		end

data/spec/daemon_controller_spec.rb

@@ -156,6 +156,7 @@ describe DaemonController, "#start" do
 			max_start_timeout = 6
 		else
 			start_timeout = 0.15
+			min_start_timeout = 0.15
 			max_start_timeout = 0.30
 		end
 		new_controller(:start_command => 'sleep 2', :start_timeout => start_timeout)
@@ -298,3 +299,46 @@ describe DaemonController, "#connect" do
 	end
 end
 
+describe DaemonController do
+	include TestHelpers
+	
+	specify "if the ping command is a block that raises Errno::ECONNREFUSED, then that's " <<
+	        "an indication that the daemon cannot be connected to" do
+		new_controller(:ping_command => lambda do
+			raise Errno::ECONNREFUSED, "dummy"
+		end)
+		@controller.send(:run_ping_command).should be_false
+	end
+	
+	specify "if the ping command is a block that returns an object that responds to #close, " <<
+	        "then the close method will be called on that object" do
+		server = TCPServer.new('localhost', 8278)
+		begin
+			socket = nil
+			new_controller(:ping_command => lambda do
+				socket = TCPSocket.new('localhost', 8278)
+			end)
+			@controller.send(:run_ping_command)
+			socket.should be_closed
+		ensure
+			server.close
+		end
+	end
+	
+	specify "if the ping command is a block that returns an object that responds to #close, " <<
+	        "and #close raises an exception, then that exception is ignored" do
+		server = TCPServer.new('localhost', 8278)
+		begin
+			o = Object.new
+			o.should_receive(:close).and_return do
+				raise StandardError, "foo"
+			end
+			new_controller(:ping_command => lambda do
+				o
+			end)
+			lambda { @controller.send(:run_ping_command) }.should_not raise_error(StandardError)
+		ensure
+			server.close
+		end
+	end
+end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification 
 name: FooBarWidget-daemon_controller
 version: !ruby/object:Gem::Version 
-  version: 0.1.0
+  version: 0.2.0
 platform: ruby
 authors: 
 - Hongli Lai
@@ -13,7 +13,7 @@ date: 2008-08-21 00:00:00 -07:00
 default_executable: 
 dependencies: []
 
-description: A library for implementing daemon management capabilities.
+description: A library for robust daemon management.
 email: hongli@phusion.nl
 executables: []
 
@@ -22,13 +22,14 @@ extensions: []
 extra_rdoc_files: []
 
 files: 
-- README.rdoc
+- README.markdown
 - LICENSE.txt
 - daemon_controller.gemspec
 - lib/daemon_controller.rb
+- lib/daemon_controller/lock_file.rb
 - spec/daemon_controller_spec.rb
 - spec/echo_server.rb
-has_rdoc: false
+has_rdoc: true
 homepage: http://github.com/FooBarWidget/daemon_controller/tree/master
 post_install_message: 
 rdoc_options: []

data/README.rdoc

@@ -1,41 +0,0 @@
-= Introduction
-
-daemon_controller is a library for implementing daemon management capabilities.
-
-Suppose that you have a Ruby on Rails application which uses the Sphinx search
-server [1] for full-text searching capbilities. In order to search the index,
-the search daemon (searchd) must be running. Furthermore, you're using the Riddle
-library [2] for interfacing with the search daemon.
-
-You can write this in your application:
-
-  require 'daemon_controller'
-  require 'riddle'
-  
-  controller = DaemonController.new(
-     :identifier    => 'Sphinx search daemon',
-     :start_command => 'searchd -c config/sphinx.conf',
-     :ping_command  => proc { Riddle::Client.new('localhost', 1234) },
-     :pid_file      => 'tmp/pids/sphinx.pid',
-     :log_file      => 'log/sphinx.log'
-  )
-  client = controller.connect do
-     Riddle::Client.new('localhost', 1234)
-  end
-  client.query("some search query...")
-
-controller.connect will start the Sphinx search daemon if it isn't already
-started. Then, it will connect to the Sphinx search daemon by running the
-given block.
-
-Basically you just tell the library how to start the daemon, how to check
-whether it's responding to connections, and which PID file and log file it
-uses. daemon_controller will automatically take care of things like:
-
- * concurrency control, e.g. to ensure that no two processes will try to start
-   the Sphinx search daemon at the same time.
- * error handling: if 'searchd' failed to start, then its error message will
-   be propagated into the exception that will be thrown. This makes it much
-   easier to handle daemon startup errors in your application. This can also
-   allow the system administrator to see the error message directly in your
-   application, instead of having to consult the daemon's log file.