zk 1.5.3 → 1.6.0

data/Guardfile

@@ -13,6 +13,11 @@ end
 guard 'rspec', :version => 2 do
   watch(%r{^spec/.+_spec\.rb$})
 
+  # run all specs when the support files change
+  watch(%r{^spec/support/.+\.rb$}) { 'spec' }
+
+  watch('spec/shared/client_examples.rb') { 'spec/zk/client_spec.rb' }
+
   watch(%r%^spec/support/client_forker.rb$%) { 'spec/zk/00_forked_client_integration_spec.rb' }
 
   watch(%r{^lib/(.+)\.rb$}) do |m| 
@@ -23,10 +28,13 @@ guard 'rspec', :version => 2 do
     when 'zk/client/threaded'
       ["spec/zk/client_spec.rb", "spec/zk/zookeeper_spec.rb"]
 
+    when 'zk/locker'
+      'spec/zk/locker_spec.rb'
+
     when %r{^(?:zk/locker/locker_base|spec/shared/locker)}
       Dir["spec/zk/locker/*_spec.rb"]
 
-    when %r{^zk/client/(?:base|state_mixin)}
+    when %r{^zk/client/(?:base|state_mixin|unixisms)}
       Dir['spec/zk/{client,client/*,zookeeper}_spec.rb']
 
     when 'zk' # .rb

data/README.markdown

@@ -65,6 +65,38 @@ In addition to all of that, I would like to think that the public API the ZK::Cl
 [zk-eventmachine]: https://github.com/slyphon/zk-eventmachine
 
 ## NEWS ##
+### v1.6.0 ###
+
+* Locker cleanup code!
+
+When a session is lost, it's likely that the locker's node name was left behind. so for `zk.locker('foo')` if the session is interrupted, it's very likely that the `/_zklocking/foo` znode has been left behind. A method has been added to allow you to safely clean up these stale znodes:
+
+```ruby
+ZK.open('localhost:2181') do |zk|
+  ZK::Locker.cleanup(zk)
+end
+```
+
+Will go through your locker nodes one by one and try to lock and unlock them. If it succeeds, the lock is naturally cleaned up (as part of the normal teardown code), if it doesn't acquire the lock, then no harm, it knows that lock is still in use.
+
+* Added `create('/path', 'data', :or => :set)` which will create a node (and all parent paths) with the given data or set its contents if it already exists. It's intended as a convenience when you just want a node to exist with a particular value.
+
+* Added a bunch of shorter aliases on `ZK::Event`, so you can say `event.deleted?`, `event.changed?`, etc.
+
+### v1.5.3 ###
+
+* Fixed reconnect code. There was an occasional race/deadlock condition caused because the reopen call was done on the underlying connection's dispatch thread. Closing the dispatch thread is part of reopen, so this would cause a deadlock in real-world use. Moved the reconnect logic to a separate, single-purpose thread on ZK::Client::Threaded that watches for connection state changes. 
+
+* 'private' is not 'protected'. I've been writing ruby for several years now, and apparently I'd forgotten that 'protected' does not work like how it does in java. The visibility of these methods has been corrected, and all specs pass, so I don't expect issues...but please report if this change causes any bugs in user code.
+
+### v1.5.2 ###
+
+* Fix locker cleanup code to avoid a nasty race when a session is lost, see [issue #34](https://github.com/slyphon/zk/issues/34)
+
+* Fix potential deadlock in ForkHook code so the mutex is unlocked in the case of an exception
+
+* Do not hang forever when shutting down and the shutdown thread does not exit (wait 30 seconds).
+
 ### v1.5.1 ###
 
 * Added a `:retry_duration` option to the Threaded client constructor which will allows the user to specify for how long in the case of a connection loss, should an operation wait for the connection to be re-established before retrying the operation. This can be set at a global level and overridden on a per-call basis. The default is to not retry (which may change at a later date). Generally speaking, a timeout of > 30s is probably excessive, and care should be taken because during a connection loss, the server-side state may change without you being aware of it (i.e. events will not be delivered). 
@@ -118,20 +150,6 @@ zk.delete('/some/path', :ignore => :no_node)
 * MASSIVE fork/parent/child test around event delivery and much greater stability expected for linux (with the zookeeper-1.0.3 gem). Again, please see the documentation on the wiki about [proper fork procedure](http://github.com/slyphon/zk/wiki/Forking).
 
 
-### v1.3.1 ###
-
-* [fix a bug][bug 1.3.1] where a forked client would not have its 'outstanding watches' cleared, so some events would never be delivered
-
-[bug 1.3.1]: https://github.com/slyphon/zk/compare/release/1.3.0...9f68cee958fdaad8d32b6d042bf0a2c9ab5ec9b0
-
-### v1.3.0 ###
-
-Phusion Passenger and Unicorn users are encouraged to upgrade!
-
-* __fork()__: ZK should now work reliably after a fork() if you call `reopen()` ASAP in the child process (before continuing any ZK work). Additionally, your event-handler (blocks set up with `zk.register`) will still work in the child. You will have to make calls like `zk.stat(path, :watch => true)` to tell ZooKeeper to notify you of events (as the child will have a new session), but everything should work.
-
-* See the fork-handling documentation [on the wiki](http://github.com/slyphon/zk/wiki/Forking).
-
 
 ## Caveats
 

data/RELEASES.markdown

@@ -1,5 +1,28 @@
 This file notes feature differences and bugfixes contained between releases. 
 
+### v1.6.0 ###
+
+* Locker cleanup code!
+
+When a session is lost, it's likely that the locker's node name was left behind. so for `zk.locker('foo')` if the session is interrupted, it's very likely that the `/_zklocking/foo` znode has been left behind. A method has been added to allow you to safely clean up these stale znodes:
+
+```ruby
+ZK.open('localhost:2181') do |zk|
+  ZK::Locker.cleanup(zk)
+end
+```
+
+Will go through your locker nodes one by one and try to lock and unlock them. If it succeeds, the lock is naturally cleaned up (as part of the normal teardown code), if it doesn't acquire the lock, then no harm, it knows that lock is still in use.
+
+* Added `create('/path', 'data', :or => :set)` which will create a node (and all parent paths) with the given data or set its contents if it already exists. It's intended as a convenience when you just want a node to exist with a particular value.
+
+### v1.5.3 ###
+
+* Fixed reconnect code. There was an occasional race/deadlock condition caused because the reopen call was done on the underlying connection's dispatch thread. Closing the dispatch thread is part of reopen, so this would cause a deadlock in real-world use. Moved the reconnect logic to a separate, single-purpose thread on ZK::Client::Threaded that watches for connection state changes. 
+
+* 'private' is not 'protected'. I've been writing ruby for several years now, and apparently I'd forgotten that 'protected' does not work like how it does in java. The visibility of these methods has been corrected, and all specs pass, so I don't expect issues...but please report if this change causes any bugs in user code.
+
+
 ### v1.5.2 ###
 
 * Fix locker cleanup code to avoid a nasty race when a session is lost, see [issue #34](https://github.com/slyphon/zk/issues/34)

data/lib/zk/client/base.rb

@@ -128,10 +128,6 @@ module ZK
       #
       # @return [Symbol] state of connection after operation
       def reopen(timeout=nil)
-#         timeout ||= @session_timeout # XXX: @session_timeout ?
-#         cnx.reopen(timeout)
-#         @threadpool.start!    
-#         state
       end
 
       # close the underlying connection and clear all pending events.
@@ -202,6 +198,14 @@ module ZK
       #   @option opts [Object] :context (nil) an object passed to the `:callback`
       #     given as the `context` param
       #
+      #   @option opts [:create,nil] :or (nil) syntactic sugar to say 'if this
+      #     path already exists, then set its contents.' Note that this will
+      #     also create all intermediate paths as it delegates to
+      #     {ZK::Client::Unixisms#mkdir_p}.  Note that this option can only be
+      #     used to create or set persistent, non-sequential paths. If an
+      #     option is used to specify either, an ArgumentError will be raised.
+      #     (note: not available for zk-eventmachine)
+      #
       #   @option opts [:ephemeral_sequential, :persistent_sequential, :persistent, :ephemeral] :mode (nil)
       #     may be specified instead of :ephemeral and :sequence options. If `:mode` *and* either of
       #     the `:ephermeral` or `:sequential` options are given, the `:mode` option will win
@@ -229,6 +233,14 @@ module ZK
       #   @option opts [Object] :context (nil) an object passed to the `:callback`
       #     given as the `context` param
       #
+      #   @option opts [:create,nil] :or (nil) syntactic sugar to say 'if this
+      #     path already exists, then set its contents.' Note that this will
+      #     also create all intermediate paths as it delegates to
+      #     {ZK::Client::Unixisms#mkdir_p}.  Note that this option can only be
+      #     used to create or set persistent, non-sequential paths. If an
+      #     option is used to specify either, an ArgumentError will be raised.
+      #     (note: not available for zk-eventmachine)
+      #
       #   @option opts [:ephemeral_sequential, :persistent_sequential, :persistent, :ephemeral] :mode (nil)
       #     may be specified instead of :ephemeral and :sequence options. If `:mode` *and* either of
       #     the `:ephermeral` or `:sequential` options are given, the `:mode` option will win
@@ -287,7 +299,6 @@ module ZK
       #   zk.create("/path", '', :mode => :persistent_sequential)
       #   # => "/path0"
       #
-      #
       # @example create ephemeral and sequential node
       #
       #   zk.create("/path", '', :sequence => true, :ephemeral => true)
@@ -337,6 +348,14 @@ module ZK
       #   zk.create("/path", "foo", :callback => callback, :context => context)
       #
       def create(path, *args)
+        h = parse_create_args(path, *args)
+        rv = call_and_check_rc(:create, h)
+        h[:callback] ? rv : rv[:path]
+      end
+
+      # parses the arguments and returns a hash for passing to
+      # call_and_check_rc. this is so subclasses can override easily
+      def parse_create_args(path, *args)
         opts = args.extract_options!
 
         # be somewhat strict about how many arguments we accept.
@@ -355,30 +374,29 @@ module ZK
 
         data = args.first || ''
 
-        h = { :path => path, :data => data, :ephemeral => false, :sequence => false }.merge(opts)
+        rval = { :path => path, :data => data, :ephemeral => false, :sequence => false }.merge(opts)
 
-        if mode = h.delete(:mode)
+        if mode = rval.delete(:mode)
           mode = mode.to_sym
 
           case mode
           when :ephemeral_sequential
-            h[:ephemeral] = h[:sequence] = true
+            rval[:ephemeral] = rval[:sequence] = true
           when :persistent_sequential
-            h[:ephemeral] = false
-            h[:sequence] = true
+            rval[:ephemeral] = false
+            rval[:sequence] = true
           when :persistent
-            h[:ephemeral] = false
+            rval[:ephemeral] = false
           when :ephemeral
-            h[:ephemeral] = true
+            rval[:ephemeral] = true
           else
             raise ArgumentError, "Unknown mode: #{mode.inspect}"
           end
         end
 
-        rv = call_and_check_rc(:create, h)
-
-        h[:callback] ? rv : rv[:path]
+        rval
       end
+      private :parse_create_args
 
       # Return the data and stat of the node of the given path.  
       # 

data/lib/zk/client/state_mixin.rb

@@ -48,6 +48,14 @@ module ZK
         end
       end
 
+      # Register a block to be called when *any* connection event occurs
+      #
+      # @yield [event] yields the connection event to the block
+      # @yieldparam event [ZK::Event] the event that occurred
+      def on_state_change(&block)
+        watcher.register_state_handler(:all, &block)
+      end
+
       # Register a block to be called on connection, when the client has
       # connected. 
       # 

data/lib/zk/client/threaded.rb

@@ -347,7 +347,7 @@ module ZK
         on_tpool ? shutdown_thread : shutdown_thread.join(30)
       end
 
-      # {see Base#close}
+      # {see ZK::Client::Base#close}
       def close
         super
         subs, @fork_subs = @fork_subs, []
@@ -365,6 +365,33 @@ module ZK
         @threadpool.on_exception(&blk)
       end
 
+      def closed?
+        return true if @mutex.synchronize { @client_state == CLOSED }
+        super
+      end
+
+      # this is where the :on option is implemented for {Base#create}
+      def create(path, *args)
+        opts = args.extract_options!
+
+        or_opt = opts.delete(:or)
+        args << opts
+
+        if or_opt
+          hash = parse_create_args(path, *args)
+ 
+          raise ArgumentError, "valid options for :or are nil or :set, not #{or_opt.inspect}" unless or_opt == :set 
+          raise ArgumentError, "you cannot create an ephemeral node when using the :or option" if hash[:ephemeral]
+          raise ArgumentError, "you cannot create an sequence node when using the :or option"  if hash[:sequence]
+         
+          mkdir_p(path, :data => hash[:data])
+          path
+        else
+          # ok, none of our business, hand it up to mangement
+          super(path, *args)
+        end
+      end
+
       # @private
       def raw_event_handler(event)
         return unless event.session_event?
@@ -378,43 +405,47 @@ module ZK
         logger.error { "BUG: Exception caught in raw_event_handler: #{e.to_std_format}" } 
       end
 
-      def closed?
-        return true if @mutex.synchronize { @client_state == CLOSED }
-        super
-      end
-
-      # are we in running (not-paused) state?
-      # @private
-      def running?
-        @mutex.synchronize { @client_state == RUNNING }
-      end
-
-      # are we in paused state?
-      # @private
-      def paused?
-        @mutex.synchronize { @client_state == PAUSED }
-      end
-
-      # has shutdown time arrived?
-      # @private
-      def close_requested?
-        @mutex.synchronize { @client_state == CLOSE_REQ }
-      end
-
       # @private
       def wait_until_connected_or_dying(timeout)
         time_to_stop = Time.now + timeout
 
         @mutex.synchronize do
-          while (@last_cnx_state != Zookeeper::ZOO_CONNECTED_STATE) && (Time.now < time_to_stop) && (@client_state == RUNNING)
-            @cond.wait(timeout)
+          while true
+            now = Time.now
+            break if (@last_cnx_state == Zookeeper::ZOO_CONNECTED_STATE) || (now > time_to_stop) || (@client_state != RUNNING)
+            deadline = time_to_stop.to_f - now.to_f
+            @cond.wait(deadline)
           end
 
-          logger.debug { "@last_cnx_state: #{@last_cnx_state.inspect}, time_left? #{Time.now.to_f < time_to_stop.to_f}, @client_state: #{@client_state.inspect}" }
+          logger.debug { "#{__method__} @last_cnx_state: #{@last_cnx_state.inspect}, time_left? #{Time.now.to_f < time_to_stop.to_f}, @client_state: #{@client_state.inspect}" }
         end
       end
 
+      # @private
+      def client_state
+        @mutex.synchronize { @client_state }
+      end
+
       private
+        # are we in running (not-paused) state?
+        def running?
+          @client_state == RUNNING
+        end
+
+        # are we in paused state?
+        def paused?
+          @client_state == PAUSED
+        end
+
+        # has shutdown time arrived?
+        def close_requested?
+          @client_state == CLOSE_REQ
+        end
+
+        def dead_or_dying?
+          (@client_state == CLOSE_REQ) || (@client_state == CLOSED)
+        end
+
         # this is just here so we can see it in stack traces
         def reopen_after_session_expired
           reopen
@@ -433,11 +464,11 @@ module ZK
             @mutex.synchronize do
               # either we havne't seen a valid session update from this
               # connection yet, or we're doing fine, so just wait
-              @cond.wait_while { !seen_session_state_event? or (valid_session_state? and (@client_state == RUNNING)) }
+              @cond.wait_while { !seen_session_state_event? or (valid_session_state? and running?) }
 
               # we've entered into a non-running state, so we exit
               # note: need to restart this thread after a fork in parent
-              if @client_state != RUNNING
+              unless running?
                 logger.debug { "session failure watcher thread exiting, @client_state: #{@client_state}" }
                 return
               end
@@ -457,8 +488,6 @@ module ZK
               end
             end
           end
-        ensure
-          logger.debug { "reconnect thread exiting" }
         end
 
         def join_and_clear_reconnect_thread
@@ -488,7 +517,7 @@ module ZK
 
               wait_until_connected_or_dying(retry_duration)
 
-              if (@last_cnx_state != Zookeeper::ZOO_CONNECTED_STATE) || (Time.now > time_to_stop) || (@client_state != RUNNING)
+              if (@last_cnx_state != Zookeeper::ZOO_CONNECTED_STATE) || (Time.now > time_to_stop) || !running?
                 raise e
               else
                 retry
@@ -519,10 +548,6 @@ module ZK
           ::Zookeeper.new(*args)
         end
 
-        def dead_or_dying?
-          (@client_state == CLOSE_REQ) || (@client_state == CLOSED)
-        end
-
         def unlocked_connect(opts={})
           return if @cnx
           timeout = opts.fetch(:timeout, @connection_timeout)

data/lib/zk/client/unixisms.rb

@@ -8,6 +8,8 @@ module ZK
       # zero data.
       # 
       # @param [String] path An absolute znode path to create
+      #
+      # @option opts [String] :data ('') The data to place at path
       # 
       # @example
       #
@@ -17,12 +19,20 @@ module ZK
       #   zk.mkdir_p('/path/to/blah')
       #   # => "/path/to/blah"  
       #
-      def mkdir_p(path)
-        # TODO: write a non-recursive version of this. ruby doesn't have TCO, so
-        # this could get expensive w/ psychotically long paths
+      def mkdir_p(path, opts={})
+        data = ''
+
+        # if we haven't recursed, or we recursed and now we're back at the top
+        if !opts.has_key?(:orig_path) or (path == opts[:orig_path])
+          data = opts.fetch(:data, '')  # only put the data at the leaf node
+        end
 
-        create(path, '', :mode => :persistent)
+        create(path, data, :mode => :persistent)
       rescue NodeExists
+        if !opts.has_key?(:orig_path) or (path == opts[:orig_path])  # we're at the leaf node
+          set(path, data)
+        end
+
         return
       rescue NoNode
         if File.dirname(path) == '/'
@@ -30,7 +40,9 @@ module ZK
           raise NonExistentRootError, "could not create '/', are you chrooted into a non-existent path?", caller
         end
 
-        mkdir_p(File.dirname(path))
+        opts[:orig_path] ||= path
+
+        mkdir_p(File.dirname(path), opts)
         retry
       end
 

data/lib/zk/event.rb

@@ -102,21 +102,25 @@ module ZK
     def node_created?
       @type == ZOO_CREATED_EVENT
     end
+    alias created? node_created?
 
     # Has a node been deleted? 
     def node_deleted?
       @type == ZOO_DELETED_EVENT
     end
+    alias deleted? node_deleted?
 
     # Has a node changed?
     def node_changed?
       @type == ZOO_CHANGED_EVENT
     end
+    alias changed? node_changed?
 
     # Has a node's list of children changed?
     def node_child?
       @type == ZOO_CHILD_EVENT
     end
+    alias child? node_child?
 
     # Is this a session-related event?
     #
@@ -148,12 +152,14 @@ module ZK
       @type == ZOO_SESSION_EVENT
     end
     alias state_event? session_event?
+    alias session? session_event?
     
     # has this watcher been called because of a change to a zookeeper node?
     # `node_event?` and `session_event?` are mutually exclusive.
     def node_event?
       path and not path.empty?
     end
+    alias node? node_event?
 
     # according to [the programmer's guide](http://zookeeper.apache.org/doc/r3.3.4/zookeeperProgrammers.html#Java+Binding)
     #

data/lib/zk/event_handler.rb

@@ -14,6 +14,9 @@ module ZK
     # @private
     ALL_NODE_EVENTS_KEY = :all_node_events
 
+    # @private
+    ALL_STATE_EVENTS_KEY = :all_state_events
+
     # @private
     ZOOKEEPER_WATCH_TYPE_MAP = {
       Zookeeper::ZOO_CREATED_EVENT => :data,
@@ -100,7 +103,13 @@ module ZK
     # or when there's a temporary loss in connection and Zookeeper recommends
     # you go into 'safe mode'.
     #
-    # @param [String] state The state you want to register for.
+    # Note that these callbacks are *not* one-shot like the path callbacks,
+    # these will be called back with every relative state event, there is 
+    # no need to re-register
+    #
+    # @param [String,:all] state The state you want to register for or :all
+    #   to be called back with every state change
+    #
     # @param [Block] block the block to execute on state changes
     # @yield [event] yields your block with
     #
@@ -156,8 +165,8 @@ module ZK
       cb_keys = 
         if event.node_event?
           [event.path, ALL_NODE_EVENTS_KEY]
-        elsif event.state_event?
-          [state_key(event.state)]
+        elsif event.session_event?
+          [state_key(event.state), ALL_STATE_EVENTS_KEY]
         else
           raise ZKError, "don't know how to process event: #{event.inspect}"
         end
@@ -180,22 +189,21 @@ module ZK
       safe_call(cb_ary, event)
     end
 
-    private
-      # happens inside the lock, clears the restriction on setting new watches
-      # for a given path/event type combination
-      #
-      def clear_watch_restrictions(event)
-        return unless event.node_event?
-
-        if watch_type = ZOOKEEPER_WATCH_TYPE_MAP[event.type]
-          #logger.debug { "re-allowing #{watch_type.inspect} watches on path #{event.path.inspect}" }
-          
-          # we recieved a watch event for this path, now we allow code to set new watchers
-          @outstanding_watches[watch_type].delete(event.path)
-        end
+    # happens inside the lock, clears the restriction on setting new watches
+    # for a given path/event type combination
+    #
+    def clear_watch_restrictions(event)
+      return unless event.node_event?
+
+      if watch_type = ZOOKEEPER_WATCH_TYPE_MAP[event.type]
+        #logger.debug { "re-allowing #{watch_type.inspect} watches on path #{event.path.inspect}" }
+        
+        # we recieved a watch event for this path, now we allow code to set new watchers
+        @outstanding_watches[watch_type].delete(event.path)
       end
+    end
+    private :clear_watch_restrictions
 
-    public
     # used during shutdown to clear registered listeners
     # @private
     def clear! #:nodoc:
@@ -237,11 +245,6 @@ module ZK
       @callbacks.values.flatten.each(&:resume_after_fork_in_parent)
     end
 
-    # @private
-    def synchronize
-      @mutex.synchronize { yield }
-    end
-
     # @private
     def get_default_watcher_block
       @default_watcher_block ||= lambda do |hash|
@@ -303,6 +306,10 @@ module ZK
     end
 
     private
+      def synchronize
+        @mutex.synchronize { yield }
+      end
+
       def watcher_callback
         Zookeeper::Callbacks::WatcherCallback.create { |event| process(event) }
       end
@@ -310,12 +317,15 @@ module ZK
       def state_key(arg)
         int = 
           case arg
+          when :all
+            # XXX: this is a nasty side-exit
+            return ALL_STATE_EVENTS_KEY
           when String, Symbol
             Zookeeper::Constants.const_get(:"ZOO_#{arg.to_s.upcase}_STATE")
           when Integer
             arg
           else
-            raise NameError # ugh lame
+            raise NameError, "unrecognized state: #{arg.inspect}" # ugh lame
           end
 
         "state_#{int}"

data/lib/zk/locker.rb

@@ -99,24 +99,55 @@ module ZK
       # the default root path we will use when a value is not given to a
       # constructor
       attr_accessor :default_root_lock_node
-    end
 
-    # Create a {SharedLocker} instance
-    #
-    # @param client (see LockerBase#initialize)
-    # @param name (see LockerBase#initialize)
-    # @return [SharedLocker]
-    def self.shared_locker(client, name, *args)
-      SharedLocker.new(client, name, *args)
-    end
+      # Create a {SharedLocker} instance
+      #
+      # @param client (see LockerBase#initialize)
+      # @param name (see LockerBase#initialize)
+      # @return [SharedLocker]
+      def shared_locker(client, name, *args)
+        SharedLocker.new(client, name, *args)
+      end
+
+      # Create an {ExclusiveLocker} instance
+      #
+      # @param client (see LockerBase#initialize)
+      # @param name (see LockerBase#initialize)
+      # @return [ExclusiveLocker]
+      def exclusive_locker(client, name, *args)
+        ExclusiveLocker.new(client, name, *args)
+      end
 
-    # Create an {ExclusiveLocker} instance
-    #
-    # @param client (see LockerBase#initialize)
-    # @param name (see LockerBase#initialize)
-    # @return [ExclusiveLocker]
-    def self.exclusive_locker(client, name, *args)
-      ExclusiveLocker.new(client, name, *args)
+      # Clean up dead locker directories. There are situations (particularly
+      # session expiration) where a lock's directory will never be cleaned up.
+      #
+      # It is intened to be run periodically (perhaps from cron).
+      #
+      #
+      # This implementation goes through each lock directory and attempts to
+      # acquire an exclusive lock. If the lock is acquired then when it unlocks
+      # it will remove the locker directory. This is safe because the unlock 
+      # code is designed to deal with the inherent race conditions.
+      #
+      # @example
+      #   
+      #   ZK.open do |zk|
+      #     ZK::Locker.cleanup!(zk)
+      #   end
+      #
+      # @param client [ZK::Client::Threaded] the client connection to use
+      #
+      # @param root_lock_node [String] if given, use an alternate root lock node to base
+      #   each Locker's path on. You probably don't need to touch this. Uses
+      #   {Locker.default_root_lock_node} by default (if value is nil)
+      #
+      def cleanup(client, root_lock_node=default_root_lock_node)
+        client.children(root_lock_node).each do |name|
+          exclusive_locker(client, name, root_lock_node).tap do |locker|
+            locker.unlock if locker.lock
+          end
+        end
+      end
     end
     
     # @private

data/lib/zk/locker/exclusive_locker.rb

@@ -75,16 +75,23 @@ module ZK
             path = "#{root_lock_path}/#{next_lowest_node}"
             logger.debug { "#{self.class}##{__method__} path=#{path.inspect}" }
 
-            synchronize do
+            @mutex.synchronize do
+              logger.debug { "assigning the @node_deletion_watcher" }
               @node_deletion_watcher = NodeDeletionWatcher.new(zk, path)
+              logger.debug { "broadcasting" }
               @cond.broadcast
             end
 
+            logger.debug { "calling block_until_deleted" }
+            Thread.pass
+
             @node_deletion_watcher.block_until_deleted
           rescue WeAreTheLowestLockNumberException
+          ensure
+            logger.debug { "block_until_deleted returned" } 
           end
 
-          synchronize { @locked = true }
+          @mutex.synchronize { @locked = true }
         end
     end # ExclusiveLocker
   end # Locker

data/lib/zk/locker/locker_base.rb

@@ -129,11 +129,13 @@ module ZK
       #
       def unlock
         rval = false
-        synchronize do
+        @mutex.synchronize do
           if @locked
+            logger.debug { "unlocking" }
             rval = cleanup_lock_path!
             @locked = false
             @node_deletion_watcher = nil
+            @cond.broadcast
           end
         end
         rval
@@ -176,7 +178,7 @@ module ZK
       #
       # @private
       def waiting? 
-        synchronize do
+        @mutex.synchronize do
           !!(@node_deletion_watcher and @node_deletion_watcher.blocked?)
         end
       end
@@ -184,9 +186,19 @@ module ZK
       # blocks the caller until this lock is blocked
       # @private
       def wait_until_blocked(timeout=nil)
-        synchronize do
-          @cond.wait_until { @node_deletion_watcher }
+        time_to_stop = timeout ? (Time.now + timeout) : nil
+
+        @mutex.synchronize do
+          if @node_deletion_watcher
+            logger.debug { "@node_deletion_watcher already assigned, not waiting" }
+          else
+            logger.debug { "going to wait up to #{timeout} sec for a @node_deletion_watcher to be assigned" }
+
+            @cond.wait(timeout) 
+            raise "Timeout waiting for @node_deletion_watcher" unless @node_deletion_watcher
+          end
         end
+        logger.debug { "ok, @node_deletion_watcher: #{@node_deletion_watcher}, going to call wait_until_blocked" }
 
         @node_deletion_watcher.wait_until_blocked(timeout)
       end
@@ -220,7 +232,7 @@ module ZK
       #   end
       #
       def assert!
-        synchronize do
+        @mutex.synchronize do
           raise LockAssertionFailedError, "have not obtained the lock yet"            unless locked?
           raise LockAssertionFailedError, "not connected"                             unless zk.connected?
           raise LockAssertionFailedError, "lock_path was #{lock_path.inspect}"        unless lock_path
@@ -239,11 +251,6 @@ module ZK
           self.class.digit_from_lock_path(path)
         end
 
-        # possibly lighter weight check to see if the lock path has any children
-        # (using stat, rather than getting the list of children).
-        def any_lock_children?
-        end
-
         def lock_children(watch=false)
           zk.children(root_lock_path, :watch => watch)
         end
@@ -275,7 +282,7 @@ module ZK
         # [rule #34](https://github.com/slyphon/zk/issues/34)...er, *issue* #34.
         #
         def create_lock_path!(prefix='lock')
-          synchronize do
+          @mutex.synchronize do
             @lock_path = @zk.create("#{root_lock_path}/#{prefix}", :mode => :ephemeral_sequential)
             @parent_stat = @zk.stat(root_lock_path)
           end
@@ -294,7 +301,7 @@ module ZK
         # see [issue #34](https://github.com/slyphon/zk/issues/34)
         #
         def root_lock_path_same?
-          synchronize do
+          @mutex.synchronize do
             return false unless @parent_stat
 
             cur_stat = zk.stat(root_lock_path)  
@@ -311,7 +318,7 @@ module ZK
         def cleanup_lock_path!
           rval = false
 
-          synchronize do
+          @mutex.synchronize do
             if root_lock_path_same?
               logger.debug { "removing lock path #{@lock_path}" }
 

data/lib/zk/node_deletion_watcher.rb

@@ -24,7 +24,7 @@ module ZK
       @mutex  = Monitor.new # ffs, 1.8.7 compatibility w/ timeouts
       @cond   = @mutex.new_cond
 
-      @blocked  = :not_yet
+      @blocked  = NOT_YET
       @result   = nil
     end
 
@@ -55,6 +55,7 @@ module ZK
         start = Time.now
         time_to_stop = timeout ? (start + timeout) : nil
 
+        logger.debug { "#{__method__} @blocked: #{@blocked.inspect} about to wait" } 
         @cond.wait(timeout)
 
         if (time_to_stop and (Time.now > time_to_stop)) and (@blocked == NOT_YET)
@@ -100,27 +101,25 @@ module ZK
 
         logger.debug { "ok, going to block: #{path}" }
 
-        while true # this is probably unnecessary
-          @blocked = BLOCKED
-          @cond.broadcast                 # wake threads waiting for @blocked to change
-          @cond.wait_until { @result }    # wait until we get a result
-          @blocked = NOT_ANYMORE
+        @blocked = BLOCKED
+        @cond.broadcast                 # wake threads waiting for @blocked to change
+        @cond.wait_until { @result }    # wait until we get a result
+        @blocked = NOT_ANYMORE
 
-          case @result
-          when :deleted
-            logger.debug { "path #{path} was deleted" }
-            return true
-          when INTERRUPTED
-            raise ZK::Exceptions::WakeUpException
-          when ZOO_EXPIRED_SESSION_STATE
-            raise Zookeeper::Exceptions::SessionExpired
-          when ZOO_CONNECTING_STATE
-            raise Zookeeper::Exceptions::NotConnected
-          when ZOO_CLOSED_STATE
-            raise Zookeeper::Exceptions::ConnectionClosed
-          else
-            raise "Hit unexpected case in block_until_node_deleted, result was: #{@result.inspect}"
-          end
+        case @result
+        when :deleted
+          logger.debug { "path #{path} was deleted" }
+          return true
+        when INTERRUPTED
+          raise ZK::Exceptions::WakeUpException
+        when ZOO_EXPIRED_SESSION_STATE
+          raise Zookeeper::Exceptions::SessionExpired
+        when ZOO_CONNECTING_STATE
+          raise Zookeeper::Exceptions::NotConnected
+        when ZOO_CLOSED_STATE
+          raise Zookeeper::Exceptions::ConnectionClosed
+        else
+          raise "Hit unexpected case in block_until_node_deleted, result was: #{@result.inspect}"
         end
       end
     ensure

data/lib/zk/subscription.rb

@@ -18,6 +18,7 @@ module ZK
 
       def initialize(parent, block)
         raise ArgumentError, "block must repsond_to?(:call)" unless block.respond_to?(:call)
+        raise ArgumentError, "parent must respond_to?(:unregister)" unless parent.respond_to?(:unregister)
         @parent   = parent
         @callable = block
         @mutex    = Monitor.new

data/lib/zk/version.rb

@@ -1,3 +1,3 @@
 module ZK
-  VERSION = "1.5.3"
+  VERSION = "1.6.0"
 end

data/spec/shared/client_contexts.rb

@@ -9,7 +9,6 @@ shared_context 'threaded client connection' do
 
   before do
 #     logger.debug { "threaded client connection - begin before hook" }
-
     @connection_string = connection_host
     @base_path = '/zktests'
     @zk = ZK::Client::Threaded.new(*connection_args).tap { |z| wait_until { z.connected? } }
@@ -17,6 +16,9 @@ shared_context 'threaded client connection' do
     @zk.on_exception { |e| @threadpool_exception = e }
     @zk.rm_rf(@base_path)
 
+    @orig_default_root_lock_node = ZK::Locker.default_root_lock_node
+    ZK::Locker.default_root_lock_node = "#{@base_path}/_zklocking"
+
 #     logger.debug { "threaded client connection - end before hook" }
   end
 
@@ -30,6 +32,8 @@ shared_context 'threaded client connection' do
       z.rm_rf(@base_path)
     end
 
+    ZK::Locker.default_root_lock_node = @orig_default_root_lock_node
+
 #     logger.debug { "threaded client connection - end after hook" }
   end
 end

data/spec/shared/client_examples.rb

@@ -1,18 +1,35 @@
 shared_examples_for 'client' do
   describe :mkdir_p do
-    before(:each) do
-      @path_ary = %w[test mkdir_p path creation]
-      @bogus_path = File.join('/', *@path_ary)
-      @zk.rm_rf('/test')
+    before do
+      base = @base_path.sub(%r_^/_, '')
+
+      @path_ary = %W[#{base} test mkdir_p path creation]
+      @bogus_path = File.join('', *@path_ary)
     end
-    
+   
     it %[should create all intermediate paths for the path givem] do
-      @zk.rm_rf('/test')
       @zk.should_not be_exists(@bogus_path)
       @zk.should_not be_exists(File.dirname(@bogus_path))
       @zk.mkdir_p(@bogus_path)
       @zk.should be_exists(@bogus_path)
     end
+
+    it %[should place the data only at the leaf node] do
+      @zk.mkdir_p(@bogus_path, :data => 'foobar')
+      @zk.get(@bogus_path).first.should == 'foobar'
+
+      path = ''
+      @path_ary[0..-2].each do |el|
+        path = File.join(path, el)
+        @zk.get(path).first.should == ''
+      end
+    end
+
+    it %[should replace the data at the leaf node if it already exists] do
+      @zk.mkdir_p(@bogus_path, :data => 'blahfoo')
+      @zk.mkdir_p(@bogus_path, :data => 'foodink')
+      @zk.get(@bogus_path).first.should == 'foodink'
+    end
   end
 
   # nail down all possible cases
@@ -114,6 +131,45 @@ shared_examples_for 'client' do
         proc { @zk.create("#{@base_path}/foo/bar/baz", :ignore => :no_node).should be_nil }.should_not raise_error(ZK::Exceptions::NoNode)
       end
     end
+
+    describe %[:or option] do
+      let(:path) { "#{@base_path}/foo/bar" }
+
+      it %[should barf if anything but the the :set value is given] do
+        proc { @zk.create(path, :or => :GFY) }.should raise_error(ArgumentError)
+      end
+
+      def create_args(opts={})
+        proc do 
+          begin
+            @zk.create(path, opts.merge(:or => :set))
+          ensure
+            @zk.rm_rf(path)
+          end
+        end
+      end
+
+      it %[should barf if any node option besides 'persistent' is given] do
+        create_args(:persistent => true).should_not         raise_error
+        create_args(:sequential => true).should             raise_error(ArgumentError)
+        create_args(:mode => :ephemeral).should             raise_error(ArgumentError)
+        create_args(:mode => :ephemeral_sequential).should  raise_error(ArgumentError)
+        create_args(:mode => :sequential).should            raise_error(ArgumentError)
+      end
+
+      it %[should replace the data at the leaf node if it already exists] do
+        @zk.mkdir_p(path, :data => 'foodink')
+        @zk.create(path, 'blahfoo', :or => :set)
+        @zk.get(path).first.should == 'blahfoo'
+      end
+
+      it %[should create the intermediate paths] do
+        proc { @zk.create(path, 'foobar', :or => :set) }.should_not raise_error
+
+        @zk.stat(@base_path).should exist
+        @zk.stat("#{@base_path}/foo").should exist
+      end
+    end
   end
 
   describe :delete do
@@ -298,20 +354,54 @@ shared_examples_for 'client' do
 
   describe 'reconnection' do
     it %[should if it receives a client_invalid? event] do
-      logger.debug { "about to cause the reconnection" }
+      # note: we can't trust the events to be delivered in any particular order
+      # since they're happening on two different threads. if we see we're connected
+      # in the beginning, that there was a disconnection, then a reopen, that's
+      # probably fine. 
+      #
+      # we also check that the session_id was changed, which is the desired effect
+
+      orig_session_id = @zk.session_id
+      @zk.should be_connected
 
-      props = { :session_event? => true, :client_invalid? => true, :state_name => 'ZOO_EXPIRED_SESSION_STATE', :state => Zookeeper::ZOO_EXPIRED_SESSION_STATE }
+      props = { 
+        :session_event?   => true,
+        :node_event?      => false,
+        :client_invalid?  => true,
+        :state_name       => 'ZOO_EXPIRED_SESSION_STATE',
+        :state            => Zookeeper::ZOO_EXPIRED_SESSION_STATE,
+      }
 
       bogus_event = flexmock(:expired_session_event, props)
+      bogus_event.should_receive(:zk=).with(@zk).once
+
+      mutex = Monitor.new
+      cond  = mutex.new_cond
+      events = []
 
-      th = Thread.new do
-        @zk.raw_event_handler(bogus_event)
+      @sub = @zk.on_state_change do |event|
+        mutex.synchronize do
+          logger.debug { "event: #{event.inspect}" }
+          events << event.state
+          cond.broadcast
+        end
       end
 
-      @zk.wait_until_connected_or_dying(5)
-      @zk.should be_connected
+      mutex.synchronize do
+        events.should be_empty
+        @zk.event_handler.process(bogus_event)
+      end
+
+      logger.debug { "events: #{events.inspect}" } 
+
+      mutex.synchronize do
+        time_to_stop = Time.now + 2
+        cond.wait_while { (events.length < 2 ) && (Time.now < time_to_stop) }
+      end
 
-      th.join(2).should == th
+      events.should include(Zookeeper::ZOO_EXPIRED_SESSION_STATE) 
+      events.should include(Zookeeper::ZOO_CONNECTED_STATE)
+      @zk.session_id.should_not == orig_session_id
     end
   end # reconnection
 

data/spec/support/logging.rb

@@ -2,10 +2,13 @@ module ZK
   TEST_LOG_PATH = File.join(ZK::ZK_ROOT, 'test.log')
 
   def self.logging_gem_setup
-    layout = ::Logging.layouts.pattern(
+    layout_opts = { 
       :pattern => '%.1l, [%d #%p] (%9.9T) %25.25c{2}:  %m\n',
-      :date_pattern => '%H:%M:%S.%6N'
-    )
+    }
+
+    layout_opts[:date_pattern] = ZK.jruby? ? '%H:%M:%S.%3N' : '%H:%M:%S.%6N'
+
+    layout = ::Logging.layouts.pattern(layout_opts)
 
     appender = ENV['ZK_DEBUG'] ? ::Logging.appenders.stderr : ::Logging.appenders.file(ZK::TEST_LOG_PATH)
     appender.layout = layout

data/spec/zk/locker/exclusive_locker_spec.rb

@@ -20,8 +20,11 @@ shared_examples_for 'ZK::Locker::ExclusiveLocker' do
         locker2.lock(true)
       end
 
+      th.run
+
       logger.debug { "calling wait_until_blocked" }
-      locker2.wait_until_blocked(2)
+      proc { locker2.wait_until_blocked(2) }.should_not raise_error
+      logger.debug { "wait_until_blocked returned" }
       locker2.should be_waiting
 
       wait_until { zk.exists?(locker2.lock_path) }

data/spec/zk/locker_spec.rb

@@ -0,0 +1,23 @@
+require 'spec_helper'
+
+describe ZK::Locker do
+  include_context 'threaded client connection'
+
+  describe :cleanup do
+    it %[should remove dead lock directories] do
+      locker = @zk.locker('legit')
+      locker.lock
+      locker.assert!
+
+      bogus_lock_dir_names = %w[a b c d e f g]
+      bogus_lock_dir_names.each { |n| @zk.create("#{ZK::Locker.default_root_lock_node}/#{n}") }
+
+      ZK::Locker.cleanup(@zk)
+
+      lambda { locker.assert! }.should_not raise_error
+
+      bogus_lock_dir_names.each { |n| @zk.stat("#{ZK::Locker.default_root_lock_node}/#{n}").should_not exist }
+
+    end
+  end
+end

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification 
 name: zk
 version: !ruby/object:Gem::Version 
-  hash: 5
+  hash: 15
   prerelease: 
   segments: 
   - 1
-  - 5
-  - 3
-  version: 1.5.3
+  - 6
+  - 0
+  version: 1.6.0
 platform: ruby
 authors: 
 - Jonathan D. Simms
@@ -16,7 +16,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2012-05-24 00:00:00 Z
+date: 2012-05-29 00:00:00 Z
 dependencies: 
 - !ruby/object:Gem::Dependency 
   name: zookeeper
@@ -163,6 +163,7 @@ files:
 - spec/zk/locker/locker_basic_spec.rb
 - spec/zk/locker/shared_exclusive_integration_spec.rb
 - spec/zk/locker/shared_locker_spec.rb
+- spec/zk/locker_spec.rb
 - spec/zk/module_spec.rb
 - spec/zk/mongoid_spec.rb
 - spec/zk/node_deletion_watcher_spec.rb
@@ -239,6 +240,7 @@ test_files:
 - spec/zk/locker/locker_basic_spec.rb
 - spec/zk/locker/shared_exclusive_integration_spec.rb
 - spec/zk/locker/shared_locker_spec.rb
+- spec/zk/locker_spec.rb
 - spec/zk/module_spec.rb
 - spec/zk/mongoid_spec.rb
 - spec/zk/node_deletion_watcher_spec.rb