zk 1.0.0 → 1.1.0

data/.gitignore

@@ -8,3 +8,4 @@ html
 .rvmrc
 Gemfile.*
 wiki
+zookeeper

data/.travis.yml

@@ -3,15 +3,13 @@ notifications:
   email:
     - slyphon@gmail.com
 
+env:
+  - SPAWN_ZOOKEEPER='true'
+
 rvm:
-  - 1.8.7
-  - 1.9.2
   - 1.9.3
-  - jruby
-
-before_install:
-  - sudo apt-get update
-  - sudo apt-get install zookeeperd
+  - 1.9.2
+  - 1.8.7
 
-bundler_args: --without development
+bundler_args: --without development docs
 

data/Gemfile

@@ -2,11 +2,8 @@ source :rubygems
 
 # gem 'slyphon-zookeeper', :path => '~/zookeeper'
 
-group :development do
-  gem 'rake'
-end
-
-gem 'pry', :group => [:development, :test]
+gem 'rake', :group => [:development, :test]
+gem 'pry',  :group => [:development]
 
 group :docs do
   gem 'yard', '~> 0.7.5'
@@ -19,7 +16,8 @@ end
 group :test do
   gem 'rspec', '~> 2.8.0'
   gem 'flexmock', '~> 0.8.10'
-  gem 'ZenTest', '~> 4.5.0'
+#   gem 'zk-server', :path => '~/mbox/zk-server'
+  gem 'zk-server', '~> 0.9.1'
 end
 
 # Specify your gem's dependencies in zk.gemspec

data/README.markdown

@@ -1,5 +1,7 @@
 # ZK
 
+[![Build Status (master)](https://secure.travis-ci.org/slyphon/zk.png?branch=master)](http://travis-ci.org/slyphon/zk)
+
 ZK is a high-level interface to the Apache [ZooKeeper][] server. It is based on the [zookeeper gem][] which is a multi-Ruby low-level driver. Currently MRI 1.8.7, 1.9.2, 1.9.3, and JRuby are supported, rubinius 2.0.testing is supported-ish (it's expected to work, but upstream is unstable, so YMMV). 
 
 ZK is licensed under the [MIT][] license. 
@@ -16,7 +18,13 @@ Development is sponsored by [Snapfish][] and has been generously released to the
 [MIT]: http://www.gnu.org/licenses/license-list.html#Expat "MIT (Expat) License"
 [Snapfish]: http://www.snapfish.com/ "Snapfish"
 
-## New in 1.0 !! ##
+## New in 1.1 !! ##
+
+* NEW! Thread-per-Callback event delivery model! [Read all about it!](https://github.com/slyphon/zk/wiki/EventDeliveryModel). Provides a simple, sane way to increase the concurrency in your ZK-based app while maintaining the ordering guarantees ZooKeeper makes. Each callback can perform whatever work it needs to without blocking other callbacks from receiving events. Inspired by [Celluloid's](https://github.com/celluloid/celluloid) actor model.
+
+* Use the [zk-server](https://github.com/slyphon/zk-server) gem to run a standalone ZooKeeper server for tests (`rake SPAWN_ZOOKEEPER=1`). Makes live-fire testing of any project that uses ZK easy to run anywhere!
+
+## New in 1.0 ##
 
 * Threaded client (the default one) will now automatically reconnect (i.e. `reopen()`) if a `SESSION_EXPIRED` or `AUTH_FAILED` event is received. Thanks to @eric for pointing out the _nose-on-your-face obviousness_ and importance of this. If users want to handle these events themselves, and not automatically reopen, you can pass `:reconnect => false` to the constructor.
 
@@ -24,39 +32,51 @@ Development is sponsored by [Snapfish][] and has been generously released to the
 
 * add zk.register(:all) to recevie node updates for all nodes (i.e. not filtered on path)
 
-* add 'interest' feature to zk.register, now you can indicate what kind of events should be delivered to the given block (previously you had to do that filtering inside the block). The default behavior is still the same, if no 'interest' is given, then all event types for the given path will be delivered to that block. 
+* add 'interest' feature to zk.register, now you can indicate what kind of events should be delivered to the given block (previously you had to do that filtering inside the block). The default behavior is still the same, if no 'interest' is given, then all event types for the given path will be delivered to that block.
   
-    zk.register('/path', :created) do |event|
-      # event.node_created? will always be true
-    end
+```ruby
+zk.register('/path', :created) do |event|
+  # event.node_created? will always be true
+end
+
+# or multiple kinds of events
 
-    # or multiple kinds of events
+zk.register('/path', [:created, :changed]) do |event|
+  # (event.node_created? or event.node_changed?) will always be true
+end
 
-    zk.register('/path', [:created, :changed]) do |event|
-      # (event.node_created? or event.node_changed?) will always be true
-    end
+# this will, however, be changed in 1.1 to (backwards compatible, with a deprecation warning)
+
+zk.register('/path', :only => :created) do |event|
+end
+
+```
 
 * create now allows you to pass a path and options, instead of requiring the blank string
 
-    zk.create('/path', '', :sequential => true)
+```ruby
+zk.create('/path', '', :sequential => true)
 
-    # now also
+# now also
 
-    zk.create('/path', :sequential => true)
+zk.create('/path', :sequential => true)
+```
 
 * fix for shutdown: close! called from threadpool will do the right thing
 
 * Chroot users rejoice! By default, ZK.new will create a chrooted path for you. 
     
-    ZK.new('localhost:2181/path', :chroot => :create) # the default, create the path before returning connection
+```ruby
+ZK.new('localhost:2181/path', :chroot => :create) # the default, create the path before returning connection
 
-    ZK.new('localhost:2181/path', :chroot => :check)  # make sure the chroot exists, raise if not
+ZK.new('localhost:2181/path', :chroot => :check)  # make sure the chroot exists, raise if not
 
-    ZK.new('localhost:2181/path', :chroot => :do_nothing) # old default behavior
+ZK.new('localhost:2181/path', :chroot => :do_nothing) # old default behavior
 
-    # and, just for kicks
-    
-    ZK.new('localhost:2181', :chroot => '/path') # equivalent to 'localhost:2181/path', :chroot => :create
+# and, just for kicks
+
+ZK.new('localhost:2181', :chroot => '/path') # equivalent to 'localhost:2181/path', :chroot => :create
+```
 
 * Most of the event functionality used is now in a ZK::Event module. This is still mixed into the underlying slyphon-zookeeper class, but now all of the important and relevant methods are documented, and Event appears as a first-class citizen.
 
@@ -115,8 +135,6 @@ ZK strives to be a complete, correct, and convenient way of interacting with Zoo
 
 * ZK::Client supports asynchronous calls of all basic methods (get, set, delete, etc.) however these versions are kind of inconvenient to use. For a fully evented stack, try [zk-eventmachine][], which is designed to be compatible and convenient to use in event-driven code.
 
-* ZooKeeper "chroot" [connection syntax][chroot] should work for most cases. Right now we require that the root path exist before the chrooted client is used, but that may change [in the near future](https://github.com/slyphon/zk/issues/7).
-
 [twitter/zookeeper]: https://github.com/twitter/zookeeper
 [async-branch]: https://github.com/slyphon/zk/tree/dev%2Fasync-conveniences
 [chroot]: http://zookeeper.apache.org/doc/current/zookeeperProgrammers.html#ch_zkSessions
@@ -133,7 +151,7 @@ ZK strives to be a complete, correct, and convenient way of interacting with Zoo
 
 ## Dependencies
 
-* The [slyphon-zookeeper gem][szk-gem] ([repo][szk-repo]), which adds JRuby compatibility and a full suite of tests to the excellent [twitter/zookeeper][] project. 
+* The [slyphon-zookeeper gem][szk-gem] ([repo][szk-repo]).
 
 * For JRuby, the [slyphon-zookeeper\_jar gem][szk-jar-gem] ([repo][szk-jar-repo]), which just wraps the upstream zookeeper driver jar in a gem for easy installation
 

data/Rakefile

@@ -33,7 +33,7 @@ GEMSPEC_NAME = 'zk.gemspec'
   end
 
   task rspec_task_name => bundle_task_name do
-    sh "rvm #{ruby_with_gemset} do env BUNDLE_GEMFILE=#{phony_gemfile_link_name} bundle exec rspec spec --fail-fast"
+    sh "rvm #{ruby_with_gemset} do env JRUBY_OPTS='--1.9' BUNDLE_GEMFILE=#{phony_gemfile_link_name} bundle exec rspec spec --fail-fast"
   end
 
   task "mb:#{ns_name}" => rspec_task_name
@@ -69,7 +69,9 @@ namespace :spec do
     require 'bundler/setup'
     require 'rspec/core/rake_task'
 
-    RSpec::Core::RakeTask.new('spec:runner')
+    RSpec::Core::RakeTask.new('spec:runner') do |t|
+      t.rspec_opts = '-f d'
+    end
   end
 
   task :run => :define do

data/lib/zk.rb

@@ -16,6 +16,7 @@ require 'zk/extensions'
 require 'zk/event'
 require 'zk/stat'
 require 'zk/threadpool'
+require 'zk/threaded_callback'
 require 'zk/event_handler_subscription'
 require 'zk/event_handler'
 require 'zk/message_queue'
@@ -30,9 +31,6 @@ module ZK
   silence_warnings do
     # @private
     ZK_ROOT = File.expand_path('../..', __FILE__).freeze
-    
-    # @private
-    DEFAULT_SERVER = 'localhost:2181'.freeze
   end
 
   unless defined?(KILL_TOKEN) 
@@ -40,10 +38,32 @@ module ZK
     KILL_TOKEN = Object.new 
   end
 
-
   unless @logger
     @logger = Logger.new($stderr).tap { |n| n.level = Logger::ERROR }
   end
+ 
+  @default_host   = 'localhost' unless @default_host
+  @default_port   = 2181        unless @default_port
+  @default_chroot = ''          unless @default_chroot
+
+  class << self
+    # what host should ZK.new connect to when given no options
+    attr_accessor :default_host
+
+    # what port should ZK.new connect to when given no options
+    attr_accessor :default_port
+
+    # what chroot path should ZK.new connect to when given no options
+    attr_accessor :default_chroot
+  end
+
+  # @private
+  def self.default_connection_string
+    "#{default_host}:#{default_port}".tap do |str|
+      # XXX: this is seriously blech
+      str.replace(File.join(str, default_chroot)) if default_chroot != ''
+    end
+  end
 
   # The logger used by the ZK library. uses a Logger stderr with Logger::ERROR
   # level. The only thing that should ever be logged are exceptions that are
@@ -139,18 +159,22 @@ module ZK
   #     {ZK::Client::Threaded#initialize Threaded.new} directly. You probably
   #     also hate happiness and laughter.
   #
+  #   @option opts [:single,:per_callback] :thread (:single) see {ZK::Client::Threaded#initialize} 
+  #     for a discussion of what these options mean
+  #
   #   @raise [ChrootPathDoesNotExistError] if a chroot path is specified,
   #     `:chroot` is `:check`, and the path does not exist.
   #
   #   @raise [ArgumentError] if both a chrooted `connection_str` is given *and* a
   #     `String` value for the `:chroot` option is given
   #
+  #
   def self.new(*args, &block)
     opts = args.extract_options!
 
     chroot_opt = opts.fetch(:chroot, :create)
 
-    args = [DEFAULT_SERVER]  if args.empty?     # the ZK.new() case
+    args = [default_connection_string]  if args.empty?     # the ZK.new() case
 
     if args.first.kind_of?(String)
       if new_cnx_str = do_chroot_setup(args.first, chroot_opt)
@@ -174,6 +198,8 @@ module ZK
     yield cnx
   ensure
     cnx.close! if cnx
+    # XXX: need some way of waiting for the connection to reach closed? state
+    #      ensure there's no leakage
   end
 
   # creates a new ZK::Pool::Bounded with the default options.
@@ -186,6 +212,34 @@ module ZK
     File.join(*paths)
   end
 
+  # @private
+  def self.mri_19?
+    ruby_19? and not jruby? or rubinius?
+  end
+
+  def self.ruby_19?
+    (RUBY_VERSION =~ /\A1\.9\.[2-9]\Z/)
+  end
+
+  def self.ruby_187?
+    (RUBY_VERSION == '1.8.7')
+  end
+
+  # @private
+  def self.mri_187?
+    ruby_187? and not jruby? or rubinius?
+  end
+
+  # @private
+  def self.jruby?
+    defined?(::JRUBY_VERSION)
+  end
+
+  # @private
+  def self.rubinius?
+    defined?(::Rubinius)
+  end
+
   private
     # @return [String] a possibly modified connection string (with chroot info
     #   added)
@@ -226,6 +280,7 @@ module ZK
       open(host) do |zk|                # do path stuff with the virgin connection
         unless zk.exists?(chroot_path)  # someting must be done
           if chroot_opt == :create      # here, let me...
+            logger.debug { "creating chroot path #{chroot_path}" }
             zk.mkdir_p(chroot_path)     # ...get that for you
           else                          # careful with that axe
             raise Exceptions::ChrootPathDoesNotExistError.new(host, chroot_path)  # ...eugene

data/lib/zk/client/base.rb

@@ -770,12 +770,22 @@ module ZK
       #
       # @example only creation events
       #
-      #   sub = zk.register('/path/to/znode', :created) do |event|
+      #   sub = zk.register('/path/to/znode', :only => :created) do |event|
       #     # do something when the node is created
       #   end
       #
       # @example only changed or children events
       #
+      #   sub = zk.register('/path/to/znode', :only => [:changed, :child]) do |event|
+      #     if event.node_changed?
+      #       # do something on change
+      #     else
+      #       # we know it's a child event
+      #     end
+      #   end
+      #
+      # @example deprecated 1.0 style interests
+      #
       #   sub = zk.register('/path/to/znode', [:changed, :child]) do |event|
       #     if event.node_changed?
       #       # do something on change
@@ -787,12 +797,6 @@ module ZK
       # @param [String,:all] path the znode path you want to listen to, or the
       #   special value :all, that will cause the block to be delivered events
       #   for all znode paths
-      #
-      # @param [Array,Symbol,nil] interests a symbol or array-of-symbols indicating
-      #   which events you would like the block to be called for. Valid events
-      #   are :created, :deleted, :changed, and :child. If nil, the block will
-      #   receive all events
-      #
       # @param [Block] block the block to execute when a watch event happpens
       #
       # @yield [event] We will call your block with the watch event object (which
@@ -801,12 +805,30 @@ module ZK
       # @return [EventHandlerSubscription] the subscription object
       #   you can use to to unsubscribe from an event
       #
+      # @overload register(path, interests=nil, &block)
+      #   @since 1.0
+      #
+      #   @deprecated use the `:only => :created` form
+      #
+      #   @param [Array,Symbol,nil] interests a symbol or array-of-symbols indicating
+      #     which events you would like the block to be called for. Valid events
+      #     are :created, :deleted, :changed, and :child. If nil, the block will
+      #     receive all events
+      #
+      # @overload register(path, opts={}, &block)
+      #   @since 1.1
+      #
+      #   @option opts [Array,Symbol,nil] :only (nil) a symbol or array-of-symbols indicating
+      #     which events you would like the block to be called for. Valid events
+      #     are :created, :deleted, :changed, and :child. If nil, the block will
+      #     receive all events
+      #
       # @see ZooKeeper::WatcherEvent
       # @see ZK::EventHandlerSubscription
       # @see https://github.com/slyphon/zk/wiki/Events the wiki page on using events effectively
       #
-      def register(path, interests=nil, &block)
-        event_handler.register(path, interests, &block)
+      def register(path, opts={}, &block)
+        event_handler.register(path, opts, &block)
       end
 
       # returns true if the caller is calling from the event dispatch thread

data/lib/zk/client/threaded.rb

@@ -15,23 +15,11 @@ module ZK
     # unconfigurable, event dispatch thread. In 1.0 the number of event
     # delivery threads is configurable, but still defaults to 1. 
     #
-    # The configurability is intended to allow users to easily dispatch events to
-    # event handlers that will perform (application specific) work. Be aware,
-    # the default will give you the guarantee that only one event will be delivered
-    # at a time. The advantage to this is that you can be sure that no event will 
-    # be delivered "behind your back" while you're in an event handler. If you're
-    # comfortable with dealing with threads and concurrency, then feel free to 
-    # set the `:threadpool_size` option to the constructor to a value you feel is
-    # correct for your app. 
-    # 
     # If you use the threadpool/event callbacks to perform work, you may be
     # interested in registering an `on_exception` callback that will receive
     # all exceptions that occur on the threadpool that are not handled (i.e.
     # that bubble up to top of a block).
     #
-    # It is recommended that you not run any possibly long-running work on the
-    # event threadpool, as `close!` will attempt to shutdown the threadpool, and
-    # **WILL NOT WAIT FOREVER**. (TODO: more on this)
     #
     # @example Register on_connected callback.
     #   
@@ -67,8 +55,18 @@ module ZK
       # @note The documentation for 0.9.0 was incorrect in stating the number
       #   of threads used to deliver events. There was one, unconfigurable,
       #   event dispatch thread. In 1.0 the number of event delivery threads is
-      #   configurable, but still defaults to 1. (The Management apologizes for
-      #   any confusion this may have caused).
+      #   configurable, but still defaults to 1 and users are discouraged from
+      #   adjusting the value due to the complexity this introduces. In 1.1
+      #   there is a better option for achieving higher concurrency (see the
+      #   `:thread` option)
+      #
+      #   The Management apologizes for any confusion this may have caused. 
+      #
+      # @since __1.1__: Instead of adjusting the threadpool, users are _strongly_ encouraged
+      #   to use the `:thread => :per_callback` option to increase the
+      #   parallelism of event delivery safely and sanely. Please see 
+      #   [this wiki article](https://github.com/slyphon/zk/wiki/EventDeliveryModel) for more
+      #   information and a demonstration.
       #
       # @param [String] host (see ZK::Client::Base#initialize)
       #
@@ -77,13 +75,34 @@ module ZK
       #   case of an expired session, we will keep trying to reestablish the
       #   connection.
       #
-      # @option opts [Fixnum] :threadpool_size (1) the size of the threadpool that
-      #   should be used to deliver events. As of 1.0, this is the number of
-      #   event delivery threads and controls the amount of concurrency in your
-      #   app if you're doing work in the event callbacks.
+      # @option opts [:single,:per_callback] :thread (:single) choose your event
+      #   delivery model:
+      #
+      #   * `:single`: There is one thread, and only one callback is called at
+      #     a time. This is the default mode (for now), and will provide the most
+      #     safety for your app. All events will be delivered as received, to
+      #     callbacks in the order they were registered. This safety has the
+      #     tradeoff that if one of your callbacks performs some action that blocks
+      #     the delivery thread, you will not recieve other events until it returns.
+      #     You're also limiting the concurrency of your app. This should be fine
+      #     for most simple apps, and is a good choice to start with when
+      #     developing your application
+      #
+      #   * `:per_callback`: This option will use a new-style Actor model (inspired by 
+      #     [Celluloid](https://github.com/celluloid/celluloid)) that uses a
+      #     per-callback queue and thread to allow for greater concurrency in
+      #     your app, whille still maintaining some kind of sanity. By choosing
+      #     this option your callbacks will receive events in order, and will
+      #     receive only one at a time, but in parallel with other callbacks.
+      #     This model has the advantage you can have all of your callbacks
+      #     making progress in parallel, and if one of them happens to block,
+      #     it will not affect the others.
+      #    
+      #   * see {https://github.com/slyphon/zk/wiki/EventDeliveryModel the wiki} for a
+      #     discussion and demonstration of the effect of this setting.
       #
       # @option opts [Fixnum] :timeout how long we will wait for the connection
-      #   to be established. If timeout is nil, we will wait forever *use
+      #   to be established. If timeout is nil, we will wait forever: *use
       #   carefully*.
       #
       # @yield [self] calls the block with the new instance after the event
@@ -101,7 +120,7 @@ module ZK
         @threadpool = Threadpool.new(tp_size)
 
         @session_timeout = opts.fetch(:timeout, DEFAULT_TIMEOUT) # maybe move this into superclass?
-        @event_handler   = EventHandler.new(self)
+        @event_handler   = EventHandler.new(self, opts)
 
         @reconnect = opts.fetch(:reconnect, true)