zookeeper 0.9.3-java

data/.gitignore

@@ -0,0 +1,10 @@
+*.gem
+*.o
+*.so
+*~
+*.log
+Makefile
+pkg
+zookeeper.gemspec
+Gemfile.*
+tmp/

data/CHANGELOG

@@ -0,0 +1,119 @@
+v0.9.3 Event thread shutdown fix, Windows compatibility fix
+
+  * Use a 'shutdown thread' to coordinate cleanup if close is called from the
+    event thread (prevents deadlock)
+
+  * Default Logger now uses $stderr instead of opening /dev/null [#16]
+
+  * Gemfile/gemspec/Rakefile refactoring.
+
+v0.9.2 More efficient and simpler wrappers for GIL release
+
+  * After a code review by Andrew Wason (rectalogic), use a much simpler
+    technique for creating the arg structs and passing them to the
+    zkrb_gvl_* functions. No malloc(), no free(), no problem.
+
+v0.9.1 see v0.8.4 notes, same patch
+
+v0.9.0 RELEASE THE KRAK..er, GIL!!
+
+  * In >= 1.9.2 the ruby interpreter allows you to release the GIL when
+    calling into native code, sounds like a good idea.
+    
+    This release makes use of that code by parsing the zookeeper.h header file
+    and extracting the method signatures of all relevant zoo_* functions, then
+    generating boilerplate that allows us to call those functions via the
+    rb_thread_blocking_region function.
+
+    1.8.7 compatibility is maintained by stubbing out that functionality if built
+    under 1.8.7. 
+
+  * 1.8.7 is deprecated! I will continue to support 1.8.7 for the near future
+    but sometime soon, you're gonna have to upgrade.
+
+v0.8.4 fix NameError, require 'forwardable'
+  
+  * Really not sure why this didn't come up in tests
+
+  * issue here https://github.com/slyphon/zk/issues/22
+
+v0.8.3 fix NonLocalJump exception in event delivery thread shutdown code
+
+  * hit a corner case where we're waiting for the zkc handle setup
+    and the user decides to shutdown, but before we've had a chance
+    to enter the delivery loop.
+
+  * Cleaned up some nasty code in ZookeeperConstants
+
+  * removed ZookeeperConstants#print_events and ZookeeperConstants#print_states
+
+  * changed EVENT_TYPE_NAMES and EVENT_STATE_NAMES in ZookeeperConstants
+    to use string values instead of symbols
+
+v0.8.2 fix close after a fork()
+
+  * The dispatch thread will be dead in this situation, so we need to 
+    check to see if it's already dead before waiting on it to exit.
+
+v0.8.1 Java client fix, silence warnings
+
+v0.8.0 Refactor C implementaion, EventMachine client
+
+  * separated CZookeeper and ZookeeperBase implementation
+  
+    This solves issues with reopen not working properly, makes for a much
+    cleaner event delivery implementation. ZookeeperBase controls the lifecycle
+    of the event dispatch thread now, rather than it being tied to CZookeeper.
+  
+  * added support for the 'sync' API call
+  
+  * Refactored zookeeper_c.c and zookeeper_lib.c
+  
+    More error checking in zookeeper_lib.c and restructure some things to make
+    logic easier to follow
+  
+    Fix bug in method_get_next_event that made the shutdown case so complicated
+  
+  * Massively simplified EMClient implementation
+
+    Rather than trying to hook the IO used by zookeeper_lib to notify zookeeper_c
+    about event availabiltiy directly into EventMachine, use the same event delivery
+    thread, but wrap the dispatch call in EM.schedule.
+
+  * Improve implementation of spin-lock-esque code that waits for the connection to be
+    established before returning.
+
+    This cut the test runtime down from 1m 20s to 2s.
+
+  * Java client refactoring, similar correctness changes
+
+  * Change ZookeeperException base class to StandardError instead of Exception
+
+
+v0.4.5 Upgrade to ZooKeeper 3.3.3
+
+v0.4.4 Fix race condition on close, possible data corruption on async get.
+
+v0.4.3 Fix a handful of memory-related bugs, fix SIGSEGV on master change, reduce latency of event handling, fix compilation on OSX.
+
+v0.4.2 Add options to Zookeeper#initialize, silence most Zookeeper logs.
+
+v0.4.1 Upgrade to ZooKeeper 3.3.2
+
+v0.4.0. More attr-readers (StarvingMarvin) and 1.9 compatibility (tsuraan)
+
+v0.3.2. Handle close, closed connections and expired sessions a little more gracefully.
+
+v0.3.1. ACL bugfix.
+
+v0.3.0. Wickman's rewrite, breaks dependencies from myelin/emaland port.
+
+v0.2.2. Fix compatibility with stock Leopard fat-binary Ruby.
+
+v0.2.1. No more camelcase classname.
+
+v0.2. Bundle C dependencies, like memcached.gem.
+
+v0.1. First release.
+
+# vim:ft=text:ts=2:sw=2:et

data/Gemfile

@@ -0,0 +1,17 @@
+source :rubygems
+
+gemspec
+
+group :test do
+  gem "rspec", "~> 2.8.0"
+  gem 'flexmock', '~> 0.8.11'
+  gem 'eventmachine', '1.0.0.beta.4'
+  gem 'evented-spec', '~> 0.9.0'
+end
+
+group :development do
+  gem 'rake', '~> 0.9.0'
+  gem 'pry'
+end
+
+# vim:ft=ruby

data/LICENSE

@@ -0,0 +1,23 @@
+(The MIT License)
+
+Copyright (C) 2008 Phillip Pearson
+Copyright (C) 2010 Twitter, Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+'Software'), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/Manifest

@@ -0,0 +1,29 @@
+CHANGELOG
+LICENSE
+Manifest
+README
+Rakefile
+examples/cloud_config.rb
+ext/extconf.rb
+ext/zkc-3.3.4.tar.gz
+ext/zookeeper_base.rb
+ext/zookeeper_c.c
+ext/zookeeper_lib.c
+ext/zookeeper_lib.h
+java/zookeeper_base.rb
+lib/zookeeper.rb
+lib/zookeeper/acls.rb
+lib/zookeeper/callbacks.rb
+lib/zookeeper/common.rb
+lib/zookeeper/constants.rb
+lib/zookeeper/exceptions.rb
+lib/zookeeper/stat.rb
+spec/log4j.properties
+spec/spec_helper.rb
+spec/zookeeper_spec.rb
+test/test_basic.rb
+test/test_callback1.rb
+test/test_close.rb
+test/test_esoteric.rb
+test/test_watcher1.rb
+test/test_watcher2.rb

data/README.markdown

@@ -0,0 +1,59 @@
+# zookeeper #
+
+An interface to the Zookeeper cluster coordination server.
+
+For a higher-level interface with a more convenient API and features such as locks, have a look at [ZK](https://github.com/slyphon/zk) (also available is [ZK-EventMachine](https://github.com/slyphon/zk-eventmachine) for those who prefer async).
+
+## Big Plans for 1.0 ##
+
+The 1.0 release will feature a reorganization of the heirarchy. There will be a single top-level `Zookeeper` namespace (as opposed to the current layout, with 5-6 different top-level constants), and for the next several releases, there will be a backwards compatible require for users that still need to use the old names.
+
+## License
+
+Copyright 2008 Phillip Pearson, and 2010 Twitter, Inc. 
+Licensed under the MIT License.  See the included LICENSE file.  
+
+Portions copyright 2008-2010 the Apache Software Foundation, licensed under the
+Apache 2 license, and used with permission.
+
+Portions contributed to the open source community by HPDC, L.P.
+
+## Install
+
+sudo gem install zookeeper
+
+## Usage
+
+Connect to a server:
+
+	require 'rubygems'
+	require 'zookeeper'
+	z = Zookeeper.new("localhost:2181")
+	z.get_children(:path => "/")
+
+## Idioms
+
+The following methods are initially supported:
+* get
+* set
+* get\_children
+* stat
+* create
+* delete
+* get\_acl
+* set\_acl
+
+All support async callbacks.  get, get\_children and stat support both watchers and callbacks.
+
+Calls take a dictionary of parameters.  With the exception of set\_acl, the only required parameter is :path.  Each call returns a dictionary with at minimum two keys :req\_id and :rc.
+
+### A Bit about this repository ###
+
+Twitter's open source office was kind enough to transfer this repository to facilitate development and administration of this repository. The `zookeeper` gem's last three releases were recorded in branches `v0.4.2`, `v0.4.3` and `v0.4.4`. Releases of the `slyphon-zookeeper` gem were cut off of the fork, and unfortunately (due to an oversight on my part) were tagged with unrelated versions. Those were tagged with names `release/0.9.2`. 
+
+The plan is to keep the `slyphon-zookeeper` tags, and to tag the `zookeeper` releases `twitter/release/0.4.x`.
+
+Further work will be carried out on this repository. The `0.9.3` release of the zookeeper gem will be released under the 'zookeeper' name, and will bring the two divergent (conceptual) branches of development together.
+
+
+

data/Rakefile

@@ -0,0 +1,139 @@
+# def gemset_name
+#   ENV.fetch('GEM_HOME').split('@').last
+# end
+
+GEM_FILES = FileList['slyphon-zookeeper-*.gem']
+
+namespace :mb do
+  namespace :gems do
+    task :build do
+      sh "rvm 1.8.7 do gem build zookeeper.gemspec"
+      ENV['JAVA_GEM'] = '1'
+      sh "rvm 1.8.7 do gem build zookeeper.gemspec"
+    end
+
+    task :push do
+      GEM_FILES.each do |gem|
+        sh "gem push #{gem}"
+      end
+    end
+
+    task :clean do
+      rm_rf GEM_FILES
+    end
+
+    task :all => [:build, :push, :clean]
+  end
+end
+
+gemset_name = 'zookeeper'
+
+# this nonsense w/ tmp and the Gemfile is a bundler optimization
+
+%w[1.8.7 1.9.2 jruby rbx 1.9.3].each do |ns_name|
+  rvm_ruby = (ns_name == 'rbx') ? "rbx-2.0.testing" : ns_name
+
+  ruby_with_gemset = "#{rvm_ruby}@#{gemset_name}"
+
+  create_gemset_name  = "mb:#{ns_name}:create_gemset"
+  clobber_task_name   = "mb:#{ns_name}:clobber"
+  clean_task_name     = "mb:#{ns_name}:clean"
+  build_task_name     = "mb:#{ns_name}:build"
+  bundle_task_name    = "mb:#{ns_name}:bundle_install"
+  rspec_task_name     = "mb:#{ns_name}:run_rspec"
+
+  phony_gemfile_link_name = "Gemfile.#{ns_name}"
+  phony_gemfile_lock_name = "#{phony_gemfile_link_name}.lock"
+
+  file phony_gemfile_link_name do
+    # apparently, rake doesn't deal with symlinks intelligently :P
+    ln_s('Gemfile', phony_gemfile_link_name) unless File.symlink?(phony_gemfile_link_name)
+  end
+
+  task :clean do
+    rm_rf [phony_gemfile_lock_name, phony_gemfile_lock_name]
+  end
+
+  task create_gemset_name do
+    sh "rvm #{rvm_ruby} do rvm gemset create #{gemset_name}"
+  end
+
+  task clobber_task_name do
+    unless rvm_ruby == 'jruby'
+      cd 'ext' do
+        sh "rake clobber"
+      end
+    end
+  end
+
+  task clean_task_name do
+    unless rvm_ruby == 'jruby'
+      cd 'ext' do
+        sh "rake clean"
+      end
+    end
+  end
+
+  task build_task_name => [create_gemset_name, clean_task_name] do
+    unless rvm_ruby == 'jruby'
+      cd 'ext' do
+        sh "rvm #{ruby_with_gemset} do rake build"
+      end
+    end
+  end
+
+  task bundle_task_name => [phony_gemfile_link_name, build_task_name] do
+    sh "rvm #{ruby_with_gemset} do bundle install --gemfile #{phony_gemfile_link_name}"
+  end
+
+  task rspec_task_name => bundle_task_name do
+    sh "rvm #{ruby_with_gemset} do env BUNDLE_GEMFILE=#{phony_gemfile_link_name} bundle exec rspec spec --fail-fast"
+  end
+
+  task "mb:#{ns_name}" => rspec_task_name
+
+  task "mb:test_all_rubies" => rspec_task_name
+end
+
+task "mb:test_all" do
+  require 'benchmark'
+  t = Benchmark.realtime do
+    Rake::Task['mb:test_all_rubies'].invoke
+  end
+
+  $stderr.puts "Test run took: #{t} s"
+end
+
+task :default => 'mb:1.9.3'
+
+task :clobber do
+  rm_rf 'tmp'
+end
+
+# cargo culted from http://blog.flavorjon.es/2009/06/easily-valgrind-gdb-your-ruby-c.html
+VALGRIND_BASIC_OPTS = '--num-callers=50 --error-limit=no --partial-loads-ok=yes --undef-value-errors=no'
+
+task 'valgrind' do
+  cd 'ext' do
+    sh "rake clean build"
+  end
+
+  sh "valgrind #{VALGRIND_BASIC_OPTS} bundle exec rspec spec"
+end
+
+namespace :build do
+  task :clean do
+    cd 'ext' do
+      sh 'rake clean'
+    end
+
+    Rake::Task['build'].invoke
+  end
+end
+
+task :build do
+  cd 'ext' do
+    sh "rake"
+  end
+end
+

data/examples/cloud_config.rb

@@ -0,0 +1,125 @@
+require "rubygems"
+require "zookeeper"
+
+# A basic cloud-based YAML config library.  Ruby Zookeeper client example.
+#
+# If you pass in a file as 'zk:/foo.yml/blah' it will go out to zookeeper. 
+# Otherwise the file is assumed to be local.  The yml file will get parsed
+# and cached locally, and keys after the .yml get interpreted as keys into
+# the YAML.
+#
+# e.g. get(zk:/config/service.yml/key1/key2/key3..) => 
+#   zk.get(:path => /config/service.yml)
+#   yaml <= YAML.parse(data)
+#   yaml[key1][key2][key3]...
+#
+# If keys are unspecified, it returns the parsed YAML as one big object
+#
+# TODO if staleness is set to 0, read in YAML immediately before next
+# get(...)
+
+class CloudConfig
+  class NodeNotFound < StandardError; end
+  class BadPathError < StandardError; end
+
+  DEFAULT_SERVERS = "localhost:2181"
+
+  def initialize(zkservers = DEFAULT_SERVERS, staleness = 15)  # maximum allowed staleness in seconds
+    @staleness = staleness
+    @lock      = Mutex.new
+    @zkservers = DEFAULT_SERVERS
+    
+    # cache
+    @data      = {}
+    @zkcb      = Zookeeper::WatcherCallback.new { dirty_callback(@zkcb.context) }
+    @zk        = nil
+  end
+
+  def get(node)
+    filename, keys = extract_filename(node)
+
+    # read(filename) is potentially a zk call, so do not hold the lock during the read
+    if @lock.synchronize { !@data.has_key?(filename) }
+      d = YAML.load(read(filename))
+      @lock.synchronize { @data[filename] = d }
+    end
+    
+    # synchronized b/c we potentially have a background thread updating data nodes from zk
+    # if keys is empty, return the whole file, otherwise roll up the keys
+    @lock.synchronize {
+      keys.empty? ? @data[filename] : keys.inject(@data[filename]) { |hash, key| hash[key] }
+    }
+  end
+
+  # todo:
+  #   factor get-and-watch into a different subsystem (so you can have
+  #   polling stat() ops on local filesystem.)
+  def read(yaml)
+    # read yaml file and register watcher.  if watcher fires, set up
+    # background thread to do read and update data.
+    if yaml.match(/^zk:/)
+      @zk ||= init_zk
+      yaml = yaml['zk:'.length..-1] # strip off zk: from zk:/config/path.yml
+      resp = get_and_register(yaml)
+
+      if resp[:rc] != Zookeeper::ZOK
+        @zk.unregister_watcher(resp[:req_id])
+        raise NodeNotFound
+      end
+      
+      resp[:data]
+    else
+      raise NodeNotFound unless File.exists?(yaml)
+      File.read(yaml)
+    end
+  end
+
+  def extract_filename(node)
+    path_elements = node.split("/")
+    
+    yamlindex = path_elements.map{ |x| x.match("\.yml$") != nil }.index(true)
+    raise BadPathError unless yamlindex
+    
+    yamlname = path_elements[0..yamlindex].join '/'
+    yamlkeys = path_elements[(yamlindex+1)..-1]
+    
+    return yamlname, yamlkeys
+  end
+    
+ private
+  def init_zk
+    Zookeeper.new(@zkservers)
+  end
+  
+  def get_and_register(znode)
+    @zk.get(:path => znode, :watcher => @zkcb,
+                   :watcher_context => { :path => znode,
+                                         :wait => rand(@staleness) })
+  end
+  
+  def dirty_callback(context)
+    path = context[:path]
+    wait = context[:wait]
+
+    # Fire off a background update that waits a randomized period of time up
+    # to @staleness seconds.
+    Thread.new do
+      sleep wait
+      background_update(path)
+    end
+  end
+
+  def background_update(zkpath)
+    # do a synchronous get/register a new watcher
+    resp = get_and_register(zkpath)
+    if resp[:rc] != Zookeeper::ZOK
+      # puts "Unable to read #{zkpath} from Zookeeper!"  @logger.error
+      zk.unregister_watcher(resp[:req_id])
+    else
+      # puts "Updating data."
+      d = YAML.load(resp[:data])
+      @lock.synchronize { @data["zk:#{zkpath}"] = d }
+    end
+  end
+end
+