zk 0.6.5 → 0.7.1
Sign up to get free protection for your applications and to get access to all the features.
- data/.dotfiles/rspec-logging +4 -0
- data/.gitignore +2 -0
- data/.yardopts +8 -0
- data/Gemfile +6 -1
- data/README.markdown +86 -0
- data/lib/z_k/client/base.rb +692 -0
- data/lib/z_k/client/conveniences.rb +134 -0
- data/lib/z_k/client/state_mixin.rb +94 -0
- data/lib/z_k/client/unixisms.rb +89 -0
- data/lib/z_k/client.rb +12 -891
- data/lib/z_k/election.rb +3 -0
- data/lib/z_k/event_handler.rb +7 -5
- data/lib/z_k/mongoid.rb +1 -1
- data/lib/z_k/pool.rb +70 -27
- data/lib/z_k/threadpool.rb +7 -2
- data/lib/z_k/version.rb +1 -1
- data/lib/z_k.rb +1 -2
- data/spec/spec_helper.rb +1 -0
- data/spec/support/logging_progress_bar_formatter.rb +14 -0
- data/spec/watch_spec.rb +26 -8
- data/spec/{client_spec.rb → z_k/client_spec.rb} +1 -1
- data/spec/{election_spec.rb → z_k/election_spec.rb} +2 -3
- data/spec/{locker_spec.rb → z_k/locker_spec.rb} +1 -1
- data/spec/{mongoid_spec.rb → z_k/mongoid_spec.rb} +1 -1
- data/spec/{client_pool_spec.rb → z_k/pool_spec.rb} +98 -126
- data/spec/{threadpool_spec.rb → z_k/threadpool_spec.rb} +6 -3
- data/zk.gemspec +1 -0
- metadata +37 -26
data/.gitignore
CHANGED
data/.yardopts
ADDED
data/Gemfile
CHANGED
@@ -5,6 +5,11 @@ source "http://localhost:50000"
|
|
5
5
|
# Specify your gem's dependencies in zk.gemspec
|
6
6
|
gemspec
|
7
7
|
|
8
|
-
|
8
|
+
gem 'ruby-debug', :platforms => [:mri_18, :jruby]
|
9
|
+
gem 'ruby-debug19', :platforms => :mri_19
|
10
|
+
|
11
|
+
# if `uname -s`.chomp == 'Linux'
|
12
|
+
# gem 'autotest-inotify', '~> 0.0.6'
|
13
|
+
# end
|
9
14
|
|
10
15
|
# vim:ft=ruby
|
data/README.markdown
ADDED
@@ -0,0 +1,86 @@
|
|
1
|
+
# ZK
|
2
|
+
|
3
|
+
ZK is a high-level interface to the Apache [ZooKeeper][] server. It is based on the [zookeeper gem][] which is a multi-Ruby low-level driver. Currently MRI 1.8.7 and JRuby are supported, and MRI 1.9.2 is very close to being ready. It is licensed under the [MIT][] license.
|
4
|
+
|
5
|
+
This library is heavily used in a production deployment and is actively developed and maintained.
|
6
|
+
|
7
|
+
Development is sponsored by [Snapfish][] and has been generously released to the Open Source community by HPDC, L.P.
|
8
|
+
|
9
|
+
[ZooKeeper]: http://zookeeper.apache.org/ "Apache ZooKeeper"
|
10
|
+
[zookeeper gem]: https://github.com/slyphon/zookeeper "slyphon-zookeeper gem"
|
11
|
+
[MIT]: http://www.gnu.org/licenses/license-list.html#Expat "MIT (Expat) License"
|
12
|
+
[Snapfish]: http://www.snapfish.com/ "Snapfish"
|
13
|
+
|
14
|
+
## What is ZooKeeper good for?
|
15
|
+
|
16
|
+
ZooKeeper is a multi-purpose tool that is designed to allow you to write code that coordinates many nodes in a cluster. It can be used as a directory service, a configuration database, and can provide cross-cluster [locking][], [leader election][], and [group membership][] (to name a few). It presents to the user what looks like a distributed file system, with a few important differences: every node can have children _and_ data, and there is a 1MB limit on data size for any given node. ZooKeeper provides atomic semantics and a simple API for manipulating data in the heirarchy.
|
17
|
+
|
18
|
+
One of the most useful aspects of ZooKeeper is the ability to set "[watches][]" on nodes. This allows one to be notified when a node has been deleted, created, has had a child modified, or had its data modified. The asynchronous nature of these watches enables you to write code that can _react_ to changes in your environment.
|
19
|
+
|
20
|
+
ZooKeeper is also (relatively) easy to deploy in a [Highly Available][ha-config] configuration, and the clients natively understand the clustering and how to resume a session transparently when one of the cluster nodes goes away.
|
21
|
+
|
22
|
+
|
23
|
+
[watches]: http://zookeeper.apache.org/doc/current/zookeeperProgrammers.html#ch_zkWatches
|
24
|
+
[locking]: http://zookeeper.apache.org/doc/current/recipes.html#sc_recipes_Locks
|
25
|
+
[leader election]: http://zookeeper.apache.org/doc/current/recipes.html#sc_leaderElection
|
26
|
+
[group membership]: http://zookeeper.apache.org/doc/current/recipes.html#sc_outOfTheBox
|
27
|
+
[ha-config]: http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_CrossMachineRequirements "HA config"
|
28
|
+
|
29
|
+
## What does ZK do that the zookeeper gem doesn't?
|
30
|
+
|
31
|
+
The [zookeeper gem][] provides a low-level, cross platform library for interfacing with ZooKeeper. While it is full featured, it only handles the basic operations that the driver provides. ZK implements the majority of the [recipes][] in the ZooKeeper documentation, plus a number of other conveniences for a production environment.
|
32
|
+
|
33
|
+
ZK provides:
|
34
|
+
|
35
|
+
* a robust lock implementation (both shared and exclusive locks)
|
36
|
+
* an extension for the [Mongoid][] ORM to provide advisory locks on mongodb records
|
37
|
+
* a leader election implementation with both "leader" and "observer" roles
|
38
|
+
* a higher-level interface to the ZooKeeper callback/watcher mechanism than the [zookeeper gem][] provides
|
39
|
+
* a simple threadpool implementation
|
40
|
+
* a bounded, dynamically-growable (threadsafe) client pool implementation
|
41
|
+
* a recursive Find class (like the Find module in ruby-core)
|
42
|
+
* unix-like rm\_rf and mkdir\_p methods (useful for functional testing)
|
43
|
+
|
44
|
+
In addition to all of that, I would like to think that the public API the ZK::Client provides is more convenient to use for the common (synchronous) case.
|
45
|
+
|
46
|
+
[recipes]: http://zookeeper.apache.org/doc/current/recipes.html
|
47
|
+
[Mongoid]: http://mongoid.org/
|
48
|
+
|
49
|
+
## Caveats
|
50
|
+
|
51
|
+
ZK strives to be a complete, correct, and convenient way of interacting with ZooKeeper. There are a few weak points in the implementation:
|
52
|
+
|
53
|
+
* _ACLS: HOW DO THEY WORK?!_ ACL support is mainly faith-based now. I have not had a need for ACLs, and the authors of the upstream [twitter/zookeeper][] code also don't seem to have much experience with them/use for them (purely my opinion, no offense intended). If you are using ACLs and you find bugs or have suggestions, I would much appreciate feedback or examples of how they *should* work so that support and tests can be added.
|
54
|
+
|
55
|
+
* ZK::Client supports asynchronous calls of all basic methods (get, set, delete, etc.) however these versions are kind of inconvenient to use. There is a [branch][async-branch] for making improvements in this regard. This will be improved in the near-term as a related EventMachine-based project will be making use of these.
|
56
|
+
|
57
|
+
* ZooKeeper "chroot" [connection syntax][chroot] _(search for "chroot" in page)_ is not currently working in the C drivers, and I don't have tests for the Java version. This hasn't been an incredibly high priority item, but support for this feature is intended.
|
58
|
+
|
59
|
+
* I am currently in the process of cleaning up the API documentation and converting it to use [YARD][]. You can follow along on [this branch][dev/yard] which will be merged into master and released ASAP.
|
60
|
+
|
61
|
+
[twitter/zookeeper]: https://github.com/twitter/zookeeper
|
62
|
+
[async-branch]: https://github.com/slyphon/zk/tree/dev%2Fasync-conveniences
|
63
|
+
[chroot]: http://zookeeper.apache.org/doc/current/zookeeperProgrammers.html#ch_zkSessions
|
64
|
+
[YARD]: http://yardoc.org/
|
65
|
+
[dev/yard]: https://github.com/slyphon/zk/tree/dev%2Fyard
|
66
|
+
|
67
|
+
## Dependencies
|
68
|
+
|
69
|
+
* The [slyphon-zookeeper gem][szk-gem] ([repo][szk-repo], branch with Gemfile [here][szk-repo-bundler]), which adds JRuby compatibility and a full suite of tests to the excellent [twitter/zookeeper][] project. _(I'm hoping to get this merged upstream, but it's a large change and, you know, people have day jobs)_.
|
70
|
+
|
71
|
+
* For JRuby, the [slyphon-zookeeper\_jar gem][szk-jar-gem] ([repo][szk-jar-repo]), which just wraps the upstream zookeeper driver jar in a gem for easy installation
|
72
|
+
|
73
|
+
[szk-gem]: https://rubygems.org/gems/slyphon-zookeeper
|
74
|
+
[szk-repo]: https://github.com/slyphon/zookeeper/tree/dev/xplatform
|
75
|
+
[szk-repo-bundler]: https://github.com/slyphon/zookeeper/tree/dev/gemfile/
|
76
|
+
[szk-jar-gem]: https://rubygems.org/gems/slyphon-zookeeper_jar
|
77
|
+
[szk-jar-repo]: https://github.com/slyphon/zookeeper_jar
|
78
|
+
|
79
|
+
### Related Projects
|
80
|
+
|
81
|
+
There are a few related projects that extend ZK.
|
82
|
+
|
83
|
+
* [ZK::Znode][]: a simple ORM to provide ActiveModel semantics around znodes. While still in early development, may also be a useful example of how to use ZK.
|
84
|
+
|
85
|
+
[ZK::Znode]: https://github.com/slyphon/zk-znode
|
86
|
+
|