zk 0.6.5 → 0.7.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/.dotfiles/rspec-logging +4 -0
- data/.gitignore +2 -0
- data/.yardopts +8 -0
- data/Gemfile +6 -1
- data/README.markdown +86 -0
- data/lib/z_k/client/base.rb +692 -0
- data/lib/z_k/client/conveniences.rb +134 -0
- data/lib/z_k/client/state_mixin.rb +94 -0
- data/lib/z_k/client/unixisms.rb +89 -0
- data/lib/z_k/client.rb +12 -891
- data/lib/z_k/election.rb +3 -0
- data/lib/z_k/event_handler.rb +7 -5
- data/lib/z_k/mongoid.rb +1 -1
- data/lib/z_k/pool.rb +70 -27
- data/lib/z_k/threadpool.rb +7 -2
- data/lib/z_k/version.rb +1 -1
- data/lib/z_k.rb +1 -2
- data/spec/spec_helper.rb +1 -0
- data/spec/support/logging_progress_bar_formatter.rb +14 -0
- data/spec/watch_spec.rb +26 -8
- data/spec/{client_spec.rb → z_k/client_spec.rb} +1 -1
- data/spec/{election_spec.rb → z_k/election_spec.rb} +2 -3
- data/spec/{locker_spec.rb → z_k/locker_spec.rb} +1 -1
- data/spec/{mongoid_spec.rb → z_k/mongoid_spec.rb} +1 -1
- data/spec/{client_pool_spec.rb → z_k/pool_spec.rb} +98 -126
- data/spec/{threadpool_spec.rb → z_k/threadpool_spec.rb} +6 -3
- data/zk.gemspec +1 -0
- metadata +37 -26
data/.gitignore
CHANGED
data/.yardopts
ADDED
data/Gemfile
CHANGED
@@ -5,6 +5,11 @@ source "http://localhost:50000"
|
|
5
5
|
# Specify your gem's dependencies in zk.gemspec
|
6
6
|
gemspec
|
7
7
|
|
8
|
-
|
8
|
+
gem 'ruby-debug', :platforms => [:mri_18, :jruby]
|
9
|
+
gem 'ruby-debug19', :platforms => :mri_19
|
10
|
+
|
11
|
+
# if `uname -s`.chomp == 'Linux'
|
12
|
+
# gem 'autotest-inotify', '~> 0.0.6'
|
13
|
+
# end
|
9
14
|
|
10
15
|
# vim:ft=ruby
|
data/README.markdown
ADDED
@@ -0,0 +1,86 @@
|
|
1
|
+
# ZK
|
2
|
+
|
3
|
+
ZK is a high-level interface to the Apache [ZooKeeper][] server. It is based on the [zookeeper gem][] which is a multi-Ruby low-level driver. Currently MRI 1.8.7 and JRuby are supported, and MRI 1.9.2 is very close to being ready. It is licensed under the [MIT][] license.
|
4
|
+
|
5
|
+
This library is heavily used in a production deployment and is actively developed and maintained.
|
6
|
+
|
7
|
+
Development is sponsored by [Snapfish][] and has been generously released to the Open Source community by HPDC, L.P.
|
8
|
+
|
9
|
+
[ZooKeeper]: http://zookeeper.apache.org/ "Apache ZooKeeper"
|
10
|
+
[zookeeper gem]: https://github.com/slyphon/zookeeper "slyphon-zookeeper gem"
|
11
|
+
[MIT]: http://www.gnu.org/licenses/license-list.html#Expat "MIT (Expat) License"
|
12
|
+
[Snapfish]: http://www.snapfish.com/ "Snapfish"
|
13
|
+
|
14
|
+
## What is ZooKeeper good for?
|
15
|
+
|
16
|
+
ZooKeeper is a multi-purpose tool that is designed to allow you to write code that coordinates many nodes in a cluster. It can be used as a directory service, a configuration database, and can provide cross-cluster [locking][], [leader election][], and [group membership][] (to name a few). It presents to the user what looks like a distributed file system, with a few important differences: every node can have children _and_ data, and there is a 1MB limit on data size for any given node. ZooKeeper provides atomic semantics and a simple API for manipulating data in the heirarchy.
|
17
|
+
|
18
|
+
One of the most useful aspects of ZooKeeper is the ability to set "[watches][]" on nodes. This allows one to be notified when a node has been deleted, created, has had a child modified, or had its data modified. The asynchronous nature of these watches enables you to write code that can _react_ to changes in your environment.
|
19
|
+
|
20
|
+
ZooKeeper is also (relatively) easy to deploy in a [Highly Available][ha-config] configuration, and the clients natively understand the clustering and how to resume a session transparently when one of the cluster nodes goes away.
|
21
|
+
|
22
|
+
|
23
|
+
[watches]: http://zookeeper.apache.org/doc/current/zookeeperProgrammers.html#ch_zkWatches
|
24
|
+
[locking]: http://zookeeper.apache.org/doc/current/recipes.html#sc_recipes_Locks
|
25
|
+
[leader election]: http://zookeeper.apache.org/doc/current/recipes.html#sc_leaderElection
|
26
|
+
[group membership]: http://zookeeper.apache.org/doc/current/recipes.html#sc_outOfTheBox
|
27
|
+
[ha-config]: http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_CrossMachineRequirements "HA config"
|
28
|
+
|
29
|
+
## What does ZK do that the zookeeper gem doesn't?
|
30
|
+
|
31
|
+
The [zookeeper gem][] provides a low-level, cross platform library for interfacing with ZooKeeper. While it is full featured, it only handles the basic operations that the driver provides. ZK implements the majority of the [recipes][] in the ZooKeeper documentation, plus a number of other conveniences for a production environment.
|
32
|
+
|
33
|
+
ZK provides:
|
34
|
+
|
35
|
+
* a robust lock implementation (both shared and exclusive locks)
|
36
|
+
* an extension for the [Mongoid][] ORM to provide advisory locks on mongodb records
|
37
|
+
* a leader election implementation with both "leader" and "observer" roles
|
38
|
+
* a higher-level interface to the ZooKeeper callback/watcher mechanism than the [zookeeper gem][] provides
|
39
|
+
* a simple threadpool implementation
|
40
|
+
* a bounded, dynamically-growable (threadsafe) client pool implementation
|
41
|
+
* a recursive Find class (like the Find module in ruby-core)
|
42
|
+
* unix-like rm\_rf and mkdir\_p methods (useful for functional testing)
|
43
|
+
|
44
|
+
In addition to all of that, I would like to think that the public API the ZK::Client provides is more convenient to use for the common (synchronous) case.
|
45
|
+
|
46
|
+
[recipes]: http://zookeeper.apache.org/doc/current/recipes.html
|
47
|
+
[Mongoid]: http://mongoid.org/
|
48
|
+
|
49
|
+
## Caveats
|
50
|
+
|
51
|
+
ZK strives to be a complete, correct, and convenient way of interacting with ZooKeeper. There are a few weak points in the implementation:
|
52
|
+
|
53
|
+
* _ACLS: HOW DO THEY WORK?!_ ACL support is mainly faith-based now. I have not had a need for ACLs, and the authors of the upstream [twitter/zookeeper][] code also don't seem to have much experience with them/use for them (purely my opinion, no offense intended). If you are using ACLs and you find bugs or have suggestions, I would much appreciate feedback or examples of how they *should* work so that support and tests can be added.
|
54
|
+
|
55
|
+
* ZK::Client supports asynchronous calls of all basic methods (get, set, delete, etc.) however these versions are kind of inconvenient to use. There is a [branch][async-branch] for making improvements in this regard. This will be improved in the near-term as a related EventMachine-based project will be making use of these.
|
56
|
+
|
57
|
+
* ZooKeeper "chroot" [connection syntax][chroot] _(search for "chroot" in page)_ is not currently working in the C drivers, and I don't have tests for the Java version. This hasn't been an incredibly high priority item, but support for this feature is intended.
|
58
|
+
|
59
|
+
* I am currently in the process of cleaning up the API documentation and converting it to use [YARD][]. You can follow along on [this branch][dev/yard] which will be merged into master and released ASAP.
|
60
|
+
|
61
|
+
[twitter/zookeeper]: https://github.com/twitter/zookeeper
|
62
|
+
[async-branch]: https://github.com/slyphon/zk/tree/dev%2Fasync-conveniences
|
63
|
+
[chroot]: http://zookeeper.apache.org/doc/current/zookeeperProgrammers.html#ch_zkSessions
|
64
|
+
[YARD]: http://yardoc.org/
|
65
|
+
[dev/yard]: https://github.com/slyphon/zk/tree/dev%2Fyard
|
66
|
+
|
67
|
+
## Dependencies
|
68
|
+
|
69
|
+
* The [slyphon-zookeeper gem][szk-gem] ([repo][szk-repo], branch with Gemfile [here][szk-repo-bundler]), which adds JRuby compatibility and a full suite of tests to the excellent [twitter/zookeeper][] project. _(I'm hoping to get this merged upstream, but it's a large change and, you know, people have day jobs)_.
|
70
|
+
|
71
|
+
* For JRuby, the [slyphon-zookeeper\_jar gem][szk-jar-gem] ([repo][szk-jar-repo]), which just wraps the upstream zookeeper driver jar in a gem for easy installation
|
72
|
+
|
73
|
+
[szk-gem]: https://rubygems.org/gems/slyphon-zookeeper
|
74
|
+
[szk-repo]: https://github.com/slyphon/zookeeper/tree/dev/xplatform
|
75
|
+
[szk-repo-bundler]: https://github.com/slyphon/zookeeper/tree/dev/gemfile/
|
76
|
+
[szk-jar-gem]: https://rubygems.org/gems/slyphon-zookeeper_jar
|
77
|
+
[szk-jar-repo]: https://github.com/slyphon/zookeeper_jar
|
78
|
+
|
79
|
+
### Related Projects
|
80
|
+
|
81
|
+
There are a few related projects that extend ZK.
|
82
|
+
|
83
|
+
* [ZK::Znode][]: a simple ORM to provide ActiveModel semantics around znodes. While still in early development, may also be a useful example of how to use ZK.
|
84
|
+
|
85
|
+
[ZK::Znode]: https://github.com/slyphon/zk-znode
|
86
|
+
|