adhd 0.1.1 → 0.1.2
Sign up to get free protection for your applications and to get access to all the features.
- data/README.rdoc +50 -11
- data/VERSION +1 -1
- data/adhd.gemspec +8 -3
- data/bin/adhd +3 -1
- data/doc/adhd.xmi +34 -20
- data/lib/adhd/adhd_rest_server.rb +0 -1
- data/lib/adhd/models/content_shard.rb +0 -1
- data/lib/adhd/models/replication_connection.rb +146 -0
- data/lib/adhd/models/shard_range.rb +1 -0
- data/lib/adhd/node_manager.rb +3 -3
- data/lib/adhd/reactor.rb +3 -19
- data/test/unit/test_node.rb +3 -1
- data/test/unit/test_node_db.rb +133 -0
- data/test/unit/test_replication_connection.rb +112 -0
- metadata +7 -2
data/README.rdoc
CHANGED
|
@@ -2,15 +2,6 @@
|
|
|
2
2
|
|
|
3
3
|
Adhd is an asynchronous, distributed hard drive. Actually we're not sure if it's really asynchronous or even what asynchronicity would mean in the context of a hard drive, but it is definitely distributed. Adhd is essentially a management layer (written using Ruby and eventmachine) which controls clusters of CouchDB databases to replicate files across disparate machines. Unlike most clustering storage solutions, adhd assumes that machines in the cluster may have different capabilities and is designed to work both inside and outside the data centre.
|
|
4
4
|
|
|
5
|
-
== Installation
|
|
6
|
-
|
|
7
|
-
Don't use this software yet. It is experimental and may eat your mother.
|
|
8
|
-
|
|
9
|
-
Having said that,
|
|
10
|
-
|
|
11
|
-
sudo gem install adhd
|
|
12
|
-
|
|
13
|
-
may do the job.
|
|
14
5
|
|
|
15
6
|
== How it works
|
|
16
7
|
|
|
@@ -64,6 +55,54 @@ Archive.org: those sex-ed videos and Bert the turtle from the Prelinger Archive
|
|
|
64
55
|
|
|
65
56
|
BBC News: material going up on the site right now will need a lot of bandwidth, but by next week most of this media will be out of the news cycle and can be consigned to a set of servers with much less bandwidth. By next year, it is unlikely that this media will be viewed even a few times a day, so it could be smart to trade storage costs for speed and put old media on cheaper, lower-bandwidth boxes with huge hard drives.
|
|
66
57
|
|
|
58
|
+
== Installation
|
|
59
|
+
|
|
60
|
+
Don't use this software yet. It is experimental and may eat your mother.
|
|
61
|
+
|
|
62
|
+
Having said that,
|
|
63
|
+
|
|
64
|
+
sudo apt-get install couchdb; sudo gem install adhd
|
|
65
|
+
|
|
66
|
+
may do the job.
|
|
67
|
+
|
|
68
|
+
== Usage
|
|
69
|
+
|
|
70
|
+
You'll need a node_config.yml file that looks something like this:
|
|
71
|
+
|
|
72
|
+
---
|
|
73
|
+
node_name: my_adhd_node
|
|
74
|
+
node_url: "192.168.1.93"
|
|
75
|
+
couchdb_server_port: 5984
|
|
76
|
+
http_server_port: 5985
|
|
77
|
+
buddy_server_url: "http://192.168.1.74:5984"
|
|
78
|
+
buddy_server_node_name: another_adhd_node
|
|
79
|
+
|
|
80
|
+
The config parameters are:
|
|
81
|
+
|
|
82
|
+
<tt>node_name</tt>: a unique prefix name that will be used as an identifier for your nodes in the global node database.
|
|
83
|
+
|
|
84
|
+
<tt>node_url</tt>: the hostname or IP which the node should bind to.
|
|
85
|
+
|
|
86
|
+
<tt>couchdb_server_port</tt>: the port that CouchDB is running on, usually 5984.
|
|
87
|
+
|
|
88
|
+
<tt>http_server_port</tt>: the port which the adhd node should run on.
|
|
89
|
+
|
|
90
|
+
<tt>buddy_server_url</tt>: the URL (including the CouchDB port) of any other adhd node which is currently running. The node will connect to the buddy_server_url and replicate the cluster's management database information from it. This can be blank for the first node or if you want to make an entirely new cluster.
|
|
91
|
+
|
|
92
|
+
<tt>buddy_server_node_name</tt>: the node_name of the buddy node as set in its config file.
|
|
93
|
+
|
|
94
|
+
You can start adhd once it's installed by invoking it like this:
|
|
95
|
+
|
|
96
|
+
adhd -c /path/to/node_config.yml
|
|
97
|
+
|
|
98
|
+
If it worked, you should see a bunch of messages in your console telling you what's replicating, how many shards there are, etc. If you want to test out multiple nodes on the same machine, you can simply have multiple config files which use different <tt>http_server_port</tt> and <tt>node_name</tt> values.
|
|
99
|
+
|
|
100
|
+
Once a node has started, you can do something like this to add content to it:
|
|
101
|
+
|
|
102
|
+
curl -i -0 -T /path/to/somefile.txt http://192.168.1.93:5985/adhd/somefile.txt
|
|
103
|
+
|
|
104
|
+
That command should PUT somefile.txt into the adhd cluster. If there is more than one node running, somefile.txt will be automatically replicated to the proper shard.
|
|
105
|
+
|
|
67
106
|
|
|
68
107
|
== Note on Patches/Pull Requests
|
|
69
108
|
|
|
@@ -71,9 +110,9 @@ BBC News: material going up on the site right now will need a lot of bandwidth,
|
|
|
71
110
|
* Make your feature addition or bug fix.
|
|
72
111
|
* Add tests for it. This is important so I don't break it in a
|
|
73
112
|
future version unintentionally.
|
|
74
|
-
* Commit, do not mess with rakefile, version, or history
|
|
113
|
+
* Commit, do not mess with rakefile, version, or history
|
|
75
114
|
(if you want to have your own version, that is fine but
|
|
76
|
-
|
|
115
|
+
bump version in a commit by itself I can ignore when I pull).
|
|
77
116
|
* Send me a pull request. Bonus points for topic branches.
|
|
78
117
|
|
|
79
118
|
== Copyright
|
data/VERSION
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
0.1.
|
|
1
|
+
0.1.2
|
data/adhd.gemspec
CHANGED
|
@@ -5,11 +5,11 @@
|
|
|
5
5
|
|
|
6
6
|
Gem::Specification.new do |s|
|
|
7
7
|
s.name = %q{adhd}
|
|
8
|
-
s.version = "0.1.
|
|
8
|
+
s.version = "0.1.2"
|
|
9
9
|
|
|
10
10
|
s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
|
|
11
11
|
s.authors = ["dave.hrycyszyn@headlondon.com"]
|
|
12
|
-
s.date = %q{2009-12-
|
|
12
|
+
s.date = %q{2009-12-28}
|
|
13
13
|
s.description = %q{More to say when something works! Do not bother installing this! }
|
|
14
14
|
s.email = %q{dave.hrycyszyn@headlondon.com}
|
|
15
15
|
s.executables = ["adhd", "adhd_cleanup"]
|
|
@@ -34,6 +34,7 @@ Gem::Specification.new do |s|
|
|
|
34
34
|
"lib/adhd/models/content_shard.rb",
|
|
35
35
|
"lib/adhd/models/node_db.rb",
|
|
36
36
|
"lib/adhd/models/node_doc.rb",
|
|
37
|
+
"lib/adhd/models/replication_connection.rb",
|
|
37
38
|
"lib/adhd/models/shard_range.rb",
|
|
38
39
|
"lib/adhd/node_manager.rb",
|
|
39
40
|
"lib/adhd/reactor.rb",
|
|
@@ -60,7 +61,9 @@ Gem::Specification.new do |s|
|
|
|
60
61
|
"test/helper.rb",
|
|
61
62
|
"test/test_adhd.rb",
|
|
62
63
|
"test/unit/test_content_doc.rb",
|
|
63
|
-
"test/unit/test_node.rb"
|
|
64
|
+
"test/unit/test_node.rb",
|
|
65
|
+
"test/unit/test_node_db.rb",
|
|
66
|
+
"test/unit/test_replication_connection.rb"
|
|
64
67
|
]
|
|
65
68
|
s.homepage = %q{http://github.com/futurechimp/adhd}
|
|
66
69
|
s.rdoc_options = ["--charset=UTF-8"]
|
|
@@ -69,6 +72,8 @@ Gem::Specification.new do |s|
|
|
|
69
72
|
s.summary = %q{An experiment in distributed file replication using CouchDB}
|
|
70
73
|
s.test_files = [
|
|
71
74
|
"test/unit/test_content_doc.rb",
|
|
75
|
+
"test/unit/test_replication_connection.rb",
|
|
76
|
+
"test/unit/test_node_db.rb",
|
|
72
77
|
"test/unit/test_node.rb",
|
|
73
78
|
"test/helper.rb",
|
|
74
79
|
"test/test_adhd.rb"
|
data/bin/adhd
CHANGED
|
@@ -40,8 +40,10 @@ EM.run {
|
|
|
40
40
|
@node_manager.run
|
|
41
41
|
end
|
|
42
42
|
|
|
43
|
+
|
|
43
44
|
# Start the server
|
|
44
|
-
EventMachine::start_server @config.node_url, @config.
|
|
45
|
+
EventMachine::start_server @config.node_url, @config.http_server_port, AdhdRESTServer, @node_manager
|
|
46
|
+
|
|
45
47
|
|
|
46
48
|
|
|
47
49
|
}
|
data/doc/adhd.xmi
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
<?xml version="1.0" encoding="UTF-8"?>
|
|
2
|
-
<XMI verified="false" xmi.version="1.2" timestamp="2009-12-
|
|
2
|
+
<XMI verified="false" xmi.version="1.2" timestamp="2009-12-27T19:12:20" xmlns:UML="http://schema.omg.org/spec/UML/1.3" >
|
|
3
3
|
<XMI.header>
|
|
4
4
|
<XMI.documentation>
|
|
5
5
|
<XMI.exporter>umbrello uml modeller http://uml.sf.net</XMI.exporter>
|
|
@@ -163,21 +163,23 @@
|
|
|
163
163
|
</UML:Namespace.ownedElement>
|
|
164
164
|
<XMI.extension xmi.extender="umbrello" >
|
|
165
165
|
<diagrams>
|
|
166
|
-
<diagram showopsig="1" linecolor="#ff0000" snapx="10" showattribassocs="1" snapy="10" linewidth="0" showattsig="1" showpubliconly="1" showpackage="1" showstereotype="1" name="class diagram" font="DejaVu Sans,9,-1,0,50,0,0,0,0,0" canvasheight="
|
|
166
|
+
<diagram showopsig="1" linecolor="#ff0000" snapx="10" showattribassocs="1" snapy="10" linewidth="0" showattsig="1" showpubliconly="1" showpackage="1" showstereotype="1" name="class diagram" font="DejaVu Sans,9,-1,0,50,0,0,0,0,0" canvasheight="903" canvaswidth="1354" localid="" snapcsgrid="0" showgrid="0" showops="1" usefillcolor="1" fillcolor="#ffff00" zoom="100" xmi.id="GBfDa8xOOgRz" documentation="" showscope="1" snapgrid="0" showatts="1" type="1" >
|
|
167
167
|
<widgets>
|
|
168
|
-
<classwidget linecolor="#ff0000" usesdiagramfillcolor="0" linewidth="none" showoperations="1" usesdiagramusefillcolor="0" showpubliconly="1" showpackage="1" x="427" showattsigs="601" showstereotype="1" y="49" showattributes="1" font="DejaVu Sans,9,-1,5,
|
|
169
|
-
<classwidget linecolor="#ff0000" usesdiagramfillcolor="0" linewidth="none" showoperations="1" usesdiagramusefillcolor="0" showpubliconly="1" showpackage="1" x="358" showattsigs="601" showstereotype="1" y="316" showattributes="1" font="DejaVu Sans,9,-1,5,
|
|
170
|
-
<classwidget linecolor="#ff0000" usesdiagramfillcolor="0" linewidth="none" showoperations="1" usesdiagramusefillcolor="0" showpubliconly="1" showpackage="1" x="627" showattsigs="601" showstereotype="1" y="639" showattributes="1" font="DejaVu Sans,9,-1,5,
|
|
171
|
-
<classwidget linecolor="#ff0000" usesdiagramfillcolor="0" linewidth="none" showoperations="1" usesdiagramusefillcolor="0" showpubliconly="1" showpackage="1" x="698" showattsigs="601" showstereotype="1" y="759" showattributes="1" font="DejaVu Sans,9,-1,5,
|
|
168
|
+
<classwidget linecolor="#ff0000" usesdiagramfillcolor="0" linewidth="none" showoperations="1" usesdiagramusefillcolor="0" showpubliconly="1" showpackage="1" x="427" showattsigs="601" showstereotype="1" y="49" showattributes="1" font="DejaVu Sans,9,-1,5,50,0,0,0,0,0" width="211" isinstance="0" usefillcolor="1" fillcolor="#ffff00" xmi.id="ZOusZTQl90qE" showscope="1" height="202" showopsigs="601" />
|
|
169
|
+
<classwidget linecolor="#ff0000" usesdiagramfillcolor="0" linewidth="none" showoperations="1" usesdiagramusefillcolor="0" showpubliconly="1" showpackage="1" x="358" showattsigs="601" showstereotype="1" y="316" showattributes="1" font="DejaVu Sans,9,-1,5,50,0,0,0,0,0" width="172" isinstance="0" usefillcolor="1" fillcolor="#ffff00" xmi.id="F64MWYR8QXAj" showscope="1" height="97" showopsigs="601" />
|
|
170
|
+
<classwidget linecolor="#ff0000" usesdiagramfillcolor="0" linewidth="none" showoperations="1" usesdiagramusefillcolor="0" showpubliconly="1" showpackage="1" x="627" showattsigs="601" showstereotype="1" y="639" showattributes="1" font="DejaVu Sans,9,-1,5,50,0,0,0,0,0" width="219" isinstance="0" usefillcolor="1" fillcolor="#ffc0ff" xmi.id="wUO4Nz4auhzE" showscope="1" height="37" showopsigs="601" />
|
|
171
|
+
<classwidget linecolor="#ff0000" usesdiagramfillcolor="0" linewidth="none" showoperations="1" usesdiagramusefillcolor="0" showpubliconly="1" showpackage="1" x="698" showattsigs="601" showstereotype="1" y="759" showattributes="1" font="DejaVu Sans,9,-1,5,50,0,0,0,0,0" width="140" isinstance="0" usefillcolor="1" fillcolor="#ffc0ff" xmi.id="sTW11oYXAewC" showscope="1" height="97" showopsigs="601" />
|
|
172
172
|
<boxwidget width="744" showstereotype="1" x="276" y="584" usesdiagramusefillcolor="1" usesdiagramfillcolor="1" isinstance="0" fillcolor="none" height="315" linecolor="#000000" xmi.id="OFA6aXy1qwJZ" usefillcolor="1" linewidth="none" font="DejaVu Sans,9,-1,0,50,0,0,0,0,0" />
|
|
173
173
|
<boxwidget width="814" showstereotype="1" x="85" y="33" usesdiagramusefillcolor="1" usesdiagramfillcolor="1" isinstance="0" fillcolor="none" height="433" linecolor="#000000" xmi.id="nESEEtzKAb1x" usefillcolor="1" linewidth="none" font="DejaVu Sans,9,-1,0,50,0,0,0,0,0" />
|
|
174
|
-
<notewidget width="244" showstereotype="1" x="289" noteType="0" y="593" usesdiagramusefillcolor="1" usesdiagramfillcolor="
|
|
174
|
+
<notewidget width="244" showstereotype="1" x="289" noteType="0" y="593" usesdiagramusefillcolor="1" usesdiagramfillcolor="0" isinstance="0" fillcolor="#ffc0ff" height="104" linecolor="none" xmi.id="vZuKv7S2fyye" usefillcolor="1" linewidth="none" font="DejaVu Sans,9,-1,0,50,0,0,0,0,0" text="STORAGE SERVER 
(pink = CouchDB only)

Stores some nice big files for us. The documents inside each server are sharded by internal_id." />
|
|
175
175
|
<notewidget width="259" showstereotype="1" x="104" noteType="0" y="53" usesdiagramusefillcolor="1" usesdiagramfillcolor="1" isinstance="0" fillcolor="none" height="182" linecolor="none" xmi.id="ovwNVBPINpWc" usefillcolor="1" linewidth="none" font="DejaVu Sans,9,-1,0,50,0,0,0,0,0" text="GLOBAL STATE

All nodes in the system, no matter whether they are only storage servers or whether they are management or directory servers, have all of this information all the time (or at least, as quickly as it can be replicated to any particular node)." />
|
|
176
|
-
<classwidget linecolor="#ff0000" usesdiagramfillcolor="0" linewidth="none" showoperations="1" usesdiagramusefillcolor="0" showpubliconly="1" showpackage="1" x="1075" showattsigs="601" showstereotype="1" y="247" showattributes="1" font="DejaVu Sans,9,-1,5,
|
|
176
|
+
<classwidget linecolor="#ff0000" usesdiagramfillcolor="0" linewidth="none" showoperations="1" usesdiagramusefillcolor="0" showpubliconly="1" showpackage="1" x="1075" showattsigs="601" showstereotype="1" y="247" showattributes="1" font="DejaVu Sans,9,-1,5,50,0,0,0,0,0" width="161" isinstance="0" usefillcolor="1" fillcolor="#ffff00" xmi.id="hILOH0lmr6on" showscope="1" height="52" showopsigs="601" />
|
|
177
177
|
<boxwidget width="353" showstereotype="1" x="970" y="33" usesdiagramusefillcolor="1" usesdiagramfillcolor="1" isinstance="0" fillcolor="none" height="310" linecolor="#000000" xmi.id="DQReyV7fne9s" usefillcolor="1" linewidth="none" font="DejaVu Sans,9,-1,0,50,0,0,0,0,0" />
|
|
178
178
|
<notewidget width="324" showstereotype="1" x="977" noteType="0" y="43" usesdiagramusefillcolor="1" usesdiagramfillcolor="1" isinstance="0" fillcolor="none" height="159" linecolor="none" xmi.id="lr4y3tzEesX4" usefillcolor="1" linewidth="none" font="DejaVu Sans,9,-1,0,50,0,0,0,0,0" text="DIRECTORY SERVER

Takes external requests, asking the ShardFinder which Shard a given Document exists in (based on the internal_id of the Document). Any server can be a directory server, but in practice, not every server will be a directory server at all times (we may need only 1/10 of servers to be a directory server)." />
|
|
179
|
-
<classwidget linecolor="#ff0000" usesdiagramfillcolor="0" linewidth="none" showoperations="1" usesdiagramusefillcolor="0" showpubliconly="1" showpackage="1" x="575" showattsigs="601" showstereotype="1" y="325" showattributes="1" font="DejaVu Sans,9,-1,5,
|
|
179
|
+
<classwidget linecolor="#ff0000" usesdiagramfillcolor="0" linewidth="none" showoperations="1" usesdiagramusefillcolor="0" showpubliconly="1" showpackage="1" x="575" showattsigs="601" showstereotype="1" y="325" showattributes="1" font="DejaVu Sans,9,-1,5,50,0,0,0,0,0" width="296" isinstance="0" usefillcolor="1" fillcolor="#ffff00" xmi.id="EeuiBhd3b9sY" showscope="1" height="45" showopsigs="601" />
|
|
180
180
|
<boxwidget width="1290" showstereotype="1" x="60" y="13" usesdiagramusefillcolor="1" usesdiagramfillcolor="1" isinstance="0" fillcolor="none" height="466" linecolor="#000000" xmi.id="9qGy57wvLTod" usefillcolor="1" linewidth="none" font="DejaVu Sans,9,-1,0,50,0,0,0,0,0" />
|
|
181
|
+
<notewidget width="119" showstereotype="1" x="884" noteType="0" y="631" usesdiagramusefillcolor="1" usesdiagramfillcolor="0" isinstance="0" fillcolor="#ffc0ff" height="50" linecolor="none" xmi.id="91FQ8XmhMEAr" usefillcolor="1" linewidth="none" font="DejaVu Sans,9,-1,0,50,0,0,0,0,0" text="A CouchDB database." />
|
|
182
|
+
<notewidget width="180" showstereotype="1" x="463" noteType="0" y="780" usesdiagramusefillcolor="1" usesdiagramfillcolor="0" isinstance="0" fillcolor="#ffc0ff" height="62" linecolor="none" xmi.id="dDNh80rednIm" usefillcolor="1" linewidth="none" font="DejaVu Sans,9,-1,0,50,0,0,0,0,0" text="A CouchDB document containing a single attachment." />
|
|
181
183
|
</widgets>
|
|
182
184
|
<messages/>
|
|
183
185
|
<associations>
|
|
@@ -186,7 +188,7 @@
|
|
|
186
188
|
<startpoint startx="490" starty="316" />
|
|
187
189
|
<endpoint endx="490" endy="251" />
|
|
188
190
|
</linepath>
|
|
189
|
-
<floatingtext linecolor="none" usesdiagramfillcolor="1" linewidth="none" usesdiagramusefillcolor="1" x="392" showstereotype="1" y="253" text="master_node" font="DejaVu Sans,9,-1,0,50,0,0,0,0,0" pretext="+" role="710" width="96" isinstance="0" posttext="" usefillcolor="1" fillcolor="none" xmi.id="
|
|
191
|
+
<floatingtext linecolor="none" usesdiagramfillcolor="1" linewidth="none" usesdiagramusefillcolor="1" x="392" showstereotype="1" y="253" text="master_node" font="DejaVu Sans,9,-1,0,50,0,0,0,0,0" pretext="+" role="710" width="96" isinstance="0" posttext="" usefillcolor="1" fillcolor="none" xmi.id="jFCVzQKfOAMB" height="19" />
|
|
190
192
|
</assocwidget>
|
|
191
193
|
<assocwidget indexa="1" indexb="1" widgetaid="wUO4Nz4auhzE" linecolor="none" totalcounta="2" xmi.id="O5m0C53pXZgr" widgetbid="sTW11oYXAewC" totalcountb="2" type="510" linewidth="none" >
|
|
192
194
|
<linepath>
|
|
@@ -199,7 +201,7 @@
|
|
|
199
201
|
<startpoint startx="530" starty="413" />
|
|
200
202
|
<endpoint endx="627" endy="639" />
|
|
201
203
|
</linepath>
|
|
202
|
-
<floatingtext linecolor="none" usesdiagramfillcolor="1" linewidth="none" usesdiagramusefillcolor="1" x="598" showstereotype="1" y="515" text="shard" font="DejaVu Sans,9,-1,0,50,0,0,0,0,0" pretext="+" role="710" width="52" isinstance="0" posttext="" usefillcolor="1" fillcolor="none" xmi.id="
|
|
204
|
+
<floatingtext linecolor="none" usesdiagramfillcolor="1" linewidth="none" usesdiagramusefillcolor="1" x="598" showstereotype="1" y="515" text="shard" font="DejaVu Sans,9,-1,0,50,0,0,0,0,0" pretext="+" role="710" width="52" isinstance="0" posttext="" usefillcolor="1" fillcolor="none" xmi.id="Zbb9HbaWCrDh" height="19" />
|
|
203
205
|
</assocwidget>
|
|
204
206
|
<assocwidget indexa="1" indexb="1" visibilityA="0" widgetaid="hILOH0lmr6on" visibilityB="0" linecolor="none" changeabilityA="900" totalcounta="2" xmi.id="ERd1IA9cAzjx" changeabilityB="900" widgetbid="sTW11oYXAewC" totalcountb="2" type="510" linewidth="none" >
|
|
205
207
|
<linepath>
|
|
@@ -207,9 +209,9 @@
|
|
|
207
209
|
<endpoint endx="838" endy="832" />
|
|
208
210
|
<point x="1162" y="832" />
|
|
209
211
|
</linepath>
|
|
210
|
-
<floatingtext linecolor="none" usesdiagramfillcolor="1" linewidth="none" usesdiagramusefillcolor="1" x="1203" showstereotype="1" y="302" text="1" font="DejaVu Sans,9,-1,0,50,0,0,0,0,0" pretext="" role="701" width="16" isinstance="0" posttext="" usefillcolor="1" fillcolor="none" xmi.id="
|
|
211
|
-
<floatingtext linecolor="none" usesdiagramfillcolor="1" linewidth="none" usesdiagramusefillcolor="1" x="841" showstereotype="1" y="810" text="1" font="DejaVu Sans,9,-1,0,50,0,0,0,0,0" pretext="" role="702" width="16" isinstance="0" posttext="" usefillcolor="1" fillcolor="none" xmi.id="
|
|
212
|
-
<floatingtext linecolor="none" usesdiagramfillcolor="1" linewidth="none" usesdiagramusefillcolor="1" x="840" showstereotype="1" y="834" text="document" font="DejaVu Sans,9,-1,0,50,0,0,0,0,0" pretext="+" role="710" width="80" isinstance="0" posttext="" usefillcolor="1" fillcolor="none" xmi.id="
|
|
212
|
+
<floatingtext linecolor="none" usesdiagramfillcolor="1" linewidth="none" usesdiagramusefillcolor="1" x="1203" showstereotype="1" y="302" text="1" font="DejaVu Sans,9,-1,0,50,0,0,0,0,0" pretext="" role="701" width="16" isinstance="0" posttext="" usefillcolor="1" fillcolor="none" xmi.id="qewFpGOMrzU6" height="19" />
|
|
213
|
+
<floatingtext linecolor="none" usesdiagramfillcolor="1" linewidth="none" usesdiagramusefillcolor="1" x="841" showstereotype="1" y="810" text="1" font="DejaVu Sans,9,-1,0,50,0,0,0,0,0" pretext="" role="702" width="16" isinstance="0" posttext="" usefillcolor="1" fillcolor="none" xmi.id="Y2dktqPGJo0S" height="19" />
|
|
214
|
+
<floatingtext linecolor="none" usesdiagramfillcolor="1" linewidth="none" usesdiagramusefillcolor="1" x="840" showstereotype="1" y="834" text="document" font="DejaVu Sans,9,-1,0,50,0,0,0,0,0" pretext="+" role="710" width="80" isinstance="0" posttext="" usefillcolor="1" fillcolor="none" xmi.id="OVRk97GsMSMJ" height="19" />
|
|
213
215
|
</assocwidget>
|
|
214
216
|
<assocwidget indexa="1" indexb="1" widgetaid="EeuiBhd3b9sY" linecolor="none" totalcounta="2" xmi.id="GxipgWChsqCG" widgetbid="F64MWYR8QXAj" totalcountb="2" type="510" linewidth="none" >
|
|
215
217
|
<linepath>
|
|
@@ -217,6 +219,18 @@
|
|
|
217
219
|
<endpoint endx="530" endy="352" />
|
|
218
220
|
</linepath>
|
|
219
221
|
</assocwidget>
|
|
222
|
+
<assocwidget indexa="1" indexb="1" visibilityA="0" widgetaid="wUO4Nz4auhzE" visibilityB="0" roleBdoc="" roleAdoc="" linecolor="none" changeabilityA="900" totalcounta="2" changeabilityB="900" widgetbid="91FQ8XmhMEAr" totalcountb="2" type="513" documentation="" linewidth="none" >
|
|
223
|
+
<linepath>
|
|
224
|
+
<startpoint startx="846" starty="650" />
|
|
225
|
+
<endpoint endx="884" endy="650" />
|
|
226
|
+
</linepath>
|
|
227
|
+
</assocwidget>
|
|
228
|
+
<assocwidget indexa="1" indexb="1" visibilityA="0" widgetaid="dDNh80rednIm" visibilityB="0" roleBdoc="" roleAdoc="" linecolor="none" changeabilityA="900" totalcounta="2" changeabilityB="900" widgetbid="sTW11oYXAewC" totalcountb="2" type="513" documentation="" linewidth="none" >
|
|
229
|
+
<linepath>
|
|
230
|
+
<startpoint startx="643" starty="806" />
|
|
231
|
+
<endpoint endx="698" endy="806" />
|
|
232
|
+
</linepath>
|
|
233
|
+
</assocwidget>
|
|
220
234
|
</associations>
|
|
221
235
|
</diagram>
|
|
222
236
|
</diagrams>
|
|
@@ -235,7 +249,7 @@
|
|
|
235
249
|
</UML:Namespace.ownedElement>
|
|
236
250
|
<XMI.extension xmi.extender="umbrello" >
|
|
237
251
|
<diagrams>
|
|
238
|
-
<diagram showopsig="1" linecolor="#ff0000" snapx="10" showattribassocs="1" snapy="10" linewidth="0" showattsig="1" showpubliconly="1" showpackage="1" showstereotype="1" name="System Overview" font="DejaVu Sans,9,-1,0,50,0,0,0,0,0" canvasheight="
|
|
252
|
+
<diagram showopsig="1" linecolor="#ff0000" snapx="10" showattribassocs="1" snapy="10" linewidth="0" showattsig="1" showpubliconly="1" showpackage="1" showstereotype="1" name="System Overview" font="DejaVu Sans,9,-1,0,50,0,0,0,0,0" canvasheight="589" canvaswidth="900" localid="" snapcsgrid="0" showgrid="0" showops="1" usefillcolor="1" fillcolor="#ffff00" zoom="100" xmi.id="052X7SM5OGmL" documentation="" showscope="1" snapgrid="0" showatts="1" type="8" >
|
|
239
253
|
<widgets>
|
|
240
254
|
<nodewidget width="133" showstereotype="1" x="726" y="215" usesdiagramusefillcolor="1" usesdiagramfillcolor="1" isinstance="0" fillcolor="none" height="60" linecolor="none" xmi.id="Nuj9xaygETYj" usefillcolor="1" linewidth="none" font="DejaVu Sans,9,-1,0,75,0,0,0,0,0" />
|
|
241
255
|
<nodewidget width="142" showstereotype="1" x="254" y="172" usesdiagramusefillcolor="1" usesdiagramfillcolor="1" isinstance="0" fillcolor="none" height="60" linecolor="none" xmi.id="j1zm1UuTqxq4" usefillcolor="1" linewidth="none" font="DejaVu Sans,9,-1,0,75,0,0,0,0,0" />
|
|
@@ -259,7 +273,7 @@
|
|
|
259
273
|
</UML:Model>
|
|
260
274
|
</XMI.content>
|
|
261
275
|
<XMI.extensions xmi.extender="umbrello" >
|
|
262
|
-
<docsettings viewid="GBfDa8xOOgRz" uniqueid="
|
|
276
|
+
<docsettings viewid="GBfDa8xOOgRz" uniqueid="dDNh80rednIm" documentation="" />
|
|
263
277
|
<listview>
|
|
264
278
|
<listitem open="1" type="800" label="Views" >
|
|
265
279
|
<listitem open="1" type="801" id="Logical View" >
|
|
@@ -277,10 +291,6 @@
|
|
|
277
291
|
<listitem open="0" type="814" id="Ba2fkY4nqcUG" />
|
|
278
292
|
</listitem>
|
|
279
293
|
<listitem open="1" type="813" id="Mm8apLIAakhF" />
|
|
280
|
-
<listitem open="0" type="813" id="EeuiBhd3b9sY" >
|
|
281
|
-
<listitem open="0" type="814" id="X8v0Kas2jXfm" />
|
|
282
|
-
<listitem open="0" type="815" id="w6uusFVwtzxH" />
|
|
283
|
-
</listitem>
|
|
284
294
|
<listitem open="0" type="813" id="hILOH0lmr6on" >
|
|
285
295
|
<listitem open="0" type="814" id="qom9C28voz1c" />
|
|
286
296
|
<listitem open="0" type="814" id="ERd1IA9cAzjx" />
|
|
@@ -305,6 +315,10 @@
|
|
|
305
315
|
<listitem open="0" type="813" id="wUO4Nz4auhzE" >
|
|
306
316
|
<listitem open="0" type="814" id="zT7sAzXvkfbr" />
|
|
307
317
|
</listitem>
|
|
318
|
+
<listitem open="0" type="813" id="EeuiBhd3b9sY" >
|
|
319
|
+
<listitem open="0" type="814" id="X8v0Kas2jXfm" />
|
|
320
|
+
<listitem open="0" type="815" id="w6uusFVwtzxH" />
|
|
321
|
+
</listitem>
|
|
308
322
|
<listitem open="0" type="813" id="F64MWYR8QXAj" >
|
|
309
323
|
<listitem open="0" type="814" id="LymfSI7aiciQ" />
|
|
310
324
|
<listitem open="0" type="814" id="4lmtm4EA2GI6" />
|
|
@@ -0,0 +1,146 @@
|
|
|
1
|
+
|
|
2
|
+
|
|
3
|
+
module Adhd
|
|
4
|
+
|
|
5
|
+
module ReplicationNotifier
|
|
6
|
+
|
|
7
|
+
def initialize(conn_obj)
|
|
8
|
+
@conn_obj = conn_obj
|
|
9
|
+
@buffer = ""
|
|
10
|
+
conn_obj.connection_inside = self # We tell the outer object who we are
|
|
11
|
+
# pending_connect_timeout= 5.0
|
|
12
|
+
end
|
|
13
|
+
|
|
14
|
+
# Makes a long-running request to a CouchDB instance
|
|
15
|
+
# Implement PULL replication, as it is most efficient
|
|
16
|
+
# in CouchDB 0.9
|
|
17
|
+
#
|
|
18
|
+
def post_init
|
|
19
|
+
# Build a JSON representation
|
|
20
|
+
r = {:source => "#{@conn_obj.remote_db}",
|
|
21
|
+
:target => "#{@conn_obj.our_db}"}
|
|
22
|
+
#, :continuous => true }
|
|
23
|
+
r_json = r.to_json
|
|
24
|
+
|
|
25
|
+
# Create the HTTP request
|
|
26
|
+
req = "POST /_replicate HTTP/1.1\r\n"
|
|
27
|
+
req += "Content-Length: #{r_json.length}\r\n\r\n"
|
|
28
|
+
req += "#{r_json}"
|
|
29
|
+
|
|
30
|
+
# Push it to the network
|
|
31
|
+
send_data req
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
# Shoots replication events from CouchDB to the @conn.
|
|
35
|
+
# Buffers data until a JSON object is detected
|
|
36
|
+
#
|
|
37
|
+
def receive_data data
|
|
38
|
+
|
|
39
|
+
@buffer += data # Add the data to the current buffer
|
|
40
|
+
updates = []
|
|
41
|
+
if @buffer =~ /(\{[^\n]+\}\n)/
|
|
42
|
+
updates += ($~.to_a)[1..-1]
|
|
43
|
+
# Trim the buffer to $_.end(0)
|
|
44
|
+
@buffer = @buffer[$~.end(0)..-1]
|
|
45
|
+
end
|
|
46
|
+
|
|
47
|
+
# Regexp for JSON updates is /\{[\n]\}+/
|
|
48
|
+
updates.each do |json_event|
|
|
49
|
+
@conn_obj.event_handler(json_event) unless data == "\n"
|
|
50
|
+
end
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
def unbind
|
|
54
|
+
# TODO: detect when the remote node is down and update their
|
|
55
|
+
# status
|
|
56
|
+
end
|
|
57
|
+
|
|
58
|
+
end
|
|
59
|
+
|
|
60
|
+
|
|
61
|
+
# Manages a connection to our CouchDB that maintains continuous replications
|
|
62
|
+
# to remote CouchDB instances. We make sure that the connection does not go
|
|
63
|
+
# down while it is needed.
|
|
64
|
+
#
|
|
65
|
+
class ReplicationConnection
|
|
66
|
+
|
|
67
|
+
attr_accessor :our_node, :our_db, :remote_node, :remote_db, :connection_inside, :name
|
|
68
|
+
|
|
69
|
+
# Initiate a replication connection, by passing the local and remote
|
|
70
|
+
# nodes and databases.
|
|
71
|
+
#
|
|
72
|
+
def initialize our_node, our_db, remote_node, remote_db, event_block
|
|
73
|
+
@our_node = our_node
|
|
74
|
+
@our_db = our_db
|
|
75
|
+
@remote_node = remote_node
|
|
76
|
+
@remote_db = remote_db
|
|
77
|
+
@name = "#{our_db}->#{remote_db}"
|
|
78
|
+
@keep_alive = true
|
|
79
|
+
@status = "NOTRUNNING"
|
|
80
|
+
@event_block = event_block
|
|
81
|
+
end
|
|
82
|
+
|
|
83
|
+
def kill
|
|
84
|
+
@keep_alive = false
|
|
85
|
+
end
|
|
86
|
+
|
|
87
|
+
def start
|
|
88
|
+
|
|
89
|
+
puts "Registering the replication connection for: #{@db_name}"
|
|
90
|
+
node_uri = URI.parse(our_node.url)
|
|
91
|
+
|
|
92
|
+
begin
|
|
93
|
+
puts "Connecting to #{node_uri.host}:#{node_uri.port}"
|
|
94
|
+
EM.connect node_uri.host, node_uri.port, Adhd::ReplicationNotifier, self
|
|
95
|
+
@status = "RUNNING"
|
|
96
|
+
rescue Exception => e
|
|
97
|
+
# TODO: tag nodes as unavailable if we blow up here
|
|
98
|
+
puts "PROBLEM: #{e.message}"
|
|
99
|
+
throw e
|
|
100
|
+
end
|
|
101
|
+
end
|
|
102
|
+
|
|
103
|
+
def event_handler data
|
|
104
|
+
#puts "Run a crazy sync on db: #{@db_name}"
|
|
105
|
+
#@sync_block.call(data)
|
|
106
|
+
@event_block.call(:rec, data)
|
|
107
|
+
end
|
|
108
|
+
|
|
109
|
+
def close_handler
|
|
110
|
+
puts "Closed abnormally: #{reason}"
|
|
111
|
+
@status = "NOTRUNNING"
|
|
112
|
+
end
|
|
113
|
+
|
|
114
|
+
def down_for_good(reason)
|
|
115
|
+
if reason
|
|
116
|
+
puts "Closed for good: #{reason}"
|
|
117
|
+
end
|
|
118
|
+
end
|
|
119
|
+
|
|
120
|
+
|
|
121
|
+
# Returns the truth value of the predicate
|
|
122
|
+
#
|
|
123
|
+
def keep_alive?
|
|
124
|
+
@keep_alive
|
|
125
|
+
end
|
|
126
|
+
|
|
127
|
+
def keep_alive_or_kill!
|
|
128
|
+
if ! keep_alive?
|
|
129
|
+
# Schedule this connection for close
|
|
130
|
+
connection_inside.close_connection_after_writing
|
|
131
|
+
@status = "NOTRUNNING"
|
|
132
|
+
end
|
|
133
|
+
end
|
|
134
|
+
|
|
135
|
+
def should_start?
|
|
136
|
+
!(@status == "RUNNING")
|
|
137
|
+
end
|
|
138
|
+
|
|
139
|
+
def is_closed?
|
|
140
|
+
(@status == "NOTRUNNING")
|
|
141
|
+
end
|
|
142
|
+
|
|
143
|
+
|
|
144
|
+
end
|
|
145
|
+
|
|
146
|
+
end
|
|
@@ -162,6 +162,7 @@ class ShardRange < CouchRest::ExtendedDocument
|
|
|
162
162
|
# SHARDSERVER = CouchRest.new("#{ARGV[1]}")
|
|
163
163
|
# SHARDSERVER.default_database = "#{ARGV[0]}_shard_db"
|
|
164
164
|
# use_database SHARDSERVER.default_database
|
|
165
|
+
unique_id :shard_db_name
|
|
165
166
|
|
|
166
167
|
property :range_start
|
|
167
168
|
property :range_end
|
data/lib/adhd/node_manager.rb
CHANGED
|
@@ -17,7 +17,7 @@ module Adhd
|
|
|
17
17
|
@couch_server = CouchRest.new("http://#{config.node_url}:#{config.couchdb_server_port}")
|
|
18
18
|
# @couch_server.default_database = "#{config.node_name}_node_db"
|
|
19
19
|
@couch_db = @couch_server.database!("#{config.node_name}_node_db") # CouchRest::Database.new(@couch_server, "#{config.node_name}_node_db")
|
|
20
|
-
sync_with_buddy_node if config.buddy_server_url && config.
|
|
20
|
+
sync_with_buddy_node if config.buddy_server_url && config.buddy_server_node_name
|
|
21
21
|
@our_node = initialize_node
|
|
22
22
|
build_node_admin_databases
|
|
23
23
|
set_as_management_node_if_necessary
|
|
@@ -35,10 +35,10 @@ module Adhd
|
|
|
35
35
|
def sync_with_buddy_node
|
|
36
36
|
begin
|
|
37
37
|
buddy_server = CouchRest.new("#{@config.buddy_server_url}")
|
|
38
|
-
buddy_db = buddy_server.database!(@config.
|
|
38
|
+
buddy_db = buddy_server.database!(@config.buddy_server_node_name + "_node_db")
|
|
39
39
|
@couch_db.replicate_from(buddy_db)
|
|
40
40
|
rescue
|
|
41
|
-
puts "Could not buddy up with node #{@config.
|
|
41
|
+
puts "Could not buddy up with node #{@config.buddy_server_node_name}"
|
|
42
42
|
end
|
|
43
43
|
end
|
|
44
44
|
|
data/lib/adhd/reactor.rb
CHANGED
|
@@ -61,7 +61,7 @@ module Adhd
|
|
|
61
61
|
@buffer += data # Add the data to the current buffer
|
|
62
62
|
updates = []
|
|
63
63
|
if @buffer =~ /(\{[^\n]+\}\n)/
|
|
64
|
-
updates += $~.to_a
|
|
64
|
+
updates += ($~.to_a)[1..-1]
|
|
65
65
|
# Trim the buffer to $_.end(0)
|
|
66
66
|
@buffer = @buffer[$~.end(0)..-1]
|
|
67
67
|
end
|
|
@@ -72,13 +72,8 @@ module Adhd
|
|
|
72
72
|
end
|
|
73
73
|
end
|
|
74
74
|
|
|
75
|
-
#def close_connection
|
|
76
|
-
# @conn_obj.close_handler(data)
|
|
77
|
-
#end
|
|
78
|
-
|
|
79
75
|
end
|
|
80
76
|
|
|
81
|
-
|
|
82
77
|
# Note: Some of manos's thoughts on how to manage our connections and events.
|
|
83
78
|
# We should build a class called connection_manager that we ask to build
|
|
84
79
|
# and listen to connections, as well as route events. Its job is to
|
|
@@ -154,19 +149,8 @@ module Adhd
|
|
|
154
149
|
end
|
|
155
150
|
|
|
156
151
|
end
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
def should_start?
|
|
161
|
-
!(@status == "RUNNING")
|
|
162
|
-
end
|
|
163
|
-
|
|
164
|
-
def is_closed?
|
|
165
|
-
(@status == "NOTRUNNING")
|
|
166
|
-
end
|
|
167
|
-
|
|
168
|
-
end
|
|
169
|
-
|
|
152
|
+
|
|
153
|
+
|
|
170
154
|
# Manage a bunch of connections for us
|
|
171
155
|
#
|
|
172
156
|
class ConnectionBank
|
data/test/unit/test_node.rb
CHANGED
|
@@ -4,7 +4,8 @@ require 'shoulda'
|
|
|
4
4
|
require 'couchrest'
|
|
5
5
|
require File.dirname(__FILE__) + '/../../lib/adhd/models/node_doc'
|
|
6
6
|
|
|
7
|
-
# A db that always
|
|
7
|
+
# A db that always pretends to copy a db
|
|
8
|
+
#
|
|
8
9
|
module FakeDb
|
|
9
10
|
def get_target
|
|
10
11
|
@target
|
|
@@ -20,6 +21,7 @@ module FakeDb
|
|
|
20
21
|
end
|
|
21
22
|
|
|
22
23
|
# A db that always throws a replication exception
|
|
24
|
+
#
|
|
23
25
|
module BlowDb
|
|
24
26
|
def get_target
|
|
25
27
|
@target
|
|
@@ -0,0 +1,133 @@
|
|
|
1
|
+
require 'rubygems'
|
|
2
|
+
require 'test/unit'
|
|
3
|
+
require 'shoulda'
|
|
4
|
+
require File.dirname(__FILE__) + '/../../lib/adhd/models/node_db'
|
|
5
|
+
|
|
6
|
+
class Node
|
|
7
|
+
def initialize event_list
|
|
8
|
+
@event_list = event_list
|
|
9
|
+
end
|
|
10
|
+
|
|
11
|
+
attr_accessor :status, :name, :is_management
|
|
12
|
+
|
|
13
|
+
def self.set_nodes node_list
|
|
14
|
+
@@nodes = node_list
|
|
15
|
+
end
|
|
16
|
+
|
|
17
|
+
def self.by_is_management
|
|
18
|
+
# Return a random set of nodes
|
|
19
|
+
@@nodes.select {|n| n.is_management && n.is_management > 0}
|
|
20
|
+
end
|
|
21
|
+
|
|
22
|
+
def get_node_db
|
|
23
|
+
@name
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
def replicate_to(local_db, other_node, remote_db)
|
|
27
|
+
false if other_node.status == "UNAVAILABLE" or other_node.name == name
|
|
28
|
+
@event_list << [:rep, local_db, remote_db]
|
|
29
|
+
true
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def replicate_from(local_db, other_node, remote_db)
|
|
33
|
+
false if other_node.status == "UNAVAILABLE" or other_node.name == name
|
|
34
|
+
@event_list << [:rep, remote_db, local_db]
|
|
35
|
+
true
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
class TestNodeDb < Test::Unit::TestCase
|
|
41
|
+
|
|
42
|
+
def get_random_node
|
|
43
|
+
random_log = @node_log.sort_by { rand }
|
|
44
|
+
target_node = random_log[0]
|
|
45
|
+
|
|
46
|
+
while target_node.status == "UNAVAILABLE"
|
|
47
|
+
random_log = random_log.sort_by { rand }
|
|
48
|
+
target_node = random_log[0]
|
|
49
|
+
end
|
|
50
|
+
target_node
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
|
|
54
|
+
context "A node database" do
|
|
55
|
+
setup do
|
|
56
|
+
# Make a CouchDB node_db and map the Node object there
|
|
57
|
+
@event_log = []
|
|
58
|
+
@node_log = []
|
|
59
|
+
100.times do |i|
|
|
60
|
+
n = Node.new @event_log
|
|
61
|
+
if i < 5
|
|
62
|
+
n.is_management = 3
|
|
63
|
+
end
|
|
64
|
+
n.name = i
|
|
65
|
+
if rand < 0.1
|
|
66
|
+
n.status = "UNAVAILABLE"
|
|
67
|
+
else
|
|
68
|
+
n.status = "RUNNING"
|
|
69
|
+
end
|
|
70
|
+
@node_log << n
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
Node.set_nodes @node_log
|
|
74
|
+
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
should "return some management nodes (even when fake)" do
|
|
78
|
+
assert Node.by_is_management.length > 0
|
|
79
|
+
end
|
|
80
|
+
|
|
81
|
+
should "sync to a management node (PROBABILISTIC)" do
|
|
82
|
+
old_log = @node_log.clone
|
|
83
|
+
target_node = nil
|
|
84
|
+
while !target_node or target_node.is_management
|
|
85
|
+
target_node = get_random_node
|
|
86
|
+
end
|
|
87
|
+
ndb = NodeDB.new(target_node)
|
|
88
|
+
|
|
89
|
+
ndb.sync
|
|
90
|
+
# Two events should fire up -- a sync to and from the server
|
|
91
|
+
assert @event_log.length >= 2
|
|
92
|
+
assert (@event_log.find {|log| old_log[log[2]].is_management}).length > 0
|
|
93
|
+
assert (@event_log.find {|log| old_log[log[1]].is_management}).length > 0
|
|
94
|
+
end
|
|
95
|
+
|
|
96
|
+
should "sync eventually to all" do
|
|
97
|
+
# Ok this is going to be insane
|
|
98
|
+
# We test that eventually all nodes get some info
|
|
99
|
+
first_node = nil
|
|
100
|
+
1000.times do |i|
|
|
101
|
+
target_node = get_random_node
|
|
102
|
+
first_node = target_node if !first_node
|
|
103
|
+
ndb = NodeDB.new(target_node)
|
|
104
|
+
ndb.sync
|
|
105
|
+
end
|
|
106
|
+
|
|
107
|
+
#Now we want to show that all running nodes go the information
|
|
108
|
+
# if they called sync after the node updated
|
|
109
|
+
|
|
110
|
+
tainted = {}
|
|
111
|
+
@node_log.each do |n|
|
|
112
|
+
tainted[n.name] = false
|
|
113
|
+
end
|
|
114
|
+
|
|
115
|
+
tainted[first_node.name] = true
|
|
116
|
+
@event_log.each do |ev|
|
|
117
|
+
from_node = ev[1]
|
|
118
|
+
to_node = ev[2]
|
|
119
|
+
tainted[to_node] |= tainted[from_node]
|
|
120
|
+
end
|
|
121
|
+
|
|
122
|
+
# @node_log.each do |n|
|
|
123
|
+
# puts "#{n.name}: #{tainted[n.name]} (#{n.status})"
|
|
124
|
+
# end
|
|
125
|
+
|
|
126
|
+
assert @node_log.all? {|n| (n.status == "UNAVAILABLE") or tainted[n.name]}
|
|
127
|
+
|
|
128
|
+
end
|
|
129
|
+
|
|
130
|
+
|
|
131
|
+
end
|
|
132
|
+
|
|
133
|
+
end
|
|
@@ -0,0 +1,112 @@
|
|
|
1
|
+
require 'rubygems'
|
|
2
|
+
require 'eventmachine'
|
|
3
|
+
require 'couchrest'
|
|
4
|
+
|
|
5
|
+
require 'test/unit'
|
|
6
|
+
require 'shoulda'
|
|
7
|
+
require File.dirname(__FILE__) + '/../../lib/adhd/models/replication_connection'
|
|
8
|
+
require File.dirname(__FILE__) + '/../../lib/adhd/models/node_doc'
|
|
9
|
+
|
|
10
|
+
class TestReplicationConnection < Test::Unit::TestCase
|
|
11
|
+
|
|
12
|
+
context "A couple of nodes" do
|
|
13
|
+
setup do
|
|
14
|
+
@node1 = Node.new
|
|
15
|
+
@node1.name = "test1"
|
|
16
|
+
@node1.url = "http://192.168.1.74:5984"
|
|
17
|
+
@node1_db = @node1.get_node_db
|
|
18
|
+
@node2 = Node.new
|
|
19
|
+
@node2.name = "test2"
|
|
20
|
+
@node2.url = "http://192.168.1.74:5984"
|
|
21
|
+
@node2_db = @node2.get_node_db
|
|
22
|
+
puts "#{@node1_db}"
|
|
23
|
+
puts "#{@node2_db}"
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
teardown do
|
|
27
|
+
@node1_db.delete!
|
|
28
|
+
@node2_db.delete!
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
should "build a replication connection" do
|
|
32
|
+
endconn = Proc.new do |ev, data|
|
|
33
|
+
assert ev == :rec
|
|
34
|
+
|
|
35
|
+
# Stop the event machine
|
|
36
|
+
EM::stop_event_loop()
|
|
37
|
+
end
|
|
38
|
+
conn = Adhd::ReplicationConnection.new @node1, @node1_db, @node2, @node2_db, endconn
|
|
39
|
+
@node2_db.save_doc(@node2)
|
|
40
|
+
@node2_db.save_doc(@node1)
|
|
41
|
+
|
|
42
|
+
assert @node2_db.get(@node2["_id"])
|
|
43
|
+
assert @node2_db.get(@node1["_id"])
|
|
44
|
+
EM::run {
|
|
45
|
+
conn.start
|
|
46
|
+
|
|
47
|
+
}
|
|
48
|
+
assert @node1_db.get(@node2["_id"])
|
|
49
|
+
assert @node1_db.get(@node1["_id"])
|
|
50
|
+
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
should "throw an exception if the second does not exist" do
|
|
54
|
+
endconn = Proc.new do |ev, data|
|
|
55
|
+
assert ev == :rec
|
|
56
|
+
|
|
57
|
+
# Stop the event machine
|
|
58
|
+
EM::stop_event_loop()
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
node3 = Node.new
|
|
62
|
+
node3.name = "test3"
|
|
63
|
+
node3.url = "http://192.168.1.200"
|
|
64
|
+
node3_db = node3.get_node_db
|
|
65
|
+
conn = Adhd::ReplicationConnection.new @node1, @node1_db, node3, node3_db, endconn
|
|
66
|
+
|
|
67
|
+
EM::run {
|
|
68
|
+
begin
|
|
69
|
+
conn.start
|
|
70
|
+
assert false
|
|
71
|
+
EM::stop_event_loop()
|
|
72
|
+
rescue
|
|
73
|
+
assert true
|
|
74
|
+
EM::stop_event_loop()
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
}
|
|
78
|
+
|
|
79
|
+
end
|
|
80
|
+
|
|
81
|
+
should "throw an exception if the first does not exist" do
|
|
82
|
+
endconn = Proc.new do |ev, data|
|
|
83
|
+
assert ev == :rec
|
|
84
|
+
|
|
85
|
+
# Stop the event machine
|
|
86
|
+
EM::stop_event_loop()
|
|
87
|
+
end
|
|
88
|
+
|
|
89
|
+
node3 = Node.new
|
|
90
|
+
node3.name = "test3"
|
|
91
|
+
node3.url = "http://192.168.1.200:9999"
|
|
92
|
+
node3_db = node3.get_node_db
|
|
93
|
+
conn = Adhd::ReplicationConnection.new node3, node3_db, @node2, @node2_db, endconn
|
|
94
|
+
|
|
95
|
+
EM::run {
|
|
96
|
+
begin
|
|
97
|
+
conn.start
|
|
98
|
+
assert false
|
|
99
|
+
EM::stop_event_loop()
|
|
100
|
+
rescue
|
|
101
|
+
assert true
|
|
102
|
+
EM::stop_event_loop()
|
|
103
|
+
end
|
|
104
|
+
|
|
105
|
+
}
|
|
106
|
+
|
|
107
|
+
end
|
|
108
|
+
|
|
109
|
+
|
|
110
|
+
end
|
|
111
|
+
|
|
112
|
+
end
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: adhd
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.1.
|
|
4
|
+
version: 0.1.2
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- dave.hrycyszyn@headlondon.com
|
|
@@ -9,7 +9,7 @@ autorequire:
|
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
11
|
|
|
12
|
-
date: 2009-12-
|
|
12
|
+
date: 2009-12-28 00:00:00 +00:00
|
|
13
13
|
default_executable:
|
|
14
14
|
dependencies:
|
|
15
15
|
- !ruby/object:Gem::Dependency
|
|
@@ -89,6 +89,7 @@ files:
|
|
|
89
89
|
- lib/adhd/models/content_shard.rb
|
|
90
90
|
- lib/adhd/models/node_db.rb
|
|
91
91
|
- lib/adhd/models/node_doc.rb
|
|
92
|
+
- lib/adhd/models/replication_connection.rb
|
|
92
93
|
- lib/adhd/models/shard_range.rb
|
|
93
94
|
- lib/adhd/node_manager.rb
|
|
94
95
|
- lib/adhd/reactor.rb
|
|
@@ -116,6 +117,8 @@ files:
|
|
|
116
117
|
- test/test_adhd.rb
|
|
117
118
|
- test/unit/test_content_doc.rb
|
|
118
119
|
- test/unit/test_node.rb
|
|
120
|
+
- test/unit/test_node_db.rb
|
|
121
|
+
- test/unit/test_replication_connection.rb
|
|
119
122
|
has_rdoc: true
|
|
120
123
|
homepage: http://github.com/futurechimp/adhd
|
|
121
124
|
licenses: []
|
|
@@ -146,6 +149,8 @@ specification_version: 3
|
|
|
146
149
|
summary: An experiment in distributed file replication using CouchDB
|
|
147
150
|
test_files:
|
|
148
151
|
- test/unit/test_content_doc.rb
|
|
152
|
+
- test/unit/test_replication_connection.rb
|
|
153
|
+
- test/unit/test_node_db.rb
|
|
149
154
|
- test/unit/test_node.rb
|
|
150
155
|
- test/helper.rb
|
|
151
156
|
- test/test_adhd.rb
|