tango-etl 0.0.1 → 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/{changelog.md → CHANGELOG.md} +5 -1
- data/{readme.md → README.md} +1 -1
- data/doc/Multidb/Balancer.html +202 -0
- data/doc/Multidb.html +184 -0
- data/doc/Tango/AbstractModel.html +608 -0
- data/doc/Tango/App.html +1334 -0
- data/doc/Tango/DatabaseLocker.html +541 -0
- data/doc/Tango/ETL/Dispatcher.html +465 -0
- data/doc/Tango/ETL/HandlerInterface.html +639 -0
- data/doc/Tango/ETL/OperatorInterface.html +494 -0
- data/doc/Tango/ETL.html +117 -0
- data/doc/Tango/Kernel.html +432 -0
- data/doc/Tango/LinkStack.html +737 -0
- data/doc/Tango/Resource/Buffer.html +565 -0
- data/doc/Tango/Resource/Cache.html +834 -0
- data/doc/Tango/Resource.html +117 -0
- data/doc/Tango.html +131 -0
- data/doc/_index.html +297 -0
- data/doc/class_list.html +54 -0
- data/doc/css/common.css +1 -0
- data/doc/css/full_list.css +57 -0
- data/doc/css/style.css +339 -0
- data/doc/file.README.html +80 -0
- data/doc/file_list.html +56 -0
- data/doc/frames.html +26 -0
- data/doc/index.html +80 -0
- data/doc/js/app.js +219 -0
- data/doc/js/full_list.js +178 -0
- data/doc/js/jquery.js +4 -0
- data/doc/method_list.html +365 -0
- data/doc/top-level-namespace.html +112 -0
- data/lib/tango/abstract_model.rb +1 -0
- data/lib/tango/app.rb +4 -7
- data/lib/tango/database_locker.rb +23 -2
- data/lib/tango/etl/handler_interface.rb +12 -6
- data/lib/tango/resource/buffer.rb +7 -7
- data/lib/tango/resource/cache.rb +18 -13
- data/lib/tango/version.rb +1 -1
- data/test/support/lib/simple_handler.rb +5 -14
- data/test/unit/etl/test_dispatcher.rb +21 -5
- metadata +33 -6
- data/config/app.yml.sample +0 -6
- data/config/database.yml.sample +0 -27
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA1:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 4485ff984a06b8afb8a7def1067872a966abef2f
|
|
4
|
+
data.tar.gz: 7cabea1f5d96f002d30a6b38235de3b4530112ac
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 8d5ac4af5a8c2dc5e6e73d83f9f3e06eb8d4e950a76a00236960328fc6b6d62383649e77b9d16d64691883816687031d1050ea1b7da19d7f9700448c445a4a01
|
|
7
|
+
data.tar.gz: 9c4cfbddaf1edcd1da191f73b0c997c03f0e9d546dbdc82a09de0405b2c02c0487efa69cb2ee6f15b13df1407ef4e384596c1d3f6a6ff31ac75eec94df99dafa
|
data/{readme.md → README.md}
RENAMED
|
@@ -1,3 +1,3 @@
|
|
|
1
1
|
# Tango
|
|
2
2
|
|
|
3
|
-
In short: **Tango is ETL framework**. It's job is to scrap content from target website, `extract` necessary data, `transform` it to desire format and `load` it to a database.
|
|
3
|
+
In short: **Tango is ETL framework**. It's job is to scrap content from target website, `extract` necessary data, `transform` it to desire format and `load` it to a database.
|
|
@@ -0,0 +1,202 @@
|
|
|
1
|
+
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
|
|
2
|
+
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
|
3
|
+
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
|
|
4
|
+
<head>
|
|
5
|
+
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
|
|
6
|
+
<title>
|
|
7
|
+
Class: Multidb::Balancer
|
|
8
|
+
|
|
9
|
+
— Documentation by YARD 0.8.7.4
|
|
10
|
+
|
|
11
|
+
</title>
|
|
12
|
+
|
|
13
|
+
<link rel="stylesheet" href="../css/style.css" type="text/css" charset="utf-8" />
|
|
14
|
+
|
|
15
|
+
<link rel="stylesheet" href="../css/common.css" type="text/css" charset="utf-8" />
|
|
16
|
+
|
|
17
|
+
<script type="text/javascript" charset="utf-8">
|
|
18
|
+
hasFrames = window.top.frames.main ? true : false;
|
|
19
|
+
relpath = '../';
|
|
20
|
+
framesUrl = "../frames.html#!Multidb/Balancer.html";
|
|
21
|
+
</script>
|
|
22
|
+
|
|
23
|
+
|
|
24
|
+
<script type="text/javascript" charset="utf-8" src="../js/jquery.js"></script>
|
|
25
|
+
|
|
26
|
+
<script type="text/javascript" charset="utf-8" src="../js/app.js"></script>
|
|
27
|
+
|
|
28
|
+
|
|
29
|
+
</head>
|
|
30
|
+
<body>
|
|
31
|
+
<div id="header">
|
|
32
|
+
<div id="menu">
|
|
33
|
+
|
|
34
|
+
<a href="../_index.html">Index (B)</a> »
|
|
35
|
+
<span class='title'><span class='object_link'><a href="../Multidb.html" title="Multidb (module)">Multidb</a></span></span>
|
|
36
|
+
»
|
|
37
|
+
<span class="title">Balancer</span>
|
|
38
|
+
|
|
39
|
+
|
|
40
|
+
<div class="noframes"><span class="title">(</span><a href="." target="_top">no frames</a><span class="title">)</span></div>
|
|
41
|
+
</div>
|
|
42
|
+
|
|
43
|
+
<div id="search">
|
|
44
|
+
|
|
45
|
+
<a class="full_list_link" id="class_list_link"
|
|
46
|
+
href="../class_list.html">
|
|
47
|
+
Class List
|
|
48
|
+
</a>
|
|
49
|
+
|
|
50
|
+
<a class="full_list_link" id="method_list_link"
|
|
51
|
+
href="../method_list.html">
|
|
52
|
+
Method List
|
|
53
|
+
</a>
|
|
54
|
+
|
|
55
|
+
<a class="full_list_link" id="file_list_link"
|
|
56
|
+
href="../file_list.html">
|
|
57
|
+
File List
|
|
58
|
+
</a>
|
|
59
|
+
|
|
60
|
+
</div>
|
|
61
|
+
<div class="clear"></div>
|
|
62
|
+
</div>
|
|
63
|
+
|
|
64
|
+
<iframe id="search_frame"></iframe>
|
|
65
|
+
|
|
66
|
+
<div id="content"><h1>Class: Multidb::Balancer
|
|
67
|
+
|
|
68
|
+
|
|
69
|
+
|
|
70
|
+
</h1>
|
|
71
|
+
|
|
72
|
+
<dl class="box">
|
|
73
|
+
|
|
74
|
+
<dt class="r1">Inherits:</dt>
|
|
75
|
+
<dd class="r1">
|
|
76
|
+
<span class="inheritName">Object</span>
|
|
77
|
+
|
|
78
|
+
<ul class="fullTree">
|
|
79
|
+
<li>Object</li>
|
|
80
|
+
|
|
81
|
+
<li class="next">Multidb::Balancer</li>
|
|
82
|
+
|
|
83
|
+
</ul>
|
|
84
|
+
<a href="#" class="inheritanceTree">show all</a>
|
|
85
|
+
|
|
86
|
+
</dd>
|
|
87
|
+
|
|
88
|
+
|
|
89
|
+
|
|
90
|
+
|
|
91
|
+
|
|
92
|
+
|
|
93
|
+
|
|
94
|
+
|
|
95
|
+
|
|
96
|
+
<dt class="r2 last">Defined in:</dt>
|
|
97
|
+
<dd class="r2 last">lib/tango/multidb.rb</dd>
|
|
98
|
+
|
|
99
|
+
</dl>
|
|
100
|
+
<div class="clear"></div>
|
|
101
|
+
|
|
102
|
+
|
|
103
|
+
|
|
104
|
+
|
|
105
|
+
|
|
106
|
+
<h2>Instance Attribute Summary <small>(<a href="#" class="summary_toggle">collapse</a>)</small></h2>
|
|
107
|
+
<ul class="summary">
|
|
108
|
+
|
|
109
|
+
<li class="public ">
|
|
110
|
+
<span class="summary_signature">
|
|
111
|
+
|
|
112
|
+
<a href="#candidates-instance_method" title="#candidates (instance method)">- (Object) <strong>candidates</strong> </a>
|
|
113
|
+
|
|
114
|
+
|
|
115
|
+
|
|
116
|
+
</span>
|
|
117
|
+
|
|
118
|
+
|
|
119
|
+
|
|
120
|
+
|
|
121
|
+
<span class="note title readonly">readonly</span>
|
|
122
|
+
|
|
123
|
+
|
|
124
|
+
|
|
125
|
+
|
|
126
|
+
|
|
127
|
+
|
|
128
|
+
|
|
129
|
+
|
|
130
|
+
|
|
131
|
+
<span class="summary_desc"><div class='inline'>
|
|
132
|
+
<p>Returns the value of attribute candidates.</p>
|
|
133
|
+
</div></span>
|
|
134
|
+
|
|
135
|
+
</li>
|
|
136
|
+
|
|
137
|
+
|
|
138
|
+
</ul>
|
|
139
|
+
|
|
140
|
+
|
|
141
|
+
|
|
142
|
+
|
|
143
|
+
|
|
144
|
+
<div id="instance_attr_details" class="attr_details">
|
|
145
|
+
<h2>Instance Attribute Details</h2>
|
|
146
|
+
|
|
147
|
+
|
|
148
|
+
<span id=""></span>
|
|
149
|
+
<div class="method_details first">
|
|
150
|
+
<h3 class="signature first" id="candidates-instance_method">
|
|
151
|
+
|
|
152
|
+
- (<tt>Object</tt>) <strong>candidates</strong> <span class="extras">(readonly)</span>
|
|
153
|
+
|
|
154
|
+
|
|
155
|
+
|
|
156
|
+
|
|
157
|
+
|
|
158
|
+
</h3><div class="docstring">
|
|
159
|
+
<div class="discussion">
|
|
160
|
+
|
|
161
|
+
<p>Returns the value of attribute candidates</p>
|
|
162
|
+
|
|
163
|
+
|
|
164
|
+
</div>
|
|
165
|
+
</div>
|
|
166
|
+
<div class="tags">
|
|
167
|
+
|
|
168
|
+
|
|
169
|
+
</div><table class="source_code">
|
|
170
|
+
<tr>
|
|
171
|
+
<td>
|
|
172
|
+
<pre class="lines">
|
|
173
|
+
|
|
174
|
+
|
|
175
|
+
4
|
|
176
|
+
5
|
|
177
|
+
6</pre>
|
|
178
|
+
</td>
|
|
179
|
+
<td>
|
|
180
|
+
<pre class="code"><span class="info file"># File 'lib/tango/multidb.rb', line 4</span>
|
|
181
|
+
|
|
182
|
+
<span class='kw'>def</span> <span class='id identifier rubyid_candidates'>candidates</span>
|
|
183
|
+
<span class='ivar'>@candidates</span>
|
|
184
|
+
<span class='kw'>end</span></pre>
|
|
185
|
+
</td>
|
|
186
|
+
</tr>
|
|
187
|
+
</table>
|
|
188
|
+
</div>
|
|
189
|
+
|
|
190
|
+
</div>
|
|
191
|
+
|
|
192
|
+
|
|
193
|
+
</div>
|
|
194
|
+
|
|
195
|
+
<div id="footer">
|
|
196
|
+
Generated on Tue Apr 15 02:47:42 2014 by
|
|
197
|
+
<a href="http://yardoc.org" title="Yay! A Ruby Documentation Tool" target="_parent">yard</a>
|
|
198
|
+
0.8.7.4 (ruby-2.0.0).
|
|
199
|
+
</div>
|
|
200
|
+
|
|
201
|
+
</body>
|
|
202
|
+
</html>
|
data/doc/Multidb.html
ADDED
|
@@ -0,0 +1,184 @@
|
|
|
1
|
+
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
|
|
2
|
+
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
|
3
|
+
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
|
|
4
|
+
<head>
|
|
5
|
+
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
|
|
6
|
+
<title>
|
|
7
|
+
Module: Multidb
|
|
8
|
+
|
|
9
|
+
— Documentation by YARD 0.8.7.4
|
|
10
|
+
|
|
11
|
+
</title>
|
|
12
|
+
|
|
13
|
+
<link rel="stylesheet" href="css/style.css" type="text/css" charset="utf-8" />
|
|
14
|
+
|
|
15
|
+
<link rel="stylesheet" href="css/common.css" type="text/css" charset="utf-8" />
|
|
16
|
+
|
|
17
|
+
<script type="text/javascript" charset="utf-8">
|
|
18
|
+
hasFrames = window.top.frames.main ? true : false;
|
|
19
|
+
relpath = '';
|
|
20
|
+
framesUrl = "frames.html#!Multidb.html";
|
|
21
|
+
</script>
|
|
22
|
+
|
|
23
|
+
|
|
24
|
+
<script type="text/javascript" charset="utf-8" src="js/jquery.js"></script>
|
|
25
|
+
|
|
26
|
+
<script type="text/javascript" charset="utf-8" src="js/app.js"></script>
|
|
27
|
+
|
|
28
|
+
|
|
29
|
+
</head>
|
|
30
|
+
<body>
|
|
31
|
+
<div id="header">
|
|
32
|
+
<div id="menu">
|
|
33
|
+
|
|
34
|
+
<a href="_index.html">Index (M)</a> »
|
|
35
|
+
|
|
36
|
+
|
|
37
|
+
<span class="title">Multidb</span>
|
|
38
|
+
|
|
39
|
+
|
|
40
|
+
<div class="noframes"><span class="title">(</span><a href="." target="_top">no frames</a><span class="title">)</span></div>
|
|
41
|
+
</div>
|
|
42
|
+
|
|
43
|
+
<div id="search">
|
|
44
|
+
|
|
45
|
+
<a class="full_list_link" id="class_list_link"
|
|
46
|
+
href="class_list.html">
|
|
47
|
+
Class List
|
|
48
|
+
</a>
|
|
49
|
+
|
|
50
|
+
<a class="full_list_link" id="method_list_link"
|
|
51
|
+
href="method_list.html">
|
|
52
|
+
Method List
|
|
53
|
+
</a>
|
|
54
|
+
|
|
55
|
+
<a class="full_list_link" id="file_list_link"
|
|
56
|
+
href="file_list.html">
|
|
57
|
+
File List
|
|
58
|
+
</a>
|
|
59
|
+
|
|
60
|
+
</div>
|
|
61
|
+
<div class="clear"></div>
|
|
62
|
+
</div>
|
|
63
|
+
|
|
64
|
+
<iframe id="search_frame"></iframe>
|
|
65
|
+
|
|
66
|
+
<div id="content"><h1>Module: Multidb
|
|
67
|
+
|
|
68
|
+
|
|
69
|
+
|
|
70
|
+
</h1>
|
|
71
|
+
|
|
72
|
+
<dl class="box">
|
|
73
|
+
|
|
74
|
+
|
|
75
|
+
|
|
76
|
+
|
|
77
|
+
|
|
78
|
+
|
|
79
|
+
|
|
80
|
+
|
|
81
|
+
<dt class="r1 last">Defined in:</dt>
|
|
82
|
+
<dd class="r1 last">lib/tango/multidb.rb</dd>
|
|
83
|
+
|
|
84
|
+
</dl>
|
|
85
|
+
<div class="clear"></div>
|
|
86
|
+
|
|
87
|
+
<h2>Defined Under Namespace</h2>
|
|
88
|
+
<p class="children">
|
|
89
|
+
|
|
90
|
+
|
|
91
|
+
|
|
92
|
+
|
|
93
|
+
<strong class="classes">Classes:</strong> <span class='object_link'><a href="Multidb/Balancer.html" title="Multidb::Balancer (class)">Balancer</a></span>
|
|
94
|
+
|
|
95
|
+
|
|
96
|
+
</p>
|
|
97
|
+
|
|
98
|
+
|
|
99
|
+
|
|
100
|
+
|
|
101
|
+
|
|
102
|
+
|
|
103
|
+
|
|
104
|
+
|
|
105
|
+
<h2>
|
|
106
|
+
Class Method Summary
|
|
107
|
+
<small>(<a href="#" class="summary_toggle">collapse</a>)</small>
|
|
108
|
+
</h2>
|
|
109
|
+
|
|
110
|
+
<ul class="summary">
|
|
111
|
+
|
|
112
|
+
<li class="public ">
|
|
113
|
+
<span class="summary_signature">
|
|
114
|
+
|
|
115
|
+
<a href="#databases-class_method" title="databases (class method)">+ (Object) <strong>databases</strong> </a>
|
|
116
|
+
|
|
117
|
+
|
|
118
|
+
|
|
119
|
+
</span>
|
|
120
|
+
|
|
121
|
+
|
|
122
|
+
|
|
123
|
+
|
|
124
|
+
|
|
125
|
+
|
|
126
|
+
|
|
127
|
+
|
|
128
|
+
|
|
129
|
+
<span class="summary_desc"><div class='inline'></div></span>
|
|
130
|
+
|
|
131
|
+
</li>
|
|
132
|
+
|
|
133
|
+
|
|
134
|
+
</ul>
|
|
135
|
+
|
|
136
|
+
|
|
137
|
+
|
|
138
|
+
|
|
139
|
+
<div id="class_method_details" class="method_details_list">
|
|
140
|
+
<h2>Class Method Details</h2>
|
|
141
|
+
|
|
142
|
+
|
|
143
|
+
<div class="method_details first">
|
|
144
|
+
<h3 class="signature first" id="databases-class_method">
|
|
145
|
+
|
|
146
|
+
+ (<tt>Object</tt>) <strong>databases</strong>
|
|
147
|
+
|
|
148
|
+
|
|
149
|
+
|
|
150
|
+
|
|
151
|
+
|
|
152
|
+
</h3><table class="source_code">
|
|
153
|
+
<tr>
|
|
154
|
+
<td>
|
|
155
|
+
<pre class="lines">
|
|
156
|
+
|
|
157
|
+
|
|
158
|
+
7
|
|
159
|
+
8
|
|
160
|
+
9</pre>
|
|
161
|
+
</td>
|
|
162
|
+
<td>
|
|
163
|
+
<pre class="code"><span class="info file"># File 'lib/tango/multidb.rb', line 7</span>
|
|
164
|
+
|
|
165
|
+
<span class='kw'>def</span> <span class='kw'>self</span><span class='period'>.</span><span class='id identifier rubyid_databases'>databases</span>
|
|
166
|
+
<span class='id identifier rubyid_balancer'>balancer</span><span class='period'>.</span><span class='id identifier rubyid_candidates'>candidates</span><span class='period'>.</span><span class='id identifier rubyid_keys'>keys</span>
|
|
167
|
+
<span class='kw'>end</span></pre>
|
|
168
|
+
</td>
|
|
169
|
+
</tr>
|
|
170
|
+
</table>
|
|
171
|
+
</div>
|
|
172
|
+
|
|
173
|
+
</div>
|
|
174
|
+
|
|
175
|
+
</div>
|
|
176
|
+
|
|
177
|
+
<div id="footer">
|
|
178
|
+
Generated on Tue Apr 15 02:47:42 2014 by
|
|
179
|
+
<a href="http://yardoc.org" title="Yay! A Ruby Documentation Tool" target="_parent">yard</a>
|
|
180
|
+
0.8.7.4 (ruby-2.0.0).
|
|
181
|
+
</div>
|
|
182
|
+
|
|
183
|
+
</body>
|
|
184
|
+
</html>
|