active_column 0.0.1 → 0.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/README.md +53 -10
- data/active_column.gemspec +3 -3
- data/lib/active_column/base.rb +10 -11
- data/lib/active_column/version.rb +1 -1
- data/spec/support/aggregating_tweet.rb +1 -1
- data/spec/support/compound_key.rb +3 -1
- data/spec/support/simple_key.rb +1 -1
- data/spec/support/tweet.rb +1 -1
- data/spec/support/tweet_dm.rb +2 -1
- metadata +5 -6
- data/README.html +0 -156
data/README.md
CHANGED
@@ -68,14 +68,15 @@ development and testing.
|
|
68
68
|
### Saving data
|
69
69
|
|
70
70
|
To make a model in to an ActiveColumn model, just extend ActiveColumn::Base, and provide two pieces of information:
|
71
|
-
|
72
|
-
|
71
|
+
|
72
|
+
- Column Family
|
73
|
+
- Function(s) to generate keys for your rows of data
|
73
74
|
|
74
75
|
The most basic form of using ActiveColumn looks like this:
|
75
76
|
<pre>
|
76
77
|
class Tweet < ActiveColumn::Base
|
77
78
|
column_family :tweets
|
78
|
-
|
79
|
+
key :user_id
|
79
80
|
end
|
80
81
|
</pre>
|
81
82
|
|
@@ -97,7 +98,7 @@ by telling it the name of a function to use to generate the keys during a save.
|
|
97
98
|
<pre>
|
98
99
|
class Tweet < ActiveColumn::Base
|
99
100
|
column_family :tweets
|
100
|
-
|
101
|
+
key :user_id, :values => :generate_user_keys
|
101
102
|
|
102
103
|
def generate_user_keys
|
103
104
|
[ attributes[:user_id], 'all']
|
@@ -117,7 +118,8 @@ and looks like this:
|
|
117
118
|
<pre>
|
118
119
|
class TweetDM < ActiveColumn::Base
|
119
120
|
column_family :tweet_dms
|
120
|
-
|
121
|
+
key :user_id, :values => :generate_user_keys
|
122
|
+
key :recipient_id, :values => :recipient_ids
|
121
123
|
|
122
124
|
def generate_user_keys
|
123
125
|
[ attributes[:user_id], 'all ]
|
@@ -132,10 +134,11 @@ dm = TweetDM.new( :user_id => 'mwynholds', :recipient_ids => [ 'fsinatra', 'dmar
|
|
132
134
|
</pre>
|
133
135
|
|
134
136
|
This tweet direct message will saved to four different rows in the "tweet_dms" column family, under these keys:
|
135
|
-
|
136
|
-
|
137
|
-
|
138
|
-
|
137
|
+
|
138
|
+
- mwynholds:fsinatra
|
139
|
+
- mwynholds:dmartin
|
140
|
+
- all:fsinatra
|
141
|
+
- all:dmartin
|
139
142
|
|
140
143
|
Now my app can pretty easily figure find all DMs I sent to Old Blue Eyes, or to Dino, and it can also easily find all
|
141
144
|
DMs sent from *anyone* to Frank or Dino.
|
@@ -147,4 +150,44 @@ are ordered is necessary to keep the compounds keys canonical (ie: deterministic
|
|
147
150
|
|
148
151
|
### Finding data
|
149
152
|
|
150
|
-
|
153
|
+
Ok, congratulations - now you have a bunch of fantastic data in Cassandra. How do you get it out? ActiveColumn can
|
154
|
+
help you here too.
|
155
|
+
|
156
|
+
Here is how you look up data that have a simple key:
|
157
|
+
|
158
|
+
<pre>
|
159
|
+
tweets = Tweet.find( 'mwynholds', :reversed => true, :count => 3 )
|
160
|
+
</pre>
|
161
|
+
|
162
|
+
This code will find the last 10 tweets for the 'mwynholds' user in reverse order. It comes back as a hash of arrays,
|
163
|
+
and would looks like this if represented in JSON:
|
164
|
+
|
165
|
+
<pre>
|
166
|
+
{
|
167
|
+
'mwynholds': [ { 'user_id': 'mwynholds', 'message': 'I\'m going to bed now' },
|
168
|
+
{ 'user_id': 'mwynholds', 'message': 'It\'s lunch time' },
|
169
|
+
{ 'user_id': 'mwynholds', 'message': 'Just woke up' } ]
|
170
|
+
}
|
171
|
+
</pre>
|
172
|
+
|
173
|
+
Here are some other examples and their return values:
|
174
|
+
|
175
|
+
<pre>
|
176
|
+
Tweet.find( [ 'mwynholds', 'all' ], :count => 2 )
|
177
|
+
|
178
|
+
{
|
179
|
+
'mwynholds': [ { 'user_id': 'mwynholds', 'message': 'Good morning' },
|
180
|
+
{ 'user_id': 'mwynholds', 'message': 'Good afternoon' } ],
|
181
|
+
'all': [ { 'user_id': 'mwynholds', 'message': 'Good morning' },
|
182
|
+
'user_id': 'bmurray', 'message': 'Who ya gonna call!' } ]
|
183
|
+
}
|
184
|
+
</pre>
|
185
|
+
|
186
|
+
<pre>
|
187
|
+
Tweet.find( { 'user_id' => 'all', 'recipient_id' => [ 'fsinatra', 'dmartin' ] }, :reversed => true, :count => 1 )
|
188
|
+
|
189
|
+
{
|
190
|
+
'all:fsinatra' => [ { 'user_id': 'mwynholds', 'recipient_ids' => [ 'fsinatra', 'dmartin' ], 'message' => 'Here we come Vegas!' } ],
|
191
|
+
'all:dmartin' => [ { 'user_id': 'fsinatra', 'recipient_ids' => [ 'dmartin' ], 'message' => 'Vegas was fun' } ]
|
192
|
+
}
|
193
|
+
</pre>
|
data/active_column.gemspec
CHANGED
@@ -8,7 +8,7 @@ Gem::Specification.new do |s|
|
|
8
8
|
s.platform = Gem::Platform::RUBY
|
9
9
|
s.authors = ["Michael Wynholds"]
|
10
10
|
s.email = ["mike@wynholds.com"]
|
11
|
-
s.homepage = "
|
11
|
+
s.homepage = "https://github.com/carbonfive/active_column"
|
12
12
|
s.summary = %q{Provides time line support for Cassandra}
|
13
13
|
s.description = %q{Provides time line support for Cassandra}
|
14
14
|
|
@@ -20,7 +20,7 @@ Gem::Specification.new do |s|
|
|
20
20
|
s.require_paths = ["lib"]
|
21
21
|
|
22
22
|
s.add_dependency 'simple_uuid'
|
23
|
-
|
24
|
-
|
23
|
+
s.add_dependency 'cassandra'
|
24
|
+
|
25
25
|
s.add_development_dependency 'rspec'
|
26
26
|
end
|
data/lib/active_column/base.rb
CHANGED
@@ -13,10 +13,9 @@ module ActiveColumn
|
|
13
13
|
@column_family = column_family
|
14
14
|
end
|
15
15
|
|
16
|
-
def self.
|
17
|
-
|
18
|
-
|
19
|
-
@keys = flattened.collect { |k| KeyConfig.new(k) }
|
16
|
+
def self.key(key, options = {})
|
17
|
+
@keys ||= []
|
18
|
+
@keys << KeyConfig.new(key, options)
|
20
19
|
end
|
21
20
|
|
22
21
|
def save()
|
@@ -46,6 +45,10 @@ module ActiveColumn
|
|
46
45
|
|
47
46
|
private
|
48
47
|
|
48
|
+
def self.keys
|
49
|
+
@keys
|
50
|
+
end
|
51
|
+
|
49
52
|
def get_keys(key_config)
|
50
53
|
key_config.func.nil? ? attributes[key_config.key] : self.send(key_config.func)
|
51
54
|
end
|
@@ -73,13 +76,9 @@ module ActiveColumn
|
|
73
76
|
class KeyConfig
|
74
77
|
attr_accessor :key, :func
|
75
78
|
|
76
|
-
def initialize(
|
77
|
-
|
78
|
-
|
79
|
-
@func = key_conf[@key]
|
80
|
-
else
|
81
|
-
@key = key_conf
|
82
|
-
end
|
79
|
+
def initialize(key, options)
|
80
|
+
@key = key
|
81
|
+
@func = options[:values]
|
83
82
|
end
|
84
83
|
|
85
84
|
def to_s
|
data/spec/support/simple_key.rb
CHANGED
data/spec/support/tweet.rb
CHANGED
data/spec/support/tweet_dm.rb
CHANGED
@@ -1,7 +1,8 @@
|
|
1
1
|
class TweetDM < ActiveColumn::Base
|
2
2
|
|
3
3
|
column_family :tweet_dms
|
4
|
-
|
4
|
+
key :user_id, :values => :user_keys
|
5
|
+
key :recipient_id, :values => :recipient_keys
|
5
6
|
|
6
7
|
def user_keys
|
7
8
|
[ attributes[:user_id], 'all' ]
|
metadata
CHANGED
@@ -5,8 +5,8 @@ version: !ruby/object:Gem::Version
|
|
5
5
|
segments:
|
6
6
|
- 0
|
7
7
|
- 0
|
8
|
-
-
|
9
|
-
version: 0.0.
|
8
|
+
- 2
|
9
|
+
version: 0.0.2
|
10
10
|
platform: ruby
|
11
11
|
authors:
|
12
12
|
- Michael Wynholds
|
@@ -14,7 +14,7 @@ autorequire:
|
|
14
14
|
bindir: bin
|
15
15
|
cert_chain: []
|
16
16
|
|
17
|
-
date: 2010-12-
|
17
|
+
date: 2010-12-13 00:00:00 -08:00
|
18
18
|
default_executable:
|
19
19
|
dependencies:
|
20
20
|
- !ruby/object:Gem::Dependency
|
@@ -41,7 +41,7 @@ dependencies:
|
|
41
41
|
segments:
|
42
42
|
- 0
|
43
43
|
version: "0"
|
44
|
-
type: :
|
44
|
+
type: :runtime
|
45
45
|
version_requirements: *id002
|
46
46
|
- !ruby/object:Gem::Dependency
|
47
47
|
name: rspec
|
@@ -70,7 +70,6 @@ files:
|
|
70
70
|
- .rvmrc
|
71
71
|
- Gemfile
|
72
72
|
- Gemfile.lock
|
73
|
-
- README.html
|
74
73
|
- README.md
|
75
74
|
- Rakefile
|
76
75
|
- active_column.gemspec
|
@@ -88,7 +87,7 @@ files:
|
|
88
87
|
- spec/support/tweet.rb
|
89
88
|
- spec/support/tweet_dm.rb
|
90
89
|
has_rdoc: true
|
91
|
-
homepage:
|
90
|
+
homepage: https://github.com/carbonfive/active_column
|
92
91
|
licenses: []
|
93
92
|
|
94
93
|
post_install_message:
|
data/README.html
DELETED
@@ -1,156 +0,0 @@
|
|
1
|
-
<h1>ActiveColumn</h1>
|
2
|
-
|
3
|
-
<p>ActiveColumn is a framework for saving and retrieving data from Cassandra in a "time line" model. It is loosely based
|
4
|
-
on concepts in ActiveRecord, but is adapted to saving data in which rows in Cassandra grow indefinitely over time, such
|
5
|
-
as in the oft-used Twitter example for Cassandra.</p>
|
6
|
-
|
7
|
-
<h2>Installation</h2>
|
8
|
-
|
9
|
-
<p>Add ActiveColumn to your Gemfile:</p>
|
10
|
-
|
11
|
-
<pre>
|
12
|
-
gem 'active_column'
|
13
|
-
</pre>
|
14
|
-
|
15
|
-
<p>Install with bundler:</p>
|
16
|
-
|
17
|
-
<pre>
|
18
|
-
bundle install
|
19
|
-
</pre>
|
20
|
-
|
21
|
-
<h2>Usage</h2>
|
22
|
-
|
23
|
-
<h3>Configuration</h3>
|
24
|
-
|
25
|
-
<p>ActiveColumn requires the <a href="https://github.com/fauna/cassandra">cassandra gem</a>. You must provide ActiveColumn with an
|
26
|
-
instance of a Cassandra object. You can do this very simply like this:</p>
|
27
|
-
|
28
|
-
<pre>
|
29
|
-
ActiveColumn.connection = Cassandra.new('my_keyspace', '127.0.0.1:9160')
|
30
|
-
</pre>
|
31
|
-
|
32
|
-
<p>However, in a real app this is not flexible enough, so I often create a cassandra.yml file and configure Cassandra in an
|
33
|
-
initializer.</p>
|
34
|
-
|
35
|
-
<p>config/cassandra.yml</p>
|
36
|
-
|
37
|
-
<pre>
|
38
|
-
test:
|
39
|
-
home: ":"
|
40
|
-
servers: "127.0.0.1:9160"
|
41
|
-
keyspace: "myapp_test"
|
42
|
-
thrift:
|
43
|
-
timeout: 3
|
44
|
-
retries: 2
|
45
|
-
|
46
|
-
development:
|
47
|
-
home: ":"
|
48
|
-
servers: "127.0.0.1:9160"
|
49
|
-
keyspace: "myapp_development"
|
50
|
-
thrift:
|
51
|
-
timeout: 3
|
52
|
-
retries: 2
|
53
|
-
</pre>
|
54
|
-
|
55
|
-
<p>config/initializers/cassandra.rb</p>
|
56
|
-
|
57
|
-
<pre>
|
58
|
-
config = YAML.load_file(Rails.root.join("config", "cassandra.yml"))[Rails.env]
|
59
|
-
$cassandra = Cassandra.new(config['keyspace'],
|
60
|
-
config['servers'],
|
61
|
-
config['thrift'])
|
62
|
-
|
63
|
-
ActiveColumn.connection = $cassandra
|
64
|
-
</pre>
|
65
|
-
|
66
|
-
<p>As you can see, I create a global $cassandra variable, which I use in my tests to validate data directly in Cassandra.</p>
|
67
|
-
|
68
|
-
<p>One other thing to note is that you obviously must have Cassandra installed and running! Please take a look at the
|
69
|
-
<a href="https://github.com/carbonfive/mama_cass">mama_cass gem</a> for a quick way to get up and running with Cassandra for
|
70
|
-
development and testing.</p>
|
71
|
-
|
72
|
-
<h3>Saving data</h3>
|
73
|
-
|
74
|
-
<p>To make a model in to an ActiveColumn model, just extend ActiveColumn::Base, and provide two pieces of information:
|
75
|
-
* Column Family
|
76
|
-
* Function(s) to generate keys for your rows of data</p>
|
77
|
-
|
78
|
-
<p>The most basic form of using ActiveColumn looks like this:</p>
|
79
|
-
|
80
|
-
<pre>
|
81
|
-
class Tweet < ActiveColumn::Base
|
82
|
-
column_family :tweets
|
83
|
-
keys :user_id
|
84
|
-
end
|
85
|
-
</pre>
|
86
|
-
|
87
|
-
<p>Then in your app you can create and save a tweet like this:</p>
|
88
|
-
|
89
|
-
<pre>
|
90
|
-
tweet = Tweet.new( :user_id => 'mwynholds', :message => "I'm going for a bike ride" )
|
91
|
-
tweet.save
|
92
|
-
</pre>
|
93
|
-
|
94
|
-
<p>When you run #save, ActiveColumn saves a new column in the "tweets" column family in the row with key "mwynholds". The
|
95
|
-
content of the row is the Tweet instance JSON-encoded.</p>
|
96
|
-
|
97
|
-
<p><em>Key Generator Functions</em></p>
|
98
|
-
|
99
|
-
<p>This is great, but quite often you want to save the content in multiple rows for the sake of speedy lookups. This is
|
100
|
-
basically de-normalizing data, and is extremely common in Cassandra data. ActiveColumn lets you do this quite easily
|
101
|
-
by telling it the name of a function to use to generate the keys during a save. It works like this:</p>
|
102
|
-
|
103
|
-
<pre>
|
104
|
-
class Tweet < ActiveColumn::Base
|
105
|
-
column_family :tweets
|
106
|
-
keys :user_id => :generate_user_keys
|
107
|
-
|
108
|
-
def generate_user_keys
|
109
|
-
[ attributes[:user_id], 'all']
|
110
|
-
end
|
111
|
-
end
|
112
|
-
</pre>
|
113
|
-
|
114
|
-
<p>The code to save the tweet is the same as the previous example, but now it saves the tweet in both the "mwynholds" row
|
115
|
-
and the "all" row. This way, you can pull out the last 20 of all tweets quite easily (assuming you needed to do this
|
116
|
-
in your app).</p>
|
117
|
-
|
118
|
-
<p><em>Compound Keys</em></p>
|
119
|
-
|
120
|
-
<p>In some cases you may want to have your rows keyed by multiple values. ActiveColumn supports compound keys,
|
121
|
-
and looks like this:</p>
|
122
|
-
|
123
|
-
<pre>
|
124
|
-
class TweetDM < ActiveColumn::Base
|
125
|
-
column_family :tweet_dms
|
126
|
-
keys [ { :user_id => :generate_user_keys }, { :recipient_id => :recipient_ids } ]
|
127
|
-
|
128
|
-
def generate_user_keys
|
129
|
-
[ attributes[:user_id], 'all ]
|
130
|
-
end
|
131
|
-
end
|
132
|
-
</pre>
|
133
|
-
|
134
|
-
<p>Now, when you create a new TweetDM, it might look like this:</p>
|
135
|
-
|
136
|
-
<pre>
|
137
|
-
dm = TweetDM.new( :user_id => 'mwynholds', :recipient_ids => [ 'fsinatra', 'dmartin' ], :message => "Let's go to Vegas" )
|
138
|
-
</pre>
|
139
|
-
|
140
|
-
<p>This tweet direct message will saved to four different rows in the "tweet_dms" column family, under these keys:
|
141
|
-
* mwynholds:fsinatra
|
142
|
-
* mwynholds:dmartin
|
143
|
-
* all:fsinatra
|
144
|
-
* all:dmartin</p>
|
145
|
-
|
146
|
-
<p>Now my app can pretty easily figure find all DMs I sent to Old Blue Eyes, or to Dino, and it can also easily find all
|
147
|
-
DMs sent from <em>anyone</em> to Frank or Dino.</p>
|
148
|
-
|
149
|
-
<p>One thing to note about the TweetDM class above is that the "keys" configuration at the top looks a little uglier than
|
150
|
-
before. If you have a compound key and any of the keys have custom key generators, you need to pass in an array of
|
151
|
-
single-element hashes. This is in place to support Ruby 1.8, which does not have ordered hashes. Making sure the keys
|
152
|
-
are ordered is necessary to keep the compounds keys canonical (ie: deterministic).</p>
|
153
|
-
|
154
|
-
<h3>Finding data</h3>
|
155
|
-
|
156
|
-
<p>Working on this...</p>
|