mcmire-cassandra 0.12.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/CHANGELOG +108 -0
- data/LICENSE +202 -0
- data/Manifest +63 -0
- data/README.md +352 -0
- data/Rakefile +169 -0
- data/bin/cassandra_helper +16 -0
- data/conf/0.6/cassandra.in.sh +47 -0
- data/conf/0.6/log4j.properties +38 -0
- data/conf/0.6/schema.json +57 -0
- data/conf/0.6/storage-conf.xml +352 -0
- data/conf/0.7/cassandra.in.sh +46 -0
- data/conf/0.7/cassandra.yaml +336 -0
- data/conf/0.7/log4j-server.properties +41 -0
- data/conf/0.7/schema.json +57 -0
- data/conf/0.7/schema.txt +45 -0
- data/conf/0.8/cassandra.in.sh +41 -0
- data/conf/0.8/cassandra.yaml +61 -0
- data/conf/0.8/log4j-server.properties +40 -0
- data/conf/0.8/schema.json +66 -0
- data/conf/0.8/schema.txt +51 -0
- data/lib/cassandra/0.6/cassandra.rb +113 -0
- data/lib/cassandra/0.6/columns.rb +78 -0
- data/lib/cassandra/0.6/protocol.rb +90 -0
- data/lib/cassandra/0.6.rb +7 -0
- data/lib/cassandra/0.7/cassandra.rb +2 -0
- data/lib/cassandra/0.7/columns.rb +4 -0
- data/lib/cassandra/0.7/protocol.rb +5 -0
- data/lib/cassandra/0.7.rb +7 -0
- data/lib/cassandra/0.8/cassandra.rb +10 -0
- data/lib/cassandra/0.8/columns.rb +4 -0
- data/lib/cassandra/0.8/protocol.rb +21 -0
- data/lib/cassandra/0.8.rb +7 -0
- data/lib/cassandra/array.rb +8 -0
- data/lib/cassandra/cassandra.rb +1070 -0
- data/lib/cassandra/column_family.rb +3 -0
- data/lib/cassandra/columns.rb +144 -0
- data/lib/cassandra/comparable.rb +28 -0
- data/lib/cassandra/constants.rb +11 -0
- data/lib/cassandra/debug.rb +9 -0
- data/lib/cassandra/helpers.rb +41 -0
- data/lib/cassandra/keyspace.rb +3 -0
- data/lib/cassandra/long.rb +58 -0
- data/lib/cassandra/mock.rb +511 -0
- data/lib/cassandra/ordered_hash.rb +192 -0
- data/lib/cassandra/protocol.rb +120 -0
- data/lib/cassandra/time.rb +11 -0
- data/lib/cassandra.rb +38 -0
- data/mcmire-cassandra.gemspec +43 -0
- data/test/cassandra_client_test.rb +20 -0
- data/test/cassandra_mock_test.rb +116 -0
- data/test/cassandra_test.rb +863 -0
- data/test/comparable_types_test.rb +45 -0
- data/test/eventmachine_test.rb +42 -0
- data/test/ordered_hash_test.rb +386 -0
- data/test/test_helper.rb +15 -0
- data/vendor/0.6/gen-rb/cassandra.rb +1481 -0
- data/vendor/0.6/gen-rb/cassandra_constants.rb +12 -0
- data/vendor/0.6/gen-rb/cassandra_types.rb +482 -0
- data/vendor/0.7/gen-rb/cassandra.rb +1936 -0
- data/vendor/0.7/gen-rb/cassandra_constants.rb +12 -0
- data/vendor/0.7/gen-rb/cassandra_types.rb +681 -0
- data/vendor/0.8/gen-rb/cassandra.rb +2215 -0
- data/vendor/0.8/gen-rb/cassandra_constants.rb +12 -0
- data/vendor/0.8/gen-rb/cassandra_types.rb +824 -0
- metadata +200 -0
data/README.md
ADDED
@@ -0,0 +1,352 @@
|
|
1
|
+
# cassandra
|
2
|
+
A Ruby client for the Cassandra distributed database.
|
3
|
+
|
4
|
+
Supports 1.8.7, 1.9.2, and rubinius on Cassandra 0.6.13, 0.7.9, 0.8.6, 1.0.0-rc2.
|
5
|
+
|
6
|
+
## Getting Started
|
7
|
+
|
8
|
+
Here is a quick sample of the general use (more details in Read/Write
|
9
|
+
API below):
|
10
|
+
|
11
|
+
require 'cassandra'
|
12
|
+
client = Cassandra.new('Twitter', '127.0.0.1:9160')
|
13
|
+
client.insert(:Users, "5", {'screen_name' => "buttonscat"})
|
14
|
+
|
15
|
+
## License
|
16
|
+
|
17
|
+
Copyright 2009-2011 Twitter, Inc. See included LICENSE file. Portions copyright 2004-2009 David Heinemeier Hansson, and used with permission.
|
18
|
+
|
19
|
+
## Cassandra Version
|
20
|
+
|
21
|
+
The Cassandra project is under very active development, and as such
|
22
|
+
there are a few different versions that you may need to use this gem
|
23
|
+
with. We have set up an easy sure fire mechanism for selecting the
|
24
|
+
specific version that you are connecting to while requiring the gem.
|
25
|
+
|
26
|
+
#### Require Method
|
27
|
+
The default version is the currently stable release of cassandra. (0.8
|
28
|
+
at this time.)
|
29
|
+
|
30
|
+
To use the default version simply use a normal require:
|
31
|
+
|
32
|
+
require 'cassandra'
|
33
|
+
|
34
|
+
To use a specific version (1.0 in this example) you would use a
|
35
|
+
slightly differently formatted require:
|
36
|
+
|
37
|
+
require 'cassandra/1.0'
|
38
|
+
|
39
|
+
#### Environment Variable Method
|
40
|
+
These mechanisms work well when you are using the cassandra gem in your
|
41
|
+
own projects or irb, but if you would rather not hard code your app to a
|
42
|
+
specific version you can always specify an environment variable with the
|
43
|
+
version you are using:
|
44
|
+
|
45
|
+
export CASSANDRA_VERSION=0.8
|
46
|
+
|
47
|
+
Then you would use the default require as listed above:
|
48
|
+
|
49
|
+
require 'cassandra'
|
50
|
+
|
51
|
+
## Read/Write API Method Reference
|
52
|
+
|
53
|
+
### insert
|
54
|
+
|
55
|
+
* column\_family - The column\_family that you are inserting into.
|
56
|
+
* key - The row key to insert.
|
57
|
+
* hash - The columns or super columns to insert.
|
58
|
+
* options - Valid options are:
|
59
|
+
* :timestamp - Uses the current time if none specified.
|
60
|
+
* :consistency - Uses the default write consistency if none specified.
|
61
|
+
* :ttl - If specified this is the number of seconds after the insert that this value will be available.
|
62
|
+
|
63
|
+
This is the main method used to insert rows into cassandra. If the
|
64
|
+
column\_family that you are inserting into is a SuperColumnFamily then
|
65
|
+
the hash passed in should be a nested hash, otherwise it should be a
|
66
|
+
flat hash.
|
67
|
+
|
68
|
+
This method can also be called while in batch mode. If in batch mode
|
69
|
+
then we queue up the mutations (an insert in this case) and pass them to
|
70
|
+
cassandra in a single batch at the end of the block.
|
71
|
+
|
72
|
+
Example:
|
73
|
+
|
74
|
+
@client.insert(:Statuses, key, {'body' => 'v', 'user' => 'v'})
|
75
|
+
|
76
|
+
columns = {@uuids[1] => 'v1', @uuids[2] => 'v2'}
|
77
|
+
@client.insert(:StatusRelationships, key, {'user_timelines' => columns})
|
78
|
+
|
79
|
+
|
80
|
+
### remove
|
81
|
+
|
82
|
+
* column\_family - The column\_family that you are working with.
|
83
|
+
* key - The row key to remove (or remove columns from).
|
84
|
+
* columns - Either a single super_column or a list of columns to remove.
|
85
|
+
* sub_columns - The list of sub\_columns to remove.
|
86
|
+
* options - Valid options are:
|
87
|
+
* :timestamp - Uses the current time if none specified.
|
88
|
+
* :consistency - Uses the default write consistency if none specified.
|
89
|
+
|
90
|
+
This method is used to delete (actually marking them as deleted with a
|
91
|
+
tombstone) rows, columns, or super columns depending on the parameters
|
92
|
+
passed. If only a key is passed the entire row will be marked as deleted.
|
93
|
+
If a column name is passed in that column will be deleted.
|
94
|
+
|
95
|
+
Example:
|
96
|
+
|
97
|
+
@client.insert(:Statuses, key, {'body' => 'v', 'subject' => 'v'})
|
98
|
+
|
99
|
+
@client.remove(:Statuses, key, 'body') # removes the 'body' column
|
100
|
+
@client.remove(:Statuses, key) # removes the row
|
101
|
+
|
102
|
+
### count\_columns
|
103
|
+
|
104
|
+
Count the columns for the provided parameters.
|
105
|
+
|
106
|
+
* column\_family - The column\_family that you are working with.
|
107
|
+
* key - The row key.
|
108
|
+
* columns - Either a single super_column or a list of columns.
|
109
|
+
* sub_columns - The list of sub\_columns to select.
|
110
|
+
* options - Valid options are:
|
111
|
+
* :start - The column name to start from.
|
112
|
+
* :stop - The column name to stop at.
|
113
|
+
* :count - The maximum count of columns to return. (By default cassandra will count up to 100 columns)
|
114
|
+
* :consistency - Uses the default read consistency if none specified.
|
115
|
+
|
116
|
+
Example:
|
117
|
+
|
118
|
+
@client.insert(:Statuses, key, {'body' => 'v1', 'user' => 'v2'})
|
119
|
+
@client.count_columns(:Statuses, key) # returns 2
|
120
|
+
|
121
|
+
### get
|
122
|
+
|
123
|
+
Return a hash (actually, a Cassandra::OrderedHash) or a single value
|
124
|
+
representing the element at the column_family:key:[column]:[sub_column]
|
125
|
+
path you request.
|
126
|
+
|
127
|
+
* column\_family - The column\_family that you are working with.
|
128
|
+
* key - The row key to select.
|
129
|
+
* columns - Either a single super\_column or a list of columns.
|
130
|
+
* sub\_columns - The list of sub\_columns to select.
|
131
|
+
* options - Valid options are:
|
132
|
+
* :count - The number of columns requested to be returned.
|
133
|
+
* :start - The starting value for selecting a range of columns.
|
134
|
+
* :finish - The final value for selecting a range of columns.
|
135
|
+
* :reversed - If set to true the results will be returned in
|
136
|
+
reverse order.
|
137
|
+
* :consistency - Uses the default read consistency if none specified.
|
138
|
+
|
139
|
+
Example:
|
140
|
+
|
141
|
+
@client.insert(:Users, key, {'body' => 'v', 'user' => 'v'})
|
142
|
+
@client.get(:Users, key)) # returns {'body' => 'v', 'user' => 'v'}
|
143
|
+
|
144
|
+
### multi\_get
|
145
|
+
|
146
|
+
Multi-key version of Cassandra#get.
|
147
|
+
|
148
|
+
This method allows you to select multiple rows with a single query.
|
149
|
+
If a key that is passed in doesn't exist an empty hash will be
|
150
|
+
returned.
|
151
|
+
|
152
|
+
Supports the same parameters as Cassandra#get.
|
153
|
+
|
154
|
+
* column_family - The column_family that you are working with.
|
155
|
+
* key - An array of keys to select.
|
156
|
+
* columns - Either a single super_column or a list of columns.
|
157
|
+
* sub_columns - The list of sub\_columns to select.
|
158
|
+
* options - Valid options are:
|
159
|
+
* :count - The number of columns requested to be returned.
|
160
|
+
* :start - The starting value for selecting a range of columns.
|
161
|
+
* :finish - The final value for selecting a range of columns.
|
162
|
+
* :reversed - If set to true the results will be returned in reverse order.
|
163
|
+
* :consistency - Uses the default read consistency if none specified.
|
164
|
+
|
165
|
+
Example:
|
166
|
+
|
167
|
+
@client.insert(:Users, '1', {'body' => 'v1', 'user' => 'v1'})
|
168
|
+
@client.insert(:Users, '2', {'body' => 'v2', 'user' => 'v2'})
|
169
|
+
|
170
|
+
expected = OrderedHash[
|
171
|
+
'1', {'body' => 'v1', 'user' => 'v1'},
|
172
|
+
'2', {'body' => 'v2', 'user' => 'v2'},
|
173
|
+
'bogus', {}
|
174
|
+
]
|
175
|
+
result = @client.multi_get(:Users, ['1', '2', 'bogus'])
|
176
|
+
|
177
|
+
### exists?
|
178
|
+
|
179
|
+
Return true if the column\_family:key:[column]:[sub\_column] path you
|
180
|
+
request exists.
|
181
|
+
|
182
|
+
If passed in only a row key it will query for any columns (limiting
|
183
|
+
to 1) for that row key. If a column is passed in it will query for
|
184
|
+
that specific column/super column.
|
185
|
+
|
186
|
+
This method will return true or false.
|
187
|
+
|
188
|
+
* column\_family - The column\_family that you are working with.
|
189
|
+
* key - The row key to check.
|
190
|
+
* columns - Either a single super\_column or a list of columns.
|
191
|
+
* sub\_columns - The list of sub\_columns to check.
|
192
|
+
* options - Valid options are:
|
193
|
+
* :consistency - Uses the default read consistency if none specified.
|
194
|
+
|
195
|
+
Example:
|
196
|
+
|
197
|
+
@client.insert(:Statuses, 'key', {'body' => 'v'})
|
198
|
+
@client.exists?(:Statuses, 'key') # returns true
|
199
|
+
@client.exists?(:Statuses, 'bogus') # returns false
|
200
|
+
@client.exists?(:Statuses, 'key', 'body') # returns true
|
201
|
+
@client.exists?(:Statuses, 'key', 'bogus') # returns false
|
202
|
+
|
203
|
+
### get\_range
|
204
|
+
Return an Cassandra::OrderedHash containing the columns specified for the given
|
205
|
+
range of keys in the column\_family you request.
|
206
|
+
|
207
|
+
This method is just a convenience wrapper around Cassandra#get_range_single
|
208
|
+
and Cassandra#get\_range\_batch. If :key\_size, :batch\_size, or a block
|
209
|
+
is passed in Cassandra#get\_range\_batch will be called. Otherwise
|
210
|
+
Cassandra#get\_range\_single will be used.
|
211
|
+
|
212
|
+
The start\_key and finish\_key parameters are only useful for iterating of all records
|
213
|
+
as is done in the Cassandra#each and Cassandra#each\_key methods if you are using the
|
214
|
+
RandomPartitioner.
|
215
|
+
|
216
|
+
If the table is partitioned with OrderPreservingPartitioner you may
|
217
|
+
use the start\_key and finish\_key params to select all records with
|
218
|
+
the same prefix value.
|
219
|
+
|
220
|
+
If a block is passed in we will yield the row key and columns for
|
221
|
+
each record returned.
|
222
|
+
|
223
|
+
Please note that Cassandra returns a row for each row that has existed in the
|
224
|
+
system since gc\_grace\_seconds. This is because deleted row keys are marked as
|
225
|
+
deleted, but left in the system until the cluster has had resonable time to replicate the deletion.
|
226
|
+
This function attempts to suppress deleted rows (actually any row returned without
|
227
|
+
columns is suppressed).
|
228
|
+
|
229
|
+
* column\_family - The column\_family that you are working with.
|
230
|
+
* options - Valid options are:
|
231
|
+
* :start\_key - The starting value for selecting a range of keys (only useful with OPP).
|
232
|
+
* :finish\_key - The final value for selecting a range of keys (only useful with OPP).
|
233
|
+
* :key\_count - The total number of keys to return from the query. (see note regarding deleted records)
|
234
|
+
* :batch\_size - The maximum number of keys to return per query. If specified will loop until :key\_count is obtained or all records have been returned.
|
235
|
+
* :columns - A list of columns to return.
|
236
|
+
* :count - The number of columns requested to be returned.
|
237
|
+
* :start - The starting value for selecting a range of columns.
|
238
|
+
* :finish - The final value for selecting a range of columns.
|
239
|
+
* :reversed - If set to true the results will be returned in reverse order.
|
240
|
+
* :consistency - Uses the default read consistency if none specified.
|
241
|
+
|
242
|
+
Example:
|
243
|
+
|
244
|
+
10.times do |i|
|
245
|
+
@client.insert(:Statuses, i.to_s, {'body' => '1'})
|
246
|
+
end
|
247
|
+
|
248
|
+
@client.get_range_keys(:Statuses, :key_count => 4)
|
249
|
+
|
250
|
+
# returns:
|
251
|
+
#{
|
252
|
+
# '0' => {'body' => '1'},
|
253
|
+
# '1' => {'body' => '1'},
|
254
|
+
# '2' => {'body' => '1'},
|
255
|
+
# '3' => {'body' => '1'}
|
256
|
+
#}
|
257
|
+
|
258
|
+
### count\_range
|
259
|
+
|
260
|
+
Return an Array containing all of the keys within a given range.
|
261
|
+
|
262
|
+
This method just calls Cassandra#get\_range and returns the
|
263
|
+
row keys for the records returned.
|
264
|
+
|
265
|
+
See Cassandra#get\_range for options.
|
266
|
+
|
267
|
+
### get\_range\_keys
|
268
|
+
|
269
|
+
Return an Array containing all of the keys within a given range.
|
270
|
+
|
271
|
+
This method just calls Cassandra#get\_range and returns the
|
272
|
+
row keys for the records returned.
|
273
|
+
|
274
|
+
See Cassandra#get\_range for options.
|
275
|
+
|
276
|
+
### each\_key
|
277
|
+
Iterate through each key within the given range parameters. This function can be
|
278
|
+
used to iterate over each key in the given column family.
|
279
|
+
|
280
|
+
This method just calls Cassandra#get\_range and yields each row key.
|
281
|
+
|
282
|
+
See Cassandra#get\_range for options.
|
283
|
+
|
284
|
+
Example:
|
285
|
+
10.times do |i|
|
286
|
+
@client.insert(:Statuses, k + i.to_s, {"body-#{i.to_s}" => 'v'})
|
287
|
+
end
|
288
|
+
|
289
|
+
@client.each_key(:Statuses) do |key|
|
290
|
+
print key
|
291
|
+
end
|
292
|
+
|
293
|
+
# returns 0123456789
|
294
|
+
|
295
|
+
### each
|
296
|
+
Iterate through each row within the given column\_family.
|
297
|
+
|
298
|
+
This method just calls Cassandra#get\_range and yields the key and
|
299
|
+
columns.
|
300
|
+
|
301
|
+
See Cassandra#get\_range for options.
|
302
|
+
|
303
|
+
### get\_index\_slices
|
304
|
+
This method is used to query a secondary index with a set of
|
305
|
+
provided search parameters
|
306
|
+
|
307
|
+
Please note that you can either specify a
|
308
|
+
CassandraThrift::IndexClause or an array of hashes with the
|
309
|
+
format as below.
|
310
|
+
|
311
|
+
* column\_family - The Column Family this operation will be run on.
|
312
|
+
* index\_clause - This can either be a CassandraThrift::IndexClause or an array of hashes with the following keys:
|
313
|
+
* :column\_name - Column to be compared
|
314
|
+
* :value - Value to compare against
|
315
|
+
* :comparison - Type of comparison to do.
|
316
|
+
* options
|
317
|
+
* :key\_count - Set maximum number of rows to return. (Only works if CassandraThrift::IndexClause is not passed in.)
|
318
|
+
* :key\_start - Set starting row key for search. (Only works if CassandraThrift::IndexClause is not passed in.)
|
319
|
+
* :consistency
|
320
|
+
|
321
|
+
Example:
|
322
|
+
|
323
|
+
@client.create_index('Twitter', 'Statuses', 'x', 'LongType')
|
324
|
+
|
325
|
+
@client.insert(:Statuses, 'row1', { 'x' => [0,10].pack("NN") })
|
326
|
+
|
327
|
+
(2..10).to_a.each do |i|
|
328
|
+
@twitter.insert(:Statuses, 'row' + i.to_s, { 'x' => [0,20].pack("NN"), 'non_indexed' => [i].pack('N*') })
|
329
|
+
end
|
330
|
+
|
331
|
+
@client.insert(:Statuses, 'row11', { 'x' => [0,30].pack("NN") })
|
332
|
+
|
333
|
+
expressions = [{:column_name => 'x', :value => [0,20].pack("NN"), :comparison => "=="}]
|
334
|
+
|
335
|
+
# verify multiples will be returned
|
336
|
+
@client.get_indexed_slices(:Statuses, expressions).length # returns 9
|
337
|
+
|
338
|
+
# verify that GT and LT queries perform properly
|
339
|
+
expressions = [
|
340
|
+
{ :column_name => 'x',
|
341
|
+
:value => [0,20].pack("NN"),
|
342
|
+
:comparison => "=="},
|
343
|
+
{ :column_name => 'non_indexed',
|
344
|
+
:value => [5].pack("N*"),
|
345
|
+
:comparison => ">"}
|
346
|
+
]
|
347
|
+
|
348
|
+
@client.get_indexed_slices(:Statuses, expressions).length # returns 5
|
349
|
+
|
350
|
+
## Reporting Problems
|
351
|
+
|
352
|
+
The Github issue tracker is [here](http://github.com/fauna/cassandra/issues). If you have problems with this library or Cassandra itself, please use the [cassandra-user mailing list](http://mail-archives.apache.org/mod_mbox/incubator-cassandra-user/).
|
data/Rakefile
ADDED
@@ -0,0 +1,169 @@
|
|
1
|
+
require 'fileutils'
|
2
|
+
|
3
|
+
unless ENV['FROM_BIN_CASSANDRA_HELPER']
|
4
|
+
require 'rubygems'
|
5
|
+
require 'echoe'
|
6
|
+
|
7
|
+
Echoe.new("mcmire-cassandra") do |p|
|
8
|
+
p.author = "Evan Weaver, Ryan King"
|
9
|
+
p.project = "fauna"
|
10
|
+
p.summary = "A Ruby client for the Cassandra distributed database."
|
11
|
+
p.rubygems_version = ">= 0.8"
|
12
|
+
p.dependencies = ['thrift_client >=0.7.0 <0.9', 'json', 'rake', 'simple_uuid ~>0.2.0']
|
13
|
+
p.ignore_pattern = /^(data|vendor\/cassandra|cassandra|vendor\/thrift|.*\.rbc)/
|
14
|
+
p.rdoc_pattern = /^(lib|bin|tasks|ext)|^README|^CHANGELOG|^TODO|^LICENSE|^COPYING$/
|
15
|
+
p.retain_gemspec = true
|
16
|
+
end
|
17
|
+
end
|
18
|
+
|
19
|
+
CassandraBinaries = {
|
20
|
+
'0.6' => 'http://archive.apache.org/dist/cassandra/0.6.13/apache-cassandra-0.6.13-bin.tar.gz',
|
21
|
+
'0.7' => 'http://archive.apache.org/dist/cassandra/0.7.9/apache-cassandra-0.7.9-bin.tar.gz',
|
22
|
+
'0.8' => 'http://archive.apache.org/dist/cassandra/0.8.7/apache-cassandra-0.8.7-bin.tar.gz',
|
23
|
+
'1.0' => 'http://archive.apache.org/dist/cassandra/1.0.6/apache-cassandra-1.0.6-bin.tar.gz'
|
24
|
+
}
|
25
|
+
|
26
|
+
CASSANDRA_HOME = ENV['CASSANDRA_HOME'] || "#{ENV['HOME']}/cassandra"
|
27
|
+
CASSANDRA_VERSION = ENV['CASSANDRA_VERSION'] || '0.8'
|
28
|
+
CASSANDRA_PIDFILE = ENV['CASSANDRA_PIDFILE'] || "#{CASSANDRA_HOME}/cassandra.pid"
|
29
|
+
|
30
|
+
def setup_cassandra_version(version = CASSANDRA_VERSION)
|
31
|
+
FileUtils.mkdir_p CASSANDRA_HOME
|
32
|
+
|
33
|
+
destination_directory = File.join(CASSANDRA_HOME, 'cassandra-' + CASSANDRA_VERSION)
|
34
|
+
|
35
|
+
unless File.exists?(File.join(destination_directory, 'bin','cassandra'))
|
36
|
+
download_source = CassandraBinaries[CASSANDRA_VERSION]
|
37
|
+
download_destination = File.join("/tmp", File.basename(download_source))
|
38
|
+
untar_directory = File.join(CASSANDRA_HOME, File.basename(download_source,'-bin.tar.gz'))
|
39
|
+
|
40
|
+
puts "downloading cassandra"
|
41
|
+
sh "curl -L -o #{download_destination} #{download_source}"
|
42
|
+
|
43
|
+
sh "tar xzf #{download_destination} -C #{CASSANDRA_HOME}"
|
44
|
+
sh "mv #{untar_directory} #{destination_directory}"
|
45
|
+
end
|
46
|
+
end
|
47
|
+
|
48
|
+
def setup_environment
|
49
|
+
env = ""
|
50
|
+
if !ENV["CASSANDRA_INCLUDE"]
|
51
|
+
env << "CASSANDRA_INCLUDE=#{File.expand_path(Dir.pwd)}/conf/#{CASSANDRA_VERSION}/cassandra.in.sh "
|
52
|
+
env << "CASSANDRA_HOME=#{CASSANDRA_HOME}/cassandra-#{CASSANDRA_VERSION} "
|
53
|
+
env << "CASSANDRA_CONF=#{File.expand_path(Dir.pwd)}/conf/#{CASSANDRA_VERSION}"
|
54
|
+
else
|
55
|
+
env << "CASSANDRA_INCLUDE=#{ENV['CASSANDRA_INCLUDE']} "
|
56
|
+
env << "CASSANDRA_HOME=#{ENV['CASSANDRA_HOME']} "
|
57
|
+
env << "CASSANDRA_CONF=#{ENV['CASSANDRA_CONF']}"
|
58
|
+
end
|
59
|
+
|
60
|
+
env
|
61
|
+
end
|
62
|
+
|
63
|
+
def running?(pid_file = nil)
|
64
|
+
pid_file ||= CASSANDRA_PIDFILE
|
65
|
+
|
66
|
+
if File.exists?(pid_file)
|
67
|
+
pid = File.new(pid_file).read.to_i
|
68
|
+
begin
|
69
|
+
Process.kill(0, pid)
|
70
|
+
return true
|
71
|
+
rescue
|
72
|
+
File.delete(pid_file)
|
73
|
+
end
|
74
|
+
end
|
75
|
+
|
76
|
+
false
|
77
|
+
end
|
78
|
+
|
79
|
+
namespace :cassandra do
|
80
|
+
desc "Start Cassandra"
|
81
|
+
task :start, [:daemonize] => :java do |t, args|
|
82
|
+
args.with_defaults(:daemonize => true)
|
83
|
+
|
84
|
+
setup_cassandra_version
|
85
|
+
|
86
|
+
env = setup_environment
|
87
|
+
|
88
|
+
Dir.chdir(File.join(CASSANDRA_HOME, "cassandra-#{CASSANDRA_VERSION}")) do
|
89
|
+
sh("env #{env} bin/cassandra #{'-f' unless args.daemonize} -p #{CASSANDRA_PIDFILE}")
|
90
|
+
end
|
91
|
+
end
|
92
|
+
|
93
|
+
desc "Stop Cassandra"
|
94
|
+
task :stop => :java do
|
95
|
+
setup_cassandra_version
|
96
|
+
env = setup_environment
|
97
|
+
sh("kill $(cat #{CASSANDRA_PIDFILE})")
|
98
|
+
end
|
99
|
+
end
|
100
|
+
|
101
|
+
desc "Start Cassandra"
|
102
|
+
task :cassandra => :java do
|
103
|
+
begin
|
104
|
+
Rake::Task["cassandra:start"].invoke(false)
|
105
|
+
rescue RuntimeError => e
|
106
|
+
raise e unless e.message =~ /Command failed with status \(130\)/ # handle keyboard interupt errors
|
107
|
+
end
|
108
|
+
end
|
109
|
+
|
110
|
+
desc "Run the Cassandra CLI"
|
111
|
+
task :cli do
|
112
|
+
Dir.chdir(File.join(CASSANDRA_HOME, "cassandra-#{CASSANDRA_VERSION}")) do
|
113
|
+
sh("bin/cassandra-cli -host localhost -port 9160")
|
114
|
+
end
|
115
|
+
end
|
116
|
+
|
117
|
+
desc "Check Java version"
|
118
|
+
task :java do
|
119
|
+
unless `java -version 2>&1`.split("\n").first =~ /java version "1.6/ #"
|
120
|
+
puts "You need to configure your environment for Java 1.6."
|
121
|
+
puts "If you're on OS X, just export the following environment variables:"
|
122
|
+
puts ' JAVA_HOME="/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home"'
|
123
|
+
puts ' PATH="/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home/bin:$PATH"'
|
124
|
+
exit(1)
|
125
|
+
end
|
126
|
+
end
|
127
|
+
|
128
|
+
namespace :data do
|
129
|
+
desc "Reset test data"
|
130
|
+
task :reset do
|
131
|
+
puts "Resetting test data"
|
132
|
+
sh("rm -rf #{File.join(CASSANDRA_HOME, "cassandra-#{CASSANDRA_VERSION}", 'data')}")
|
133
|
+
end
|
134
|
+
|
135
|
+
desc "Load test data structures."
|
136
|
+
task :load do
|
137
|
+
unless CASSANDRA_VERSION == '0.6'
|
138
|
+
|
139
|
+
schema_path = "#{File.expand_path(Dir.pwd)}/conf/#{CASSANDRA_VERSION}/schema.txt"
|
140
|
+
puts "Loading test data structures."
|
141
|
+
Dir.chdir(File.join(CASSANDRA_HOME, "cassandra-#{CASSANDRA_VERSION}")) do
|
142
|
+
begin
|
143
|
+
sh("bin/cassandra-cli --host localhost --batch < #{schema_path}")
|
144
|
+
rescue
|
145
|
+
puts "Schema already loaded."
|
146
|
+
end
|
147
|
+
end
|
148
|
+
end
|
149
|
+
end
|
150
|
+
end
|
151
|
+
|
152
|
+
task :test => 'data:load'
|
153
|
+
|
154
|
+
# desc "Regenerate thrift bindings for Cassandra" # Dev only
|
155
|
+
task :thrift do
|
156
|
+
puts "Generating Thrift bindings"
|
157
|
+
FileUtils.mkdir_p "vendor/#{CASSANDRA_VERSION}"
|
158
|
+
|
159
|
+
system(
|
160
|
+
"cd vendor/#{CASSANDRA_VERSION} &&
|
161
|
+
rm -rf gen-rb &&
|
162
|
+
thrift -gen rb #{File.join(CASSANDRA_HOME, "cassandra-#{CASSANDRA_VERSION}")}/interface/cassandra.thrift")
|
163
|
+
end
|
164
|
+
|
165
|
+
task :fix_perms do
|
166
|
+
chmod_R 0755, './'
|
167
|
+
end
|
168
|
+
|
169
|
+
task :pkg => [:fix_perms]
|
@@ -0,0 +1,16 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
|
3
|
+
require 'rubygems'
|
4
|
+
require 'rake'
|
5
|
+
require 'cassandra'
|
6
|
+
|
7
|
+
gem_path = $LOAD_PATH.last.sub(/lib$/, "")
|
8
|
+
|
9
|
+
Dir.chdir(gem_path) do
|
10
|
+
if !ENV["CASSANDRA_INCLUDE"]
|
11
|
+
puts "Set the CASSANDRA_INCLUDE environment variable to use a non-default cassandra.in.sh and friends."
|
12
|
+
end
|
13
|
+
|
14
|
+
ARGV << "-T" if ARGV.empty?
|
15
|
+
exec("env FROM_BIN_CASSANDRA_HELPER=1 rake #{ARGV.join(' ')}")
|
16
|
+
end
|
@@ -0,0 +1,47 @@
|
|
1
|
+
# Licensed to the Apache Software Foundation (ASF) under one
|
2
|
+
# or more contributor license agreements. See the NOTICE file
|
3
|
+
# distributed with this work for additional information
|
4
|
+
# regarding copyright ownership. The ASF licenses this file
|
5
|
+
# to you under the Apache License, Version 2.0 (the
|
6
|
+
# "License"); you may not use this file except in compliance
|
7
|
+
# with the License. You may obtain a copy of the License at
|
8
|
+
#
|
9
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
10
|
+
#
|
11
|
+
# Unless required by applicable law or agreed to in writing, software
|
12
|
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
13
|
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
14
|
+
# See the License for the specific language governing permissions and
|
15
|
+
# limitations under the License.
|
16
|
+
|
17
|
+
# The directory where Cassandra's configs live (required)
|
18
|
+
CASSANDRA_CONF=$CASSANDRA_CONF
|
19
|
+
|
20
|
+
# This can be the path to a jar file, or a directory containing the
|
21
|
+
# compiled classes. NOTE: This isn't needed by the startup script,
|
22
|
+
# it's just used here in constructing the classpath.
|
23
|
+
cassandra_bin=$CASSANDRA_HOME/build/classes
|
24
|
+
|
25
|
+
# The java classpath (required)
|
26
|
+
CLASSPATH=$CASSANDRA_CONF:$CASSANDRA_BIN
|
27
|
+
|
28
|
+
for jar in $CASSANDRA_HOME/lib/*.jar $CASSANDRA_HOME/build/lib/jars/*.jar; do
|
29
|
+
CLASSPATH=$CLASSPATH:$jar
|
30
|
+
done
|
31
|
+
|
32
|
+
# Arguments to pass to the JVM
|
33
|
+
JVM_OPTS=" \
|
34
|
+
-ea \
|
35
|
+
-Xms128M \
|
36
|
+
-Xmx1G \
|
37
|
+
-XX:TargetSurvivorRatio=90 \
|
38
|
+
-XX:+AggressiveOpts \
|
39
|
+
-XX:+UseParNewGC \
|
40
|
+
-XX:+UseConcMarkSweepGC \
|
41
|
+
-XX:+CMSParallelRemarkEnabled \
|
42
|
+
-XX:+HeapDumpOnOutOfMemoryError \
|
43
|
+
-XX:SurvivorRatio=128 \
|
44
|
+
-XX:MaxTenuringThreshold=0 \
|
45
|
+
-Dcom.sun.management.jmxremote.port=8080 \
|
46
|
+
-Dcom.sun.management.jmxremote.ssl=false \
|
47
|
+
-Dcom.sun.management.jmxremote.authenticate=false"
|
@@ -0,0 +1,38 @@
|
|
1
|
+
# Licensed to the Apache Software Foundation (ASF) under one
|
2
|
+
# or more contributor license agreements. See the NOTICE file
|
3
|
+
# distributed with this work for additional information
|
4
|
+
# regarding copyright ownership. The ASF licenses this file
|
5
|
+
# to you under the Apache License, Version 2.0 (the
|
6
|
+
# "License"); you may not use this file except in compliance
|
7
|
+
# with the License. You may obtain a copy of the License at
|
8
|
+
#
|
9
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
10
|
+
#
|
11
|
+
# Unless required by applicable law or agreed to in writing, software
|
12
|
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
13
|
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
14
|
+
# See the License for the specific language governing permissions and
|
15
|
+
# limitations under the License.
|
16
|
+
|
17
|
+
# for production, you should probably set the root to INFO
|
18
|
+
# and the pattern to %c instead of %l. (%l is slower.)
|
19
|
+
|
20
|
+
# output messages into a rolling log file as well as stdout
|
21
|
+
log4j.rootLogger=DEBUG,stdout,R
|
22
|
+
|
23
|
+
# stdout
|
24
|
+
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
|
25
|
+
log4j.appender.stdout.layout=org.apache.log4j.SimpleLayout
|
26
|
+
|
27
|
+
# rolling log file ("system.log
|
28
|
+
log4j.appender.R=org.apache.log4j.DailyRollingFileAppender
|
29
|
+
log4j.appender.R.DatePattern='.'yyyy-MM-dd-HH
|
30
|
+
log4j.appender.R.layout=org.apache.log4j.PatternLayout
|
31
|
+
log4j.appender.R.layout.ConversionPattern=%5p [%t] %d{ISO8601} %F (line %L) %m%n
|
32
|
+
# Edit the next line to point to your logs directory
|
33
|
+
log4j.appender.R.File=data/logs/system.log
|
34
|
+
|
35
|
+
# Application logging options
|
36
|
+
#log4j.logger.com.facebook=DEBUG
|
37
|
+
#log4j.logger.com.facebook.infrastructure.gms=DEBUG
|
38
|
+
#log4j.logger.com.facebook.infrastructure.db=DEBUG
|
@@ -0,0 +1,57 @@
|
|
1
|
+
{"Twitter":{
|
2
|
+
"Users":{
|
3
|
+
"CompareWith":"org.apache.cassandra.db.marshal.UTF8Type",
|
4
|
+
"Type":"Standard"},
|
5
|
+
"UserAudits":{
|
6
|
+
"CompareWith":"org.apache.cassandra.db.marshal.UTF8Type",
|
7
|
+
"Type":"Standard"},
|
8
|
+
"UserRelationships":{
|
9
|
+
"CompareSubcolumnsWith":"org.apache.cassandra.db.marshal.TimeUUIDType",
|
10
|
+
"CompareWith":"org.apache.cassandra.db.marshal.UTF8Type",
|
11
|
+
"Type":"Super"},
|
12
|
+
"Usernames":{
|
13
|
+
"CompareWith":"org.apache.cassandra.db.marshal.UTF8Type",
|
14
|
+
"Type":"Standard"},
|
15
|
+
"Statuses":{
|
16
|
+
"CompareWith":"org.apache.cassandra.db.marshal.UTF8Type",
|
17
|
+
"Type":"Standard"},
|
18
|
+
"StatusAudits":{
|
19
|
+
"CompareWith":"org.apache.cassandra.db.marshal.UTF8Type",
|
20
|
+
"Type":"Standard"},
|
21
|
+
"StatusRelationships":{
|
22
|
+
"CompareSubcolumnsWith":"org.apache.cassandra.db.marshal.TimeUUIDType",
|
23
|
+
"CompareWith":"org.apache.cassandra.db.marshal.UTF8Type",
|
24
|
+
"Type":"Super"},
|
25
|
+
"Index":{
|
26
|
+
"CompareWith":"org.apache.cassandra.db.marshal.UTF8Type",
|
27
|
+
"Type":"Super"},
|
28
|
+
"TimelinishThings":{
|
29
|
+
"CompareWith":"org.apache.cassandra.db.marshal.BytesType",
|
30
|
+
"Type":"Standard"}
|
31
|
+
},
|
32
|
+
"Multiblog":{
|
33
|
+
"Blogs":{
|
34
|
+
"CompareWith":"org.apache.cassandra.db.marshal.TimeUUIDType",
|
35
|
+
"Type":"Standard"},
|
36
|
+
"Comments":{
|
37
|
+
"CompareWith":"org.apache.cassandra.db.marshal.TimeUUIDType",
|
38
|
+
"Type":"Standard"}
|
39
|
+
},
|
40
|
+
"MultiblogLong":{
|
41
|
+
"Blogs":{
|
42
|
+
"CompareWith":"org.apache.cassandra.db.marshal.LongType",
|
43
|
+
"Type":"Standard"},
|
44
|
+
"Comments":{
|
45
|
+
"CompareWith":"org.apache.cassandra.db.marshal.LongType",
|
46
|
+
"Type":"Standard"}
|
47
|
+
},
|
48
|
+
"TypeConversions":{
|
49
|
+
"UUIDColumnConversion":{
|
50
|
+
"CompareWith":"org.apache.cassandra.db.marshal.TimeUUIDType",
|
51
|
+
"Type":"Standard"},
|
52
|
+
"SuperUUID":{
|
53
|
+
"CompareSubcolumnsWith":"org.apache.cassandra.db.marshal.TimeUUIDType",
|
54
|
+
"CompareWith":"org.apache.cassandra.db.marshal.TimeUUIDType",
|
55
|
+
"Type":"Super"}
|
56
|
+
}
|
57
|
+
}
|