cassandra-mavericks 0.21.1

data/README.md

@@ -0,0 +1,373 @@
+# cassandra
+A Ruby client for the Cassandra distributed database.
+
+Supports 1.8.7, 1.9.2, and rubinius on Cassandra 0.6.13, 0.7.9, 0.8.6, 1.0.0-rc2, 1.1.5.
+
+## Getting Started
+
+Here is a quick sample of the general use (more details in Read/Write
+API below):
+
+    require 'cassandra'
+    client = Cassandra.new('Twitter', '127.0.0.1:9160')
+    client.insert(:Users, "5", {'screen_name' => "buttonscat"})
+
+## License
+
+Copyright 2009-2011 Twitter, Inc. See included LICENSE file. Portions copyright 2004-2009 David Heinemeier Hansson, and used with permission.
+
+## Cassandra Version
+
+The Cassandra project is under very active development, and as such
+there are a few different versions that you may need to use this gem
+with.  We have set up an easy sure fire mechanism for selecting the
+specific version that you are connecting to while requiring the gem.
+
+#### Require Method
+The default version is the currently stable release of cassandra.  (0.8
+at this time.)
+
+To use the default version simply use a normal require:
+
+    require 'cassandra'
+
+To use a specific version (1.0 in this example) you would use a
+slightly differently formatted require:
+
+    require 'cassandra/1.0'
+
+#### Environment Variable Method
+These mechanisms work well when you are using the cassandra gem in your
+own projects or irb, but if you would rather not hard code your app to a
+specific version you can always specify an environment variable with the
+version you are using:
+
+    export CASSANDRA_VERSION=0.8
+
+Then you would use the default require as listed above:
+
+    require 'cassandra'
+
+## Read/Write API Method Reference
+
+### insert
+
+* column\_family - The column\_family that you are inserting into.
+* key - The row key to insert.
+* hash - The columns or super columns to insert.
+* options - Valid options are:
+  * :timestamp - Uses the current time if none specified.
+  * :consistency - Uses the default write consistency if none specified.
+  * :ttl - If specified this is the number of seconds after the insert that this value will be available.
+
+This is the main method used to insert rows into cassandra. If the
+column\_family that you are inserting into is a SuperColumnFamily then
+the hash passed in should be a nested hash, otherwise it should be a
+flat hash.
+
+This method can also be called while in batch mode. If in batch mode
+then we queue up the mutations (an insert in this case) and pass them to
+cassandra in a single batch at the end of the block.
+
+Example:
+
+    @client.insert(:Statuses, key, {'body' => 'v', 'user' => 'v'})
+
+    columns = {@uuids[1] => 'v1', @uuids[2] => 'v2'}
+    @client.insert(:StatusRelationships, key, {'user_timelines' => columns})
+
+
+### remove
+
+* column\_family - The column\_family that you are working with.
+* key - The row key to remove (or remove columns from).
+* columns - Either a single super_column or a list of columns to remove.
+* sub_columns - The list of sub\_columns to remove.
+* options - Valid options are:
+  * :timestamp - Uses the current time if none specified.
+  * :consistency - Uses the default write consistency if none specified.
+
+This method is used to delete (actually marking them as deleted with a
+tombstone) rows, columns, or super columns depending on the parameters
+passed.  If only a key is passed the entire row will be marked as deleted.
+If a column name is passed in that column will be deleted.
+
+Example:
+
+    @client.insert(:Statuses, key, {'body' => 'v', 'subject' => 'v'})
+
+    @client.remove(:Statuses, key, 'body')    # removes the 'body' column
+    @client.remove(:Statuses, key)            # removes the row
+
+### count\_columns
+
+Count the columns for the provided parameters.
+
+* column\_family - The column\_family that you are working with.
+* key - The row key.
+* columns - Either a single super_column or a list of columns.
+* sub_columns - The list of sub\_columns to select.
+* options - Valid options are:
+  * :start - The column name to start from.
+  * :stop - The column name to stop at.
+  * :count - The maximum count of columns to return. (By default cassandra will count up to 100 columns)
+  * :consistency - Uses the default read consistency if none specified.
+
+Example:
+
+    @client.insert(:Statuses, key, {'body' => 'v1', 'user' => 'v2'})
+    @client.count_columns(:Statuses, key)     # returns 2
+
+### get
+
+Return a hash (actually, a Cassandra::OrderedHash) or a single value
+representing the element at the column_family:key:[column]:[sub_column]
+path you request.
+
+* column\_family - The column\_family that you are working with.
+* key - The row key to select.
+* columns - Either a single super\_column or a list of columns.
+* sub\_columns - The list of sub\_columns to select.
+* options - Valid options are:
+  * :count    - The number of columns requested to be returned.
+  * :start    - The starting value for selecting a range of columns.
+  * :finish   - The final value for selecting a range of columns.
+  * :reversed - If set to true the results will be returned in
+                reverse order.
+  * :consistency - Uses the default read consistency if none specified.
+
+Example:
+
+    @client.insert(:Users, key, {'body' => 'v', 'user' => 'v'})
+    @client.get(:Users, key))           # returns {'body' => 'v', 'user' => 'v'}
+
+### multi\_get
+
+Multi-key version of Cassandra#get.
+
+This method allows you to select multiple rows with a single query.
+If a key that is passed in doesn't exist an empty hash will be
+returned.
+
+Supports the same parameters as Cassandra#get.
+
+* column_family - The column_family that you are working with.
+* key - An array of keys to select.
+* columns - Either a single super_column or a list of columns.
+* sub_columns - The list of sub\_columns to select.
+* options - Valid options are:
+  * :count    - The number of columns requested to be returned.
+  * :start    - The starting value for selecting a range of columns.
+  * :finish   - The final value for selecting a range of columns.
+  * :reversed - If set to true the results will be returned in reverse order.
+  * :consistency - Uses the default read consistency if none specified.
+
+Example:
+
+    @client.insert(:Users, '1', {'body' => 'v1', 'user' => 'v1'})
+    @client.insert(:Users, '2', {'body' => 'v2', 'user' => 'v2'})
+
+    expected = OrderedHash[
+                  '1', {'body' => 'v1', 'user' => 'v1'},
+                  '2', {'body' => 'v2', 'user' => 'v2'},
+                  'bogus', {}
+              ]
+    result = @client.multi_get(:Users, ['1', '2', 'bogus'])
+
+### exists?
+
+Return true if the column\_family:key:[column]:[sub\_column] path you
+request exists.
+
+If passed in only a row key it will query for any columns (limiting
+to 1) for that row key. If a column is passed in it will query for
+that specific column/super column.
+
+This method will return true or false.
+
+* column\_family - The column\_family that you are working with.
+* key - The row key to check.
+* columns - Either a single super\_column or a list of columns.
+* sub\_columns - The list of sub\_columns to check.
+* options - Valid options are:
+  * :consistency - Uses the default read consistency if none specified.
+
+Example:
+
+    @client.insert(:Statuses, 'key', {'body' => 'v'})
+    @client.exists?(:Statuses, 'key')                   # returns true
+    @client.exists?(:Statuses, 'bogus')                 # returns false
+    @client.exists?(:Statuses, 'key', 'body')           # returns true
+    @client.exists?(:Statuses, 'key', 'bogus')          # returns false
+
+### get\_range
+Return an Cassandra::OrderedHash containing the columns specified for the given
+range of keys in the column\_family you request.
+
+This method is just a convenience wrapper around Cassandra#get_range_single
+and Cassandra#get\_range\_batch. If :key\_size, :batch\_size, or a block
+is passed in Cassandra#get\_range\_batch will be called. Otherwise
+Cassandra#get\_range\_single will be used.
+
+The start\_key and finish\_key parameters are only useful for iterating of all records
+as is done in the Cassandra#each and Cassandra#each\_key methods if you are using the
+RandomPartitioner.
+
+If the table is partitioned with OrderPreservingPartitioner you may
+use the start\_key and finish\_key params to select all records with
+the same prefix value.
+
+If a block is passed in we will yield the row key and columns for
+each record returned.
+
+Please note that Cassandra returns a row for each row that has existed in the
+system since gc\_grace\_seconds. This is because deleted row keys are marked as
+deleted, but left in the system until the cluster has had resonable time to replicate the deletion.
+This function attempts to suppress deleted rows (actually any row returned without
+columns is suppressed).
+
+* column\_family - The column\_family that you are working with.
+* options - Valid options are:
+  * :start\_key    - The starting value for selecting a range of keys (only useful with OPP).
+  * :finish\_key   - The final value for selecting a range of keys (only useful with OPP).
+  * :key\_count    - The total number of keys to return from the query. (see note regarding deleted records)
+  * :batch\_size   - The maximum number of keys to return per query. If specified will loop until :key\_count is obtained or all records have been returned.
+  * :columns      - A list of columns to return.
+  * :count        - The number of columns requested to be returned.
+  * :start        - The starting value for selecting a range of columns.
+  * :finish       - The final value for selecting a range of columns.
+  * :reversed     - If set to true the results will be returned in reverse order.
+  * :consistency  - Uses the default read consistency if none specified.
+
+Example:
+
+    10.times do |i|
+      @client.insert(:Statuses, i.to_s, {'body' => '1'})
+    end
+
+    @client.get_range_keys(:Statuses, :key_count => 4)
+
+    # returns:
+    #{
+    #  '0' => {'body' => '1'},
+    #  '1' => {'body' => '1'},
+    #  '2' => {'body' => '1'},
+    #  '3' => {'body' => '1'}
+    #}
+
+### count\_range
+
+Return an Array containing all of the keys within a given range.
+
+This method just calls Cassandra#get\_range and returns the
+row keys for the records returned.
+
+See Cassandra#get\_range for options.
+
+### get\_range\_keys
+
+Return an Array containing all of the keys within a given range.
+
+This method just calls Cassandra#get\_range and returns the
+row keys for the records returned.
+
+See Cassandra#get\_range for options.
+
+### each\_key
+Iterate through each key within the given range parameters. This function can be
+used to iterate over each key in the given column family.
+
+This method just calls Cassandra#get\_range and yields each row key.
+
+See Cassandra#get\_range for options.
+
+Example:
+    10.times do |i|
+      @client.insert(:Statuses, k + i.to_s, {"body-#{i.to_s}" => 'v'})
+    end
+
+    @client.each_key(:Statuses) do |key|
+      print key
+    end
+
+    # returns 0123456789
+
+### each
+Iterate through each row within the given column\_family.
+
+This method just calls Cassandra#get\_range and yields the key and
+columns.
+
+See Cassandra#get\_range for options.
+
+### get\_index\_slices
+This method is used to query a secondary index with a set of
+provided search parameters
+
+Please note that you can either specify a
+CassandraThrift::IndexClause or an array of hashes with the
+format as below.
+
+* column\_family     - The Column Family this operation will be run on.
+* index\_clause      - This can either be a CassandraThrift::IndexClause or an array of hashes with the following keys:
+  * :column\_name - Column to be compared
+  * :value       - Value to compare against
+  * :comparison  - Type of comparison to do.
+* options
+  * :key\_count    - Set maximum number of rows to return. (Only works if CassandraThrift::IndexClause is not passed in.)
+  * :key\_start    - Set starting row key for search. (Only works if CassandraThrift::IndexClause is not passed in.)
+  * :consistency
+
+Example:
+
+    @client.create_index('Twitter', 'Statuses', 'x', 'LongType')
+
+    @client.insert(:Statuses, 'row1', { 'x' => [0,10].pack("NN")  })
+
+    (2..10).to_a.each do |i|
+      @twitter.insert(:Statuses, 'row' + i.to_s, { 'x' => [0,20].pack("NN"), 'non_indexed' => [i].pack('N*') })
+    end
+
+    @client.insert(:Statuses, 'row11', { 'x' => [0,30].pack("NN")  })
+
+    expressions = [{:column_name => 'x', :value => [0,20].pack("NN"), :comparison => "=="}]
+
+    # verify multiples will be returned
+    @client.get_indexed_slices(:Statuses, expressions).length       # returns 9
+
+    # verify that GT and LT queries perform properly
+    expressions   =  [
+                      { :column_name => 'x',
+                        :value => [0,20].pack("NN"),
+                        :comparison => "=="},
+                      { :column_name => 'non_indexed',
+                        :value => [5].pack("N*"),
+                        :comparison => ">"}
+                    ]
+
+    @client.get_indexed_slices(:Statuses, expressions).length       # returns 5
+
+### batch
+Takes a block where all the mutations (inserts and deletions) inside it are 
+queued, and at the end of the block are passed to cassandra in a single batch.
+
+If you don't want to send all the mutations inside the block in a big single 
+batch, you can use the :queue\_size option to send smaller batches. If the 
+queue is not empty at the end of the block, the remaining mutations are sent.
+
+* options
+  * :consistency  - Override the consistency level from individual mutations.
+  * :queue\_size  - Maximum number of mutations to send at once.
+
+Example: 
+
+    @client.batch do 
+      @client.insert(:Statuses, 'k1', {'body' => 'v1'})
+      @client.insert(:Statuses, 'k2', {'body' => 'v2'})
+      @client.remove(:Statuses, 'k3')
+    end
+
+
+## Reporting Problems
+
+The Github issue tracker is [here](http://github.com/twitter/cassandra/issues). If you have problems with this library or Cassandra itself, please use the [cassandra-user mailing list](http://mail-archives.apache.org/mod_mbox/incubator-cassandra-user/).

data/Rakefile

@@ -0,0 +1,191 @@
+require 'fileutils'
+require 'rake/testtask'
+require 'rake/extensiontask'
+
+CassandraBinaries = {
+  '0.6' => 'http://archive.apache.org/dist/cassandra/0.6.13/apache-cassandra-0.6.13-bin.tar.gz',
+  '0.7' => 'http://archive.apache.org/dist/cassandra/0.7.9/apache-cassandra-0.7.9-bin.tar.gz',
+  '0.8' => 'http://archive.apache.org/dist/cassandra/0.8.7/apache-cassandra-0.8.7-bin.tar.gz',
+  '1.0' => 'http://archive.apache.org/dist/cassandra/1.0.6/apache-cassandra-1.0.6-bin.tar.gz',
+  '1.1' => 'http://archive.apache.org/dist/cassandra/1.1.5/apache-cassandra-1.1.5-bin.tar.gz',
+  '1.2' => 'http://archive.apache.org/dist/cassandra/1.2.1/apache-cassandra-1.2.1-bin.tar.gz'
+}
+
+CASSANDRA_HOME = ENV['CASSANDRA_HOME'] || "#{ENV['HOME']}/cassandra"
+CASSANDRA_VERSION = ENV['CASSANDRA_VERSION'] || '0.8'
+CASSANDRA_PIDFILE = ENV['CASSANDRA_PIDFILE'] || "#{CASSANDRA_HOME}/cassandra.pid"
+
+def setup_cassandra_version(version = CASSANDRA_VERSION)
+  FileUtils.mkdir_p CASSANDRA_HOME
+
+  destination_directory = File.join(CASSANDRA_HOME, 'cassandra-' + CASSANDRA_VERSION)
+
+  unless File.exists?(File.join(destination_directory, 'bin','cassandra'))
+    download_source       = CassandraBinaries[CASSANDRA_VERSION]
+    download_destination  = File.join("/tmp", File.basename(download_source))
+    untar_directory       = File.join(CASSANDRA_HOME,  File.basename(download_source,'-bin.tar.gz'))
+
+    puts "downloading cassandra"
+    sh "curl -L -o #{download_destination} #{download_source}"
+
+    sh "tar xzf #{download_destination} -C #{CASSANDRA_HOME}"
+    sh "mv #{untar_directory} #{destination_directory}"
+  end
+end
+
+def setup_environment
+  env = ""
+  if !ENV["CASSANDRA_INCLUDE"]
+    env << "CASSANDRA_INCLUDE=#{File.expand_path(Dir.pwd)}/conf/#{CASSANDRA_VERSION}/cassandra.in.sh "
+    env << "CASSANDRA_HOME=#{CASSANDRA_HOME}/cassandra-#{CASSANDRA_VERSION} "
+    env << "CASSANDRA_CONF=#{File.expand_path(Dir.pwd)}/conf/#{CASSANDRA_VERSION}"
+  else
+    env << "CASSANDRA_INCLUDE=#{ENV['CASSANDRA_INCLUDE']} "
+    env << "CASSANDRA_HOME=#{ENV['CASSANDRA_HOME']} "
+    env << "CASSANDRA_CONF=#{ENV['CASSANDRA_CONF']}"
+  end
+
+  env
+end
+
+def running?(pid_file = nil)
+  pid_file ||= CASSANDRA_PIDFILE
+
+  if File.exists?(pid_file)
+    pid = File.new(pid_file).read.to_i
+    begin
+      Process.kill(0, pid)
+      return true
+    rescue
+      File.delete(pid_file)
+    end
+  end
+
+  false
+end
+
+def listening?(host, port)
+  TCPSocket.new(host, port).close
+  true
+rescue Errno::ECONNREFUSED => e
+  false
+end
+
+namespace :cassandra do
+  desc "Start Cassandra"
+  task :start, [:daemonize] => :java do |t, args|
+    args.with_defaults(:daemonize => true)
+
+    setup_cassandra_version
+    env = setup_environment
+
+    Dir.chdir(File.join(CASSANDRA_HOME, "cassandra-#{CASSANDRA_VERSION}")) do
+      sh("env #{env} bin/cassandra #{'-f' unless args.daemonize} -p #{CASSANDRA_PIDFILE}")
+    end
+
+    if args.daemonize
+      end_time = Time.now + 30
+      host     = '127.0.0.1'
+      port     = 9160
+
+      until Time.now >= end_time || listening?(host, port)
+        puts "waiting for 127.0.0.1:9160"
+        sleep 0.1
+      end
+
+      unless listening?(host, port)
+        raise "timed out waiting for cassandra to start"
+      end
+    end
+  end
+
+  desc "Stop Cassandra"
+  task :stop => :java do
+    setup_cassandra_version
+    env = setup_environment
+    sh("kill $(cat #{CASSANDRA_PIDFILE})")
+  end
+end
+
+desc "Start Cassandra"
+task :cassandra => :java do
+  begin
+    Rake::Task["cassandra:start"].invoke(false)
+  rescue RuntimeError => e
+    raise e unless e.message =~ /Command failed with status \(130\)/ # handle keyboard interupt errors
+  end
+end
+
+desc "Run the Cassandra CLI"
+task :cli do
+  Dir.chdir(File.join(CASSANDRA_HOME, "cassandra-#{CASSANDRA_VERSION}")) do
+    sh("bin/cassandra-cli -host localhost -port 9160")
+  end
+end
+
+desc "Check Java version"
+task :java do
+  is_java16 = `java -version 2>&1`.split("\n").first =~ /java version "1.6/
+
+  if ['0.6', '0.7'].include?(CASSANDRA_VERSION) && !java16
+    puts "You need to configure your environment for Java 1.6."
+    puts "If you're on OS X, just export the following environment variables:"
+    puts '  JAVA_HOME="/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home"'
+    puts '  PATH="/System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home/bin:$PATH"'
+    exit(1)
+  end
+end
+
+namespace :data do
+  desc "Reset test data"
+  task :reset do
+    puts "Resetting test data"
+    sh("rm -rf #{File.join(CASSANDRA_HOME, "cassandra-#{CASSANDRA_VERSION}", 'data')}")
+  end
+
+  desc "Load test data structures."
+  task :load do
+    unless CASSANDRA_VERSION == '0.6'
+
+      schema_path = "#{File.expand_path(Dir.pwd)}/conf/#{CASSANDRA_VERSION}/schema.txt"
+      puts "Loading test data structures."
+      Dir.chdir(File.join(CASSANDRA_HOME, "cassandra-#{CASSANDRA_VERSION}")) do
+        begin
+          sh("bin/cassandra-cli --host localhost --batch < #{schema_path}")
+        rescue
+          puts "Schema already loaded."
+        end
+      end
+    end
+  end
+end
+
+task :test => 'data:load'
+
+# desc "Regenerate thrift bindings for Cassandra" # Dev only
+task :thrift do
+  puts "Generating Thrift bindings"
+  FileUtils.mkdir_p "vendor/#{CASSANDRA_VERSION}"
+
+  system(
+    "cd vendor/#{CASSANDRA_VERSION} &&
+    rm -rf gen-rb &&
+    thrift -gen rb #{File.join(CASSANDRA_HOME, "cassandra-#{CASSANDRA_VERSION}")}/interface/cassandra.thrift")
+end
+
+task :fix_perms do
+  chmod_R 0755, './'
+end
+
+task :pkg => [:fix_perms]
+
+Rake::ExtensionTask.new('cassandra_native') do |ext|
+  ext.ext_dir = 'ext'
+end
+
+Rake::TestTask.new do |t|
+  t.test_files = FileList['test/*.rb']
+end
+
+task :default => :test
+task :test => :compile