cassandra_model 0.9.3.2

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 811a3063acc6a1abd58de21ba424e0784606718f
+  data.tar.gz: af46520692a4f205f6cd8f996d928d09d50ef553
+SHA512:
+  metadata.gz: 3e580bb73ed0999532c19b0a6ec881caa2b265ab0fead7bef262bd772d09eeac53885217777ee3a42245aaea02d6ee2c6f33f917e8cec5d67bb1ad1dd1b832d1
+  data.tar.gz: f309db41c5212ce53996f4d5e3b71e3a1755e628a32ceeed0d3de36f451c984fb46b8406f702c15c1cdd9def00abc09f1a76b2b8e9819a1c26329b08a2b6f04c

data/LICENSE.txt

@@ -0,0 +1,13 @@
+Copyright 2014-2015 Thomas RM Rogers
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.

data/README.md

@@ -0,0 +1,170 @@
+# Cassandra Model
+
+The Cassandra Model gem aims at providing intuitive, simple data modelling capabilities for use with Ruby with Apache Cassandra, while still providing access to functionality that makes using Cassandra really powerful.
+
+## Installation
+
+As this project is currently in pre-version 1.0, it is only available through github. 
+
+To integrate Cassandra Model in to your project, add the following lines to your application's Gemfile:
+
+    gem 'thomas_utils', github: 'thomasrogers03/thomas_utils'
+    gem 'cassandra_model', github: 'thomasrogers03/cassandra_model'
+
+The Thomas Utils gem is a separate project (located at https://github.com/thomasrogers03/thomas_utils.git) used for some minor helper utitilities used within this project.
+
+## Getting started
+
+Cassandra Model offers a number of different ways to construct and uses models, from using existing tables created through your own migration framework, to building tables dynamically simply by describing how a table is intended to be used.
+
+### A familiar starting point, for those of you are used to using ActiveRecord:
+
+```ruby
+require 'cassandra_model'
+
+class Car < CassandraModel::Record
+end
+
+Car.create(make: 'Honda', year: 2014, model: 'Civic', colour: 'Green')
+Car.create(make: 'Honda', year: 2013, model: 'Civic', colour: 'Blue')
+
+recent_hondas = Car.where(make: 'Honda', :year.gt => 2.years.ago).get
+```
+
+This example illustrates how an existing **cars** table can be modelled to grab all the Honda vehicles whose year is at least 2 years ago. It assumes that you have an existing table **cars**, with a partition key *make* and a clustering column *year*.
+
+### A more interesting example, demonstrating how to take advantage of asynchronous queries and token distribution in Cassandra:
+
+```ruby
+class Car < CassandraModel::Record
+end
+
+futures = []
+futures << Car.create_async(make: 'Honda', year: 2014, model: 'Civic', colour: 'Green')
+futures << Car.create_async(make: 'Honda', year: 2013, model: 'Civic', colour: 'Blue')
+futures << Car.create_async(make: 'Toyota', year: 2014, model: 'Highlander', colour: 'Blue')
+futures.map(&:join)
+
+makes = %(Honda Toyota GM)
+cars = makes.map { |make| Car.where(make: make, :year.gt => 2.years.ago).async }.map(&:get).flatten
+```
+
+This example shows how we can use Future/Promise API that the Datastax Cassandra ruby-driver provides to execute a number of queries asynchronously and wait for the results. 
+
+More importantly, however, it shows us how we can take advantages of very fast writes across nodes in a Cassandra backed application. Such design principles quickly become very important when dealing with Cassandra in both write and read friendly requirements.
+
+## Generating tables dynamically
+
+There are two ways we can define meta tables (or tables that exist as we model them in Ruby). This can be done by defining a table definition, and then associating a MetaTable with the model, or by using built in data modelling features.
+
+### Before we can create an meta tables
+
+A table to keep track of all of these tables must be created 
+
+```ruby
+CassandraModel::TableDescriptor.create_descriptor_table
+```
+
+This method will create a table of descriptors for meta tables, if it does not already exist.
+
+### Creating a MetaTable based model
+
+```ruby
+class Car < CassandraModel::Record
+    TABLE_ATTRIBUTES = {
+        name: :cars,
+        partition_key: { make: :text },
+        clustering_columns: { year: :int, model: :text },
+        remaining_columns: { attributes: 'map<text, text>' }
+    }
+    TABLE_DEFINITION = CassandraModel::TableDefinition.new(TABLE_ATTRIBUTES)
+    self.table = CassandraModel::MetaTable.new(TABLE_DEFINITION)
+end
+```
+
+This model is now ready to be used similarly to the examples above, without having to write any CQL!
+
+__Note__: Tables defined in this way will have unique identifiers appended to their name in Cassandra. This is done so we can modify the table design without dropping (!) and re-creating tables in Cassandra. The history of these tables is recorded in the **table_descriptors** table.
+
+### Using data modelling helpers
+
+The data modelling features in Cassandra Model are meant to be a somewhat verbose, high-level table designing tool to help build performant Cassandra tables with meaning. The following code demonstrates how we can re-create the **cars** table using this method.
+
+```ruby
+class Car < CassandraModel::Record
+    extend CassandraModel::DataModelling
+
+    model_data do |inquirer, data_set|
+        inquirer.knows_about(:make)
+
+        data_set.is_defined_by(:year, :model)
+        data_set.change_type_of(:year).to(:int)
+        data_set.knows_about(:colour)
+    end
+end
+```
+
+Here, we try to define the table in terms of what kind of questions we'd like to ask about our data. In this particular case, we can ask about all of the cars given a specific make, and learn about their details. Note that a columns by default are assumed to be text columns.
+
+## Records with composite columns
+
+One of the ways to avoid numerous queries to Cassandra is to de-normalize data as much a possible when saving records. However, sometimes we want to learn about data without having complete knowledge as to how it is defined. This can be accomplished using the CompositeRecord helpers. 
+
+To use this, we define an inquirer and a data set with the pieces of information we know, and let it handle what we don't know.
+
+```ruby
+class Car < CassandraModel::Record
+    extend CassandraModel::DataModelling
+
+    model_data do |inquirer, data_set|
+        inquirer.knows_about(:make, :model, :year, :colour)
+        inquirer.knows_about(:make)
+        inquirer.knows_about(:make, :model, :year)
+        inquirer.knows_about(:vin)
+        inquirer.defaults(:year).to(1900)
+
+        data_set.is_defined_by(:price, :vin, :make, :model, :colour)
+        data_set.change_type_of(:price).to(:double)
+        data_set.knows_about(:description)
+    end
+end
+
+Car.create(make: 'Honda', model: 'Civic', year: 2003, colour: 'Blue', vin: '123456789', price: 2_000.0, description: 'A very reliable car')
+```
+
+With this model in hand, now we can ask questions like "What are all the cars we have for Toyota?" Or "How many 2001 Honda Civics do we have in blue?" We can also ask for the price range of a very specific model if we want to.
+
+## Additional features
+
+* Flexible table sharding
+* Automatically rotating tables in use for maintenance
+* Configuring multiple Cassandra connections over multiple keyspaces
+* Mixing query helpers with ActiveRecord to build intuitive relations between a relational database and Cassandra
+
+## Undocumented features
+
+There are a number of features in Cassandra Model that may have missing or incomplete documentation. This will change as time progresses.
+
+## Known issues
+
+* As Cassandra Model uses splat arguments for providing query arguments, only Datastax ruby-driver versions up to 2.0.1 are supported.
+* There is currently no elegant method of migrating data between different versions of meta tables
+* CassandraModel::TableDescriptor.create_descriptor_table does not wait for table persistence in Cassandra
+* CassandraModel::TableDescriptor.create_descriptor_table is vulnerable to being created multiple times when multiple applications are running with the same code
+
+
+## Copyright
+
+Copyright 2014-2015 Thomas Rogers.
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
+
+[http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0)
+
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
+
+
+
+
+
+

data/lib/cassandra_model.rb

@@ -0,0 +1,48 @@
+#--
+# Copyright 2014-2015 Thomas RM Rogers
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#++
+
+require 'concurrent'
+require 'cassandra'
+require 'active_support/all'
+require 'active_support/core_ext/class/attribute_accessors'
+
+require 'cassandra_model/logging'
+require 'cassandra_model/global_callbacks'
+require 'cassandra_model/single_token_batch'
+require 'cassandra_model/single_token_unlogged_batch'
+require 'cassandra_model/single_token_logged_batch'
+require 'cassandra_model/single_token_counter_batch'
+require 'cassandra_model/batch_reactor'
+require 'cassandra_model/batch_reactor/future'
+require 'cassandra_model/raw_connection'
+require 'cassandra_model/connection_cache'
+require 'cassandra_model/table_definition'
+require 'cassandra_model/table_redux'
+require 'cassandra_model/result_paginator'
+require 'cassandra_model/query_result'
+require 'cassandra_model/query_builder'
+require 'cassandra_model/displayable_attributes'
+require 'cassandra_model/record'
+require 'cassandra_model/counter_record'
+require 'cassandra_model/table_descriptor'
+require 'cassandra_model/meta_table'
+require 'cassandra_model/rotating_table'
+require 'cassandra_model/composite_record_static'
+require 'cassandra_model/composite_record'
+require 'cassandra_model/type_guessing'
+require 'cassandra_model/data_inquirer'
+require 'cassandra_model/data_set'
+require 'cassandra_model/data_modelling'

data/lib/cassandra_model/batch_reactor.rb

@@ -0,0 +1,32 @@
+module CassandraModel
+  class BatchReactor < ::BatchReactor::ReactorCluster
+
+    def initialize(cluster, session, batch_klass, options)
+      @cluster = cluster
+      @session = session
+      @batch_klass = batch_klass
+
+      define_partitioner(&method(:partition))
+      super(cluster.hosts.count, options, &method(:batch_callback))
+    end
+
+    def perform_within_batch(statement)
+      ione_future = super(statement)
+      Future.new(ione_future)
+    end
+
+    private
+
+    def partition(statement)
+      hosts = @cluster.find_replicas(@session.keyspace, statement)
+      @cluster.hosts.find_index(hosts.first) || 0
+    end
+
+    def batch_callback(_)
+      batch = @batch_klass.new
+      yield batch
+      @session.execute_async(batch).on_success { |result| batch.result = result }
+    end
+
+  end
+end

data/lib/cassandra_model/batch_reactor/future.rb

@@ -0,0 +1,49 @@
+module CassandraModel
+  class BatchReactor
+    class Future < Cassandra::Future
+      extend Forwardable
+
+      def self.define_handler(internal_name, external_name = internal_name)
+        define_method(external_name) do |&block|
+          @future.public_send(internal_name, &block)
+          self
+        end
+      end
+
+      define_handler :on_complete
+      define_handler :on_failure
+      define_handler :on_value, :on_success
+      def_delegator :@future, :get
+
+      def initialize(ione_future)
+        @future = ione_future
+      end
+
+      def add_listener(listener)
+        @future.on_complete do |value, error|
+          error ? listener.failure(error) :  listener.success(value)
+        end
+        self
+      end
+
+      def promise
+        raise NotImplementedError
+      end
+
+      def fallback(&_)
+        raise NotImplementedError
+      end
+
+      def then(&block)
+        internal_future = @future.then(&block)
+        Future.new(internal_future)
+      end
+
+      def join
+        @future.get
+        self
+      end
+
+    end
+  end
+end

data/lib/cassandra_model/composite_record.rb

@@ -0,0 +1,49 @@
+module CassandraModel
+  module CompositeRecord
+    def self.included(klass)
+      klass.extend CompositeRecordStatic
+    end
+
+    def save_async(options = {})
+      futures = composite_rows.map { |record| record.internal_save_async(options) }
+
+      futures << internal_save_async(options)
+      Cassandra::Future.all(futures).then { self }
+    end
+
+    def delete_async
+      futures = composite_rows.map { |record| record.internal_delete_async }
+
+      futures << internal_delete_async
+      Cassandra::Future.all(futures).then { self }
+    end
+
+    def update_async(new_attributes)
+      futures = composite_rows.map { |record| record.internal_update_async(new_attributes) }
+
+      futures << internal_update_async(new_attributes)
+      Cassandra::Future.all(futures).then { self }
+    end
+
+    private
+
+    def composite_rows
+      (self.class.composite_defaults || []).map do |row|
+        merged_attributes = attributes.merge(row)
+        self.class.new(merged_attributes, validate: false)
+      end
+    end
+
+    def attribute(column)
+      attributes[column] ||
+          attributes[self.class.composite_ck_map[column]] ||
+          attributes[self.class.composite_pk_map[column]]
+    end
+
+    def internal_attributes
+      internal_columns.inject({}) do |memo, column|
+        memo.merge(column => attribute(column))
+      end
+    end
+  end
+end

data/lib/cassandra_model/composite_record_static.rb

@@ -0,0 +1,169 @@
+module CassandraModel
+  module CompositeRecordStatic
+    extend Forwardable
+
+    def_delegator :table_config, :composite_defaults=
+
+    def partition_key
+      table_data.composite_partition_key ||= internal_partition_key.map { |column| trimmed_column(column, /^rk_/, composite_pk_map) || column }
+    end
+
+    def clustering_columns
+      table_data.composite_clustering_columns ||= internal_clustering_columns.map { |column| trimmed_column(column, /^ck_/, composite_ck_map) || column }
+    end
+
+    def primary_key
+      table_data.composite_primary_key ||= (internal_partition_key + internal_clustering_columns).map do |column|
+        trimmed_column(column, /^rk_/, composite_pk_map) ||
+            trimmed_column(column, /^ck_/, composite_ck_map) ||
+            column
+      end.uniq
+    end
+
+    def columns
+      table_data.composite_columns ||= composite_columns.each { |column| define_attribute(column) }
+    end
+
+    def composite_pk_map
+      unless table_data.composite_pk_map
+        table_data.composite_pk_map = {}
+        columns
+      end
+      table_data.composite_pk_map
+    end
+
+    def composite_ck_map
+      unless table_data.composite_ck_map
+        table_data.composite_ck_map = {}
+        columns
+      end
+      table_data.composite_ck_map
+    end
+
+    def composite_defaults
+      table_data.internal_defaults ||= build_composite_map
+    end
+
+    def generate_composite_defaults(column_defaults, truth_table)
+      table_config.composite_defaults = truth_table.map { |row| column_defaults.except(*row) }
+    end
+
+    def generate_composite_defaults_from_inquirer(inquirer)
+      table_config.composite_defaults = inquirer.composite_rows.map do |row|
+        row.inject({}) do |memo, column|
+          memo.merge!(column => inquirer.column_defaults[column])
+        end
+      end
+    end
+
+    def shard_key
+      table_data.composite_shard_key ||= begin
+        column = super
+        column =~ /^rk_/ ? composite_pk_map[column] : column
+      end
+    end
+
+    private
+
+    def build_composite_map
+      if table_config.composite_defaults
+        table_config.composite_defaults.map { |row| row_composite_default(row) }
+      end
+    end
+
+    def composite_columns
+      internal_columns.map do |column|
+        trimmed_column(column, /^rk_/, composite_pk_map) ||
+            trimmed_column(column, /^ck_/, composite_ck_map) ||
+            column
+      end.uniq
+    end
+
+    def trimmed_column(column, column_trim, map)
+      column_str = column.to_s
+      if column_str =~ column_trim
+        column_str.gsub(column_trim, '').to_sym.tap do |result_column|
+          map[result_column] = column
+          map[column] = result_column
+        end
+      end
+    end
+
+    def select_clause(select)
+      select = mapped_select_columns(select) if select
+      super(select)
+    end
+
+    def order_by_clause(order_by)
+      order_by = mapped_select_columns(order_by) if order_by
+      super(order_by)
+    end
+
+    def mapped_select_columns(select)
+      select.map do |column|
+        if internal_columns.include?(column)
+          column
+        else
+          mapped_column(column)
+        end
+      end
+    end
+
+    def where_params(clause)
+      updated_clause = clause.inject({}) do |memo, (key, value)|
+        updated_key = key_for_where_params(key)
+        memo.merge!(updated_key => value)
+      end
+
+      missing_keys = Set.new(internal_partition_key - updated_clause.keys)
+      default_clause = composite_defaults.find { |row| (missing_keys ^ row.keys).empty? }
+      updated_clause.merge!(default_clause) if default_clause
+
+      super(updated_clause)
+    end
+
+    def key_for_where_params(key)
+      key.is_a?(ThomasUtils::KeyComparer) ? mapped_key_comparer(key) : mapped_key(key)
+    end
+
+    def mapped_key_comparer(key)
+      mapped_key = key.key.is_a?(Array) ? key.key.map { |part| mapped_ck(part) } : mapped_ck(key.key)
+      key.new_key(mapped_key)
+    end
+
+    def mapped_key(key)
+      composite_pk_map[key] || mapped_ck(key)
+    end
+
+    def mapped_ck(key)
+      composite_ck_map[key] || key
+    end
+
+    def row_composite_default(row)
+      row.inject({}) do |memo, (key, value)|
+        memo.merge!(composite_default_row_key(key) => value)
+      end
+    end
+
+    def composite_default_row_key(key)
+      composite_pk_map[key] || key
+    end
+
+    def row_attributes(row)
+      row = super(row)
+
+      row.inject({}) do |memo, (column, value)|
+        if column =~ /^rk_/ || column =~ /^ck_/
+          memo.merge!(mapped_column(column) => value)
+        else
+          memo.merge!(column => value)
+        end
+      end
+    end
+
+    def mapped_column(column)
+      (composite_ck_map[column] || composite_pk_map[column] || column)
+    end
+
+  end
+end