bigrecord 0.0.7 → 0.0.8

data/README.rdoc

@@ -31,7 +31,15 @@ Beyond this though, there are two basic motivations that almost immediately dema
 
 == Getting Started
 
-Check out the doc/getting_started.rdoc file for more information.
+Check out the guides/getting_started.rdoc file for more information.
+
+== Deployment
+
+Refer to guides/deployment.rdoc for more information.
+
+== Documentation
+
+Please refer to http://openplaces.github.com/bigrecord/
 
 == License
 

data/VERSION

@@ -1 +1 @@
-0.0.7
+0.0.8

data/examples/bigrecord.yml

@@ -1,9 +1,6 @@
 development:
-  adapter: hbase
-  zookeeper_quorum: localhost
-  zookeeper_client_port: 2181
-  drb_host: localhost
-  drb_port: 40000
+  adapter: hbase_rest
+  api_address: http://localhost:8080
 test:
   adapter: hbase
   zookeeper_quorum: localhost

data/guides/deployment.rdoc

@@ -0,0 +1,21 @@
+= Deploying Big Record
+
+Stargate is a new implementation for HBase's web service front-end, and as such, is not currently recommended for deployment.
+
+We here at Openplaces have developed Bigrecord Driver, which uses JRuby to interact with HBase via the native Java API and connect to Bigrecord through the DRb protocol. This method is slightly more complicated to setup, but preliminary benchmarks show that it runs faster (especially for scanner functionality).
+
+== Instructions
+* Your database should already be set up (please refer to the database's own documentation) with the required information known such as the zookeeper quorum/port, etc. in order for Bigrecord to connect to it.
+* Bigrecord Driver (if your database requires it for connecting)
+  * JRuby 1.1.6+ is needed to run Bigrecord Driver.
+
+Install the Bigrecord Driver gem and its dependencies, then start up a DRb server. Please refer the Bigrecord Driver documentation for more detailed instructions. (http://github.com/openplaces/bigrecord/blob/master/bigrecord-driver/README.rdoc)
+
+Edit your bigrecord.yml config file as follows:
+
+ production:
+   adapter: hbase
+   zookeeper_quorum: localhost
+   zookeeper_client_port: 2181
+   drb_host: localhost
+   drb_port: 50001

data/{doc → guides}/getting_started.rdoc

@@ -1,18 +1,29 @@
 = Getting Started
 
-== Requirements
 
-* Database set up (please refer to the database's own documentation) with the required information known such as host, port, username, password, etc. in order for Bigrecord to connect to it.
-* Bigrecord Driver (if your database requires it for connecting)
-  * JRuby 1.1.6+ is needed to run Bigrecord Driver.
+== Setting up HBase and Stargate
 
-== *NOTE* At the time of this writing, Hbase is the only supported database for Bigrecord, and the following instructions are for specific to Hbase.
+To quickly get started with development, you can set up HBase to run as a single server on your local computer, along with Stargate, its RESTful web service front-end.
 
-== Installation
+(1) Download and unpack the most recent release of HBase from http://hadoop.apache.org/hbase/releases.html#Download
 
-(1) Install the Bigrecord Driver gem and its dependencies, then start up a DRb server. Please see the Bigrecord Driver documentation for more detailed instructions. (http://github.com/openplaces/bigrecord/blob/master/bigrecord-driver/README.rdoc)
+(2) Edit (hbase-dir)/conf/hbase-env.sh and uncomment/modify the following line to correspond to your Java home path:
+  export JAVA_HOME=/usr/lib/jvm/java-6-sun
+
+(3) Copy (hbase-dir)/contrib/stargate/hbase-<version>-stargate.jar into <hbase-dir>/lib
+
+(4) Copy all the files in the (hbase-dir)/contrib/stargate/lib folder into <hbase-dir>/lib
+
+(5) Start up HBase:
+  $ (hbase-dir)/bin/start-hbase.sh
+
+(6)Start up Stargate (append "-p 1234" at the end if you want to change the port):
+  $ (hbase-dir)/bin/hbase org.apache.hadoop.hbase.stargate.Main
 
-=== Method 1 (recommended): Install as a gem
+
+== Setting up Bigrecord
+
+(1) Install the Bigrecord Driver gem and its dependencies, then start up a DRb server. Please see the Bigrecord Driver documentation for more detailed instructions. (http://github.com/openplaces/bigrecord/blob/master/bigrecord-driver/README.rdoc)
 
 (2) Add the following line into the Rails::Initializer.run do |config| block:
 
@@ -26,15 +37,14 @@ and run the following command to install all the gems listed for your Rails app:
 
   script/generate bigrecord
 
-(4) Edit the config/bigrecord.yml[.sample] file in your Rails root to the information corresponding to your database install and Bigrecord Driver DRb server.
+(4) Edit the config/bigrecord.yml[.sample] file in your Rails root to the information corresponding to the Stargate server.
 
-=== Method 2: Install as a plugin
+  development:
+    adapter: hbase_rest
+    api_address: http://localhost:8080
 
-(2) Run the following command to install it as a plugin:
+Note: 8080 is the default port that Stargate starts up on. Make sure you modify this if you changed the port from the default.
 
-  script/plugin install git://github.com/openplaces/bigrecord.git
-
-(3) Edit the config/bigrecord.yml[.sample] file in your Rails root to the information corresponding to your database install and Bigrecord Driver DRb server.
 
 == Usage
 
@@ -50,7 +60,7 @@ This will add a model in app/models and a migration file in db/bigrecord_migrate
 
 Creates a Bigrecord specific migration and adds it into db/bigrecord_migrate
 
-=== Migration File
+=== {BigRecord::Migration Migration File}
 
 Although column-oriented databases are generally schema-less, certain ones (like Hbase) require the creation of tables and column families ahead of time. The individual columns, however, are defined in the model itself and can be modified dynamically without the need for migrations.
 
@@ -81,7 +91,7 @@ Run the following rake task to migrate your tables and column families up to the
 
   rake bigrecord:migrate
 
-=== Column and Attribute Definition
+=== {BigRecord::ConnectionAdapters::Column Column and Attribute Definition}
 
 Now that you have your tables and column families all set up, you can begin adding columns to your model. The following is an example of a model named book.rb
 
@@ -94,7 +104,7 @@ Now that you have your tables and column families all set up, you can begin addi
 
 This simple model defines 4 columns of type string. An important thing to notice here is that the first column 'attribute:title' had the column family prepended to it. This is identical to just passing the symbol :title to the column method, and the default behaviour is to prepend the column family (attribute) automatically if one is not defined. Furthermore, in Hbase, there's the option of storing collections for a given column. This will return an array for the links attribute on a Book record.
 
-=== Associations
+=== {BigRecord::BrAssociations Associations}
 
 There are also associations available in Bigrecord, as well as the ability to associate to Activerecord models. The following are a few models demonstrating this:
 
@@ -118,7 +128,7 @@ In this example, an Animal is related to Zoo and Trainer. Both Animal and Zoo ar
 
 Once the association columns are defined, you define the associations themselves with either belongs_to_bigrecord or belongs_to_many and defining the :foreign_key (this is required for all associations).
 
-=== Specifying return columns
+=== {BigRecord::ConnectionAdapters::View Specifying return columns}
 
 There are two ways to define specific columns to be returned with your models: 1) at the model level and 2) during the query.
 
@@ -158,4 +168,6 @@ Note: A Bigrecord model will return all the columns within the default column fa
 
 As you may have noticed, this functionality is synonymous with the :select option in Activerecord.
 
+=== {BigRecord::Embedded Embedded Records}
+
 === At this point, usage patterns for a Bigrecord model are similar to that of an Activerecord model, and much of that documentation applies as well. Please refer to those and see if they work!

data/lib/big_record/abstract_base.rb

@@ -139,6 +139,8 @@ module BigRecord
     # attributes but not yet saved (pass a hash with key names matching the associated table column names).
     # In both instances, valid attribute keys are determined by the column names of the associated table --
     # hence you can't have attributes that aren't part of the table columns.
+    #
+    # @param [Hash] Optional hash argument consisting of keys that match the names of the columns, and their values.
     def initialize(attrs = nil)
       preinitialize(attrs)
       @attributes = attributes_from_column_definition
@@ -157,6 +159,9 @@ module BigRecord
 
     # Safe version of attributes= so that objects can be instantiated even
     # if columns are removed.
+    #
+    # @param [Hash] Attribute hash consisting of the column name and their values.
+    # @param [true, false] Pass the attributes hash through {#remove_attributes_protected_from_mass_assignment}
     def safe_attributes=(new_attributes, guard_protected_attributes = true)
       return if new_attributes.nil?
       attributes = new_attributes.dup
@@ -189,10 +194,16 @@ module BigRecord
       read_attribute(attr_name)
     end
 
+    # Default to_s method that just returns invokes the {#id} method
+    #
+    # @return [String] The row identifier/id of the record
     def to_s
       id
     end
 
+    # Get the attributes hash of the object.
+    #
+    # @return [Hash] a duplicated attributes hash
     def attributes()
       @attributes.dup
     end
@@ -201,6 +212,8 @@ module BigRecord
     # The optional options argument is passed to find when reloading so you
     # may do e.g. record.reload(:lock => true) to reload the same record with
     # an exclusive row lock.
+    #
+    # @return Itself with reloaded attributes.
     def reload(options = nil)
       @attributes.update(self.class.find(self.id, options).instance_variable_get('@attributes'))
       self
@@ -262,12 +275,18 @@ module BigRecord
 
     # Returns true if the specified +attribute+ has been set by the user or by a database load and is neither
     # nil nor empty? (the latter only applies to objects that respond to empty?, most notably Strings).
+    #
+    # @return [String, Symbol] Name of an attribute.
+    # @return [true,false] Whether that attribute exists.
     def attribute_present?(attribute)
       value = read_attribute(attribute)
       !value.blank? or value == 0
     end
 
     # Returns true if the given attribute is in the attributes hash
+    #
+    # @return [String, Symbol] Name of an attribute.
+    # @return [true,false] Whether that attribute exists in the attributes hash.
     def has_attribute?(attr_name)
       @attributes.has_key?(attr_name.to_s)
     end
@@ -332,6 +351,9 @@ module BigRecord
       @attributes.freeze; self
     end
 
+    # Checks whether the object has had its attributes hash frozen.
+    #
+    # @return [true,false]
     def frozen?
       @attributes.frozen?
     end
@@ -345,7 +367,8 @@ module BigRecord
       write_attribute(self.class.primary_key, value)
     end
 
-    # Returns true if this object hasn't been saved yet -- that is, a record for the object doesn't exist yet.
+    # Returns true if this object hasn't been saved yet -- that is, a record for the object doesn't
+    # yet exist in the data store.
     def new_record?
       false
     end
@@ -386,6 +409,7 @@ module BigRecord
       raise NotImplemented
     end
 
+    # Returns the connection adapter of the current session.
     def connection
       self.class.connection
     end
@@ -395,6 +419,7 @@ module BigRecord
       @readonly == true
     end
 
+    # Sets the record to be readonly
     def readonly! #:nodoc:
       @readonly = true
     end
@@ -466,15 +491,12 @@ module BigRecord
       end
     end
 
-    # Generate a new id. Override this to use custom ids.
+    # Generate a new id with the UUIDTools library.
+    # Override this method to use another id generator.
     def generate_new_id
       UUIDTools::UUID.random_create.to_s
     end
 
-    def self.random_id #not necessarily unique! -- this is strictly for 'stumbling', not for assigning to new entities
-      [8,4,4,4,12].map{|l| "%0#{l}x" % rand(1 << l*4) }.join('-')
-    end
-
     # Initializes the attributes array with keys matching the columns from the linked table and
     # the values matching the corresponding default value of that column, so
     # that a new instance, or one populated from a passed-in Hash, still has all the attributes
@@ -610,6 +632,9 @@ module BigRecord
       end
     end
 
+    # Removes any attributes from the argument hash that have been declared as protected
+    # from mass assignment. See the {#attr_protected} and {#attr_accessible} macros to define
+    # these attributes.
     def remove_attributes_protected_from_mass_assignment(attributes)
       safe_attributes =
         if self.class.accessible_attributes.nil? && self.class.protected_attributes.nil?
@@ -679,7 +704,7 @@ module BigRecord
 private
     # Called on first read access to any given column and generates reader
     # methods for all columns in the columns_hash if
-    # ActiveRecord::Base.generate_read_methods is set to true.
+    # BigRecord::Base.generate_read_methods is set to true.
     def define_read_methods
       self.class.columns_hash.each do |name, column|
         unless respond_to_without_attributes?(name)
@@ -692,7 +717,7 @@ private
       end
     end
 
-    # Define an attribute reader method.  Cope with nil column.
+    # Define an attribute reader method. Cope with a nil column.
     def define_read_method(symbol, attr_name, column)
       cast_code = column.type_cast_code('v') if column
       access_code = cast_code ? "(v=@attributes['#{attr_name}']) && #{cast_code}" : "@attributes['#{attr_name}']"
@@ -760,22 +785,9 @@ private
         raise NotImplemented
       end
 
-#      # Returns a string like 'Post id:integer, title:string, body:text'
-#      def inspect
-#        if self == Base
-#          super
-#        elsif abstract_class?
-#          "#{super}(abstract)"
-#        elsif table_exists?
-#          attr_list = columns.map { |c| "#{c.name}: #{c.type}" } * ', '
-#          "#{super}(#{attr_list})"
-#        else
-#          "#{super}(Table doesn't exist)"
-#        end
-#      end
-
-      # Log and benchmark multiple statements in a single block. Example:
+      # Log and benchmark multiple statements in a single block.
       #
+      # @example
       #   Project.benchmark("Creating project") do
       #     project = Project.create("name" => "stuff")
       #     project.create_manager("name" => "David")
@@ -816,7 +828,7 @@ private
       end
 
       # Override this method in the subclasses to add new columns. This is different from ActiveRecord because
-      # the number of columns in a Hbase table is variable.
+      # the number of columns in an Hbase table is variable.
       def columns
         @columns = columns_hash.values
       end
@@ -868,6 +880,15 @@ private
         end
       end
 
+      # Macro for defining a new column for a model. Invokes {create_column} and
+      # adds the new column into the model's column hash.
+      #
+      # @param type [Symbol, String] Column type as defined in the source of {ConnectionAdapters::Column#klass}
+      # @option options [true,false] :collection Whether this column is a collection.
+      # @option options [String] :alias Define an alias for the column that cannot be inferred. By default, 'attribute:name' will be aliased to 'name'.
+      # @option options [String] :default Default value to set for this column.
+      #
+      # @return [ConnectionAdapters::Column] The column object created.
       def column(name, type, options={})
         name = name.to_s
 
@@ -884,10 +905,6 @@ private
         c
       end
 
-      def create_column(name, type, options)
-        ConnectionAdapters::Column.new(name, type, options)
-      end
-
       # Define aliases to the fully qualified attributes
       def alias_attribute(alias_name, fully_qualified_name)
         self.class_eval <<-EOF
@@ -1040,6 +1057,11 @@ private
         @content_columns = nil
       end
 
+      # Creates a {ConnectionAdapters::Column} object.
+      def create_column(name, type, options)
+        ConnectionAdapters::Column.new(name, type, options)
+      end
+
       def default_columns
         raise NotImplemented
       end

data/lib/big_record/action_view_extensions.rb

@@ -11,7 +11,7 @@ class Object
   end
 end
 
-module ActionView
+module ActionView #:nodoc:
   module Helpers
     class InstanceTag #:nodoc:
 

data/lib/big_record/base.rb

@@ -33,7 +33,6 @@ module BigRecord
       end
     end
 
-#  protected
     # Returns the value of the attribute identified by <tt>attr_name</tt> after it has been typecast (for example,
     # "2004-12-12" in a data column is cast to a date object, like Date.new(2004, 12, 12)).
     def read_attribute(attr_name, options={})
@@ -81,6 +80,7 @@ module BigRecord
       end
     end
 
+    # Read an attribute that defines a column family.
     def read_family_attributes(attr_name)
       attr_name = attr_name.to_s
       column = column_for_attribute(attr_name)
@@ -197,6 +197,7 @@ module BigRecord
 
   protected
 
+    # Invoke {#create} if {#new_record} returns true, otherwise it's an {#update}
     def create_or_update
       raise ReadOnlyRecord if readonly?
       result = new_record? ? create : update
@@ -230,11 +231,7 @@ module BigRecord
   public
     class << self
 
-      # Replaced with: class_inheritable_accessor :default_family (line 46)
-      # def default_family
-      #   "attribute"
-      # end
-
+      # Return the name of the primary key. Defaults to "id".
       def primary_key
         @primary_key ||= "id"
       end
@@ -245,12 +242,12 @@ module BigRecord
       end
 
       # HBase scanner utility -- scans the table and executes code on each record
-      # Example:
+      #
+      # @example
       #    Entity.scan(:batch_size => 200) {|e|puts "#{e.name} is a child!" if e.parent}
       #
-      # Parameters:
-      #    batch_size - number of records to retrieve from database with each scan iteration.
-      #    code - the code to execute (see example above for syntax)
+      # @option options [Integer] :batch_size - number of records to retrieve from database with each scan iteration.
+      # @option options [Block] :code - the code to execute (see example above for syntax)
       #
       def scan(options={}, &code)
         options = options.dup
@@ -327,10 +324,10 @@ module BigRecord
       # The arguments may also be given as arrays in which case the update method is called for each pair of +id+ and
       # +attributes+ and an array of objects is returned.
       #
-      # Example of updating one record:
+      # @example of updating one record:
       #   Person.update(15, {:user_name => 'Samuel', :group => 'expert'})
       #
-      # Example of updating multiple records:
+      # @example of updating multiple records:
       #   people = { 1 => { "first_name" => "David" }, 2 => { "first_name" => "Jeremy"} }
       #   Person.update(people.keys, people.values)
       def update(id, attributes)
@@ -378,7 +375,7 @@ module BigRecord
       # calling the destroy method). Example:
       #   Post.delete_all "person_id = 5 AND (category = 'Something' OR category = 'Else')"
       #
-      # TODO: take into consideration the conditions
+      # @todo take into consideration the conditions
       def delete_all(conditions = nil)
         connection.get_consecutive_rows(table_name, nil, nil, ["#{default_family}:"]).each do |row|
           connection.delete(table_name, row["id"])
@@ -398,18 +395,34 @@ module BigRecord
         @table_name = name.to_s
       end
 
+      # Get the default column family used to store attributes that have no column family set explicitly.
+      #
+      # Defaults to "attribute"
       def default_family
         (superclass == BigRecord::Base) ? (@default_family ||= "attribute") : superclass.default_family
       end
 
+      # Set the default column family used to store attributes that have no column family set explicitly.
+      #
+      # @example
+      #   set_default_family :attr  # instead of using :attribute as the default.
       def set_default_family(name)
         @default_family = name.to_s
       end
 
+      # @return [Class] The base class which inherits BigRecord::Base directly.
       def base_class
         (superclass == BigRecord::Base) ? self : superclass.base_class
       end
 
+      # Macro for defining a named view to a list of columns.
+      #
+      # @param [String, Symbol] name Give it an arbitrary name.
+      # @param [Array<String, Symbol>] columns List of columns to associate to this view. Can use column aliases or fully qualified names.
+      #
+      # @example
+      #   view :front_page, :name, :title, :description
+      #   view :summary, ["attribute:name", "attribute:title"]
       def view(name, *columns)
         name = name.to_sym
         @views_hash ||= default_views
@@ -420,14 +433,17 @@ module BigRecord
         @views_hash[name] = ConnectionAdapters::View.new(name, columns.flatten, self)
       end
 
+      # Get a list of all the views defined by the {view} macro for the model.
       def views
         @views ||= views_hash.values
       end
 
+      # Get a list of view names defined by {view}.
       def view_names
         @view_names ||= views_hash.keys
       end
 
+      # Get the full hash of views consisting of the name as keys, and the {ConnectionAdapters::View} views.
       def views_hash
         unless @all_views_hash
           # add default hbase columns
@@ -445,10 +461,12 @@ module BigRecord
         @all_views_hash
       end
 
+      # Default columns to create with the model, such as primary key.
       def default_columns
         {primary_key => ConnectionAdapters::Column.new(primary_key, 'string')}
       end
 
+      # @see BigRecord::AbstractBase.column
       def column(name, type, options={})
         name = name.to_s
         name = "#{self.default_family}:#{name}" unless (name =~ /:/)
@@ -456,21 +474,18 @@ module BigRecord
         super(name, type, options)
       end
 
+      # Return the hash of default views which consist of all columns and the :default named views.
       def default_views
         {:all=>ConnectionAdapters::View.new('all', nil, self), :default=>ConnectionAdapters::View.new('default', nil, self)}
       end
 
-      def find_all_by_id(ids, options={})
-        ids.inject([]) do |result, id|
-          begin
-            result << find_one(id, options)
-          rescue BigRecord::RecordNotFound => e
-          end
-          result
-        end
-      end
-
-      # return the list of columns to get from hbase
+      # Return the list of fully qualified column names, i.e. ["family:qualifier"].
+      #
+      # Returns the column names based on the options argument in order of
+      # :columns,then :view, i.e. disregards :view if :columns is defined.
+      #
+      # @option options [Array<String, Symbol>] :columns List of fully qualified column names or column aliases.
+      # @option options [String, Symbol] :view The name of the view as defined with {view}.
       def columns_to_find(options={})
         c =
           if options[:columns]
@@ -599,6 +614,16 @@ module BigRecord
         end
       end
 
+      def find_all_by_id(ids, options={})
+        ids.inject([]) do |result, id|
+          begin
+            result << find_one(id, options)
+          rescue BigRecord::RecordNotFound => e
+          end
+          result
+        end
+      end
+
       # Add the missing cells to the raw record and set them to nil. We know that it's
       # nil because else we would have received those cells. That way, when the value of
       # one of these cells will be requested by the client we won't try to lazy load it.