json_record 1.0.0

data/MIT_LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2010 Brian Durand
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.rdoc

@@ -0,0 +1,88 @@
+= JSON Record
+
+The purpose of this code is to add the ability to represent complex documents in ActiveRecord by using JSON to serialize the documents to a database field. This can be especially useful if you need a flexible schema, but don't want or have the means to utilize one of the new schemaless data stores that all the cool kids are talking about.
+
+After all, relational databases are pretty rock solid and widely available technology. If you can't get the cool new thing installed, or if you just feel safe sticking with what you know, this gem may work for you. As an added advantage, it is just an extension on top of ActiveRecord, so you can still use all the features of ActiveRecord and add the schemaless functionality only to models where it makes sense.
+
+== Serialized Fields
+
+To define a complex document field, simply add this code to your ActiveRecord model definition:
+
+serialize_to_json(:json_data) do |schema|
+  schema.key :name
+  schema.key :value, Integer
+end
+
+This will define for you accessors on your model for name and value and serialize those value in a database columns named json_data. These attributes will work just like other ActiveRecord attributes, so you will be able to track changes to them, include them in mass assignments, etc. Of course, that's not all that interesting since you could easily enough have added columns for name and value.
+
+== Embedded Documents
+
+To make you flexible schema really powerful, add some embedded documents to it. Embedded documents are Ruby classes that inherit from JsonRecord::EmbeddedDocument. They work very much like traditional ActiveRecord objects, except that instead of being serialized in a separate table, they are embedded right in the JSON field of their parent record. They can be used to replace has_many and has_one associations and can be far easier to work with.
+
+Embedded documents have their own schema that is serialized to JSON. This schema can also contain embedded documents allowing you to easily create very rich data structures all with only one database table. And because there is only one table, you don't need to worry at all about ensuring your changes to embedded documents are saved along with the parent record.
+
+== Example
+
+  class Post < ActiveRecord::Base
+    serialize_to_json(:json_data) do |schema|
+      schema.key :title, :required => true
+      schema.key :body, :required => true
+      schema.key :author, Person, :required => true
+      schema.many :comments, Comment
+    end
+  end
+
+  class Person < JsonRecord::EmbeddedDocument
+    schema.key :first_name, :required => true
+    schema.key :last_name
+  end
+
+  class Comment < JsonRecord::EmbeddedDocument
+    schema.key :author, Person, :required => true
+    schema.key :body, :required => true
+    schema.many :replies, Comment
+  end
+
+Create a new post with a title and author:
+
+post = Post.create!(:title => "What I think",
+                    :body => "Stuff is good",
+                    :author => {:first_name => "John", :last_name => "Doe"})
+
+Change the authors first name:
+
+  post.author.first_name = "Bill"
+
+Add a couple of comments:
+
+  post.comments.build(:author => {:first_name => "Tony"}, :body => "I like it")
+  post.comments.build(:author => {:first_name => "Jack"}, :body => "I don't like it")
+
+Add a reply:
+
+  post.comments.first.replies.build(:author => {:first_name => "Ralph"}, :body => "You're and idiot")
+
+And save it all:
+
+  post.save
+
+Unlike with traditional association, you don't need any after_save callbacks to ensure that the associations are saved. If we want to remove the last comment, all we need to do is:
+
+  post.comments.pop
+  post.save
+
+== Limitations
+
+One thing you cannot do is index the fields in the serialized JSON. If you need to be able to search on those fields, you'll need a separate search engine (i.e. Solr or Sphinx). Or, you could just move it out of the JSON fields and make it a regular database column with an index on it. The interface will be exactly the same.
+
+In order to conserve space and increase performance, blank values are not serialized to JSON. One side effect is that you cannot have fields with the empty string as the value. Also, if you look at the JSON stored in the database, you won't be able to deduce the schema since blank fields will be missing entirely. If you have any fields that are used to store Arrays or Hashes will never be nil and will always be initialized with an empty Array or Hash.
+
+== Details, details, details
+
+For optimal performance when working with JSON, you should really have the +json+ gem installed. This gem does not have a direct dependency on +json+ since it will work just fine without it, but if it is missing, you'll get a warning about it in the ActiveRecord log.
+
+For performance, attributes from a JSON serialized field are only loaded when they are accessed. When a record is saved, the JSON attributes are translated back to JSON and stored in the serialized field. If a field is encountered in the JSON that has not been declared in the schema, it will persist, but will not be accessible via an accessor.
+
+The fields you are using to store JSON must be large enough to handle any document. You should provide a :length attribute in your migration to ensure that text fields are long enough. By default, MySQL, for instance, will create a TEXT field limited to 32K. What you really want is a MEDIUMTEXT or LONGTEXT field.
+
+Since JSON can be kind of wordy and take up a lot more space than a traditional column based approach, you can also specify that the JSON should be compressed when it is stored in the database. To do this, simply create the JSON column as a binary column type instead of a text column. This is the recommended set up unless you need to browse through your database outside of your Ruby application.

data/Rakefile

@@ -0,0 +1,44 @@
+require 'rubygems'
+require 'rake'
+require 'rake/rdoctask'
+
+desc 'Default: run unit tests.'
+task :default => :test
+
+begin
+  require 'spec/rake/spectask'
+  desc 'Test json_record.'
+  Spec::Rake::SpecTask.new(:test) do |t|
+    t.spec_files = FileList.new('spec/**/*_spec.rb')
+  end
+rescue LoadError
+  tast :test do
+    STDERR.puts "You must have rspec >= 1.2.9 to run the tests"
+  end
+end
+
+desc 'Generate documentation for json_record.'
+Rake::RDocTask.new(:rdoc) do |rdoc|
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.options << '--title' << 'JSON Record' << '--line-numbers' << '--inline-source' << '--main' << 'README.rdoc'
+  rdoc.rdoc_files.include('README.rdoc')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+end
+
+begin
+  require 'jeweler'
+  Jeweler::Tasks.new do |gem|
+    gem.name = "json_record"
+    gem.summary = %Q{ActiveRecord support for mapping complex documents within a single RDBMS record via JSON serialization.}
+    gem.email = "brian@embellishedvisions.com"
+    gem.homepage = "http://github.com/bdurand/json_record"
+    gem.authors = ["Brian Durand"]
+  
+    gem.add_dependency('activerecord', '>= 2.2.2', '< 3.0')
+    gem.add_development_dependency('rspec', '>= 1.2.9')
+    gem.add_development_dependency('jeweler')
+  end
+
+  Jeweler::GemcutterTasks.new
+rescue LoadError
+end

data/VERSION

@@ -0,0 +1 @@
+1.0.0

data/init.rb

@@ -0,0 +1 @@
+require 'json_record'

data/json_record.gemspec

@@ -0,0 +1,72 @@
+# Generated by jeweler
+# DO NOT EDIT THIS FILE DIRECTLY
+# Instead, edit Jeweler::Tasks in Rakefile, and run the gemspec command
+# -*- encoding: utf-8 -*-
+
+Gem::Specification.new do |s|
+  s.name = %q{json_record}
+  s.version = "1.0.0"
+
+  s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
+  s.authors = ["Brian Durand"]
+  s.date = %q{2010-02-07}
+  s.email = %q{brian@embellishedvisions.com}
+  s.extra_rdoc_files = [
+    "README.rdoc"
+  ]
+  s.files = [
+    "MIT_LICENSE",
+     "README.rdoc",
+     "Rakefile",
+     "VERSION",
+     "init.rb",
+     "json_record.gemspec",
+     "lib/json_record.rb",
+     "lib/json_record/attribute_methods.rb",
+     "lib/json_record/embedded_document.rb",
+     "lib/json_record/embedded_document_array.rb",
+     "lib/json_record/field_definition.rb",
+     "lib/json_record/json_field.rb",
+     "lib/json_record/schema.rb",
+     "lib/json_record/serialized.rb",
+     "spec/embedded_document_array_spec.rb",
+     "spec/embedded_document_spec.rb",
+     "spec/field_definition_spec.rb",
+     "spec/serialized_spec.rb",
+     "spec/spec_helper.rb",
+     "spec/test_models.rb"
+  ]
+  s.homepage = %q{http://github.com/bdurand/json_record}
+  s.rdoc_options = ["--charset=UTF-8"]
+  s.require_paths = ["lib"]
+  s.rubygems_version = %q{1.3.5}
+  s.summary = %q{ActiveRecord support for mapping complex documents within a single RDBMS record via JSON serialization.}
+  s.test_files = [
+    "spec/embedded_document_array_spec.rb",
+     "spec/embedded_document_spec.rb",
+     "spec/field_definition_spec.rb",
+     "spec/serialized_spec.rb",
+     "spec/spec_helper.rb",
+     "spec/test_models.rb"
+  ]
+
+  if s.respond_to? :specification_version then
+    current_version = Gem::Specification::CURRENT_SPECIFICATION_VERSION
+    s.specification_version = 3
+
+    if Gem::Version.new(Gem::RubyGemsVersion) >= Gem::Version.new('1.2.0') then
+      s.add_runtime_dependency(%q<activerecord>, [">= 2.2.2", "< 3.0"])
+      s.add_development_dependency(%q<rspec>, [">= 1.2.9"])
+      s.add_development_dependency(%q<jeweler>, [">= 0"])
+    else
+      s.add_dependency(%q<activerecord>, [">= 2.2.2", "< 3.0"])
+      s.add_dependency(%q<rspec>, [">= 1.2.9"])
+      s.add_dependency(%q<jeweler>, [">= 0"])
+    end
+  else
+    s.add_dependency(%q<activerecord>, [">= 2.2.2", "< 3.0"])
+    s.add_dependency(%q<rspec>, [">= 1.2.9"])
+    s.add_dependency(%q<jeweler>, [">= 0"])
+  end
+end
+

data/lib/json_record/attribute_methods.rb

@@ -0,0 +1,55 @@
+module JsonRecord
+  # Internal methods for reading and writing fields serialized from JSON.
+  module AttributeMethods
+    # Read a field. The field param must be a FieldDefinition and the context should be the record
+    # which is being read from.
+    def read_attribute (field, context)
+      if field.multivalued?
+        arr = json_attributes[field.name]
+        unless arr
+          arr = EmbeddedDocumentArray.new(field.type, context)
+          json_attributes[field.name] = arr
+        end
+        return arr
+      else
+        val = json_attributes[field.name]
+        if val.nil? and !field.default.nil?
+          val = field.default.dup rescue field.default
+          json_attributes[field.name] = val
+        end
+        return val
+      end
+    end
+
+    # Write a field. The field param must be a FieldDefinition and the context should be the record
+    # which is being read from. Track_changes indicates if changes to the object should be tracked.
+    def write_attribute (field, val, track_changes, context)
+      if field.multivalued?
+        val = val.values if val.is_a?(Hash)
+        json_attributes[field.name] = EmbeddedDocumentArray.new(field.type, context, val)
+      else
+        old_value = read_attribute(field, context)
+        converted_value = field.convert(val)
+        converted_value.parent = context if converted_value.is_a?(EmbeddedDocument)
+        unless old_value == converted_value
+          if track_changes
+            changes = changed_attributes
+            if changes.include?(field.name)
+              changes.delete(field.name) if converted_value == changes[field.name]
+            else
+              old_value = (old_value.clone rescue old_value) unless old_value.nil?
+              changes[field.name] = old_value
+            end
+          end
+          unless converted_value.nil?
+            json_attributes[field.name] = converted_value
+          else
+            json_attributes.delete(field.name)
+          end
+        end
+        context.attributes_before_type_cast[field.name] = val
+      end
+      return val
+    end
+  end
+end

data/lib/json_record/embedded_document.rb

@@ -0,0 +1,155 @@
+module JsonRecord
+  # OK, this is ugly, but necessary to get ActiveRecord::Errors to be compatible with
+  # EmbeddedDocument. This will all be fixed with Rails 3 and ActiveModel. Until then
+  # we'll just live with this.
+  module ActiveRecordStub #:nodoc:
+    def self.included (base)
+      base.extend(ClassMethods)
+    end
+    
+    module ClassMethods
+      def human_name (options = {})
+        name.split('::').last.humanize
+      end
+      
+      def human_attribute_name (attribute, options = {})
+        attribute.to_s.humanize
+      end
+      
+      def self_and_descendants_from_active_record
+        [self]
+      end
+      
+      def self_and_descendents_from_active_record
+        [self]
+      end
+    end
+    
+    def deprecated_callback_method (*args)
+    end
+    
+    private
+    def save (*args); end;
+    def save! (*args); end;
+    def destroy (*args); end;
+    def create (*args); end;
+    def update (*args); end;
+    def new_record?; false; end;
+  end
+  
+  # Subclasses of EmbeddedDocument can be used as the type for keys or many field definitions
+  # in Schema. Embedded documents are then extensions of the schema. In this way, complex
+  # documents represented in JSON can be deserialized as complex objects.
+  class EmbeddedDocument
+    include ActiveRecordStub
+    include ActiveRecord::Validations
+    include AttributeMethods
+    
+    class_inheritable_reader :schema
+    
+    class << self
+      # Define a field for the schema. This is a shortcut for calling schema.key.
+      # See Schema#key for details. 
+      def key (name, *args)
+        write_inheritable_attribute(:schema, Schema.new(self, nil)) unless schema
+        schema.key(name, *args)
+      end
+      
+      # Define a multivalued field for the schema. This is a shortcut for calling schema.many.
+      # See Schema#many for details. 
+      def many (name, *args)
+        write_inheritable_attribute(:schema, Schema.new(self, nil)) unless schema
+        schema.many(name, *args)
+      end
+    end
+    
+    # The parent object of the document.
+    attr_accessor :parent
+    
+    # Create an embedded document with the specified attributes.
+    def initialize (attrs = {})
+      @attributes = {}
+      @json_attributes = {}
+      attrs.each_pair do |name, value|
+        field = schema.fields[name.to_s] || FieldDefinition.new(name, :type => value.class)
+        write_attribute(field, value, false, self)
+      end
+    end
+    
+    # Get the attributes of the document.
+    def attributes
+      @json_attributes.reject{|k,v| !schema.fields.include?(k)}
+    end
+    
+    # Get the attribute values of the document before they were type cast.
+    def attributes_before_type_cast
+      @attributes
+    end
+    
+    # Determine if the document has been changed.
+    def changed?
+      !changed_attributes.empty?
+    end
+
+    # Get the list of attributes changed.
+    def changed
+      changed_attributes.keys
+    end
+
+    # Get a list of changes to the document.
+    def changes
+      changed.inject({}) {|h, attr| h[attr] = attribute_change(attr); h}
+    end
+    
+    def to_json (*args)
+      @json_attributes.to_json(*args)
+    end
+    
+    def eql? (val)
+      val.class == self.class && val.attributes == attributes && val.parent == parent
+    end
+    
+    def == (val)
+      eql?(val)
+    end
+    
+    def hash
+      attributes.hash + parent.hash
+    end
+    
+    protected
+    
+    def json_attributes
+      @json_attributes
+    end
+    
+    def read_json_attribute (json_field_name, field)
+      read_attribute(field, self)
+    end
+    
+    def write_json_attribute (json_field_name, field, value, track_changes)
+      write_attribute(field, value, track_changes, self)
+    end
+  
+    def changed_attributes
+      @changed_attributes ||= {}
+    end
+    
+    def read_attribute_before_type_cast (name)
+      @attributes[name.to_s]
+    end
+    
+    def attribute_changed? (name)
+      changed_attributes.include?(name.to_s)
+    end
+    
+    def attribute_change (name)
+      name = name.to_s
+      [changed_attributes[name], read_json_attribute(nil, schema.fields[name])] if attribute_changed?(name)
+    end
+    
+    def attribute_was (name)
+      changed_attributes[name.to_s]
+    end
+  end
+end

data/lib/json_record/embedded_document_array.rb

@@ -0,0 +1,54 @@
+module JsonRecord
+  # This is an array of EmbeddedDocument objects. All elments of the array must be of the same class
+  # and all belong to the same parent. If an array of hashes are passed in, they will all be converted
+  # to EmbeddedDocument objects of the class specified.
+  class EmbeddedDocumentArray < Array
+    def initialize (klass, parent, objects = [])
+      @klass = klass
+      @parent = parent
+      objects = [] unless objects
+      objects = [objects] unless objects.is_a?(Array)
+      objects = objects.collect do |obj|
+        obj = @klass.new(obj) if obj.is_a?(Hash)
+        if obj.is_a?(@klass)
+          obj.parent = parent
+          obj
+        else
+          raise ArgumentError.new("#{obj.inspect} is not a #{@klass}") unless obj.is_a?(@klass)
+        end
+      end
+      super(objects)
+    end
+    
+    # Append an object to the array. The object must either be an EmbeddedDocument of the
+    # correct class, or a Hash.
+    def << (obj)
+      obj = @klass.new(obj) if obj.is_a?(Hash)
+      raise ArgumentError.new("#{obj.inspect} is not a #{@klass}") unless obj.is_a?(@klass)
+      obj.parent = @parent
+      super(obj)
+    end
+    
+    # Concatenate an array of objects to the array. The objects must either be an EmbeddedDocument of the
+    # correct class, or a Hash.
+    def concat (objects)
+      objects = objects.collect do |obj|
+        obj = @klass.new(obj) if obj.is_a?(Hash)
+        raise ArgumentError.new("#{obj.inspect} is not a #{@klass}") unless obj.is_a?(@klass)
+        obj.parent = @parent
+        obj
+      end
+      super(objects)
+    end
+    
+    # Similar add an EmbeddedDocument to the array and return the object. If the object passed
+    # in is a Hash, it will be used to make a new EmbeddedDocument.
+    def build (obj)
+      obj = @klass.new(obj) if obj.is_a?(Hash)
+      raise ArgumentError.new("#{obj.inspect} is not a #{@klass}") unless obj.is_a?(@klass)
+      obj.parent = @parent
+      self << obj
+      obj
+    end
+  end
+end

data/lib/json_record/field_definition.rb

@@ -0,0 +1,87 @@
+module JsonRecord
+  # A definition of a JSON field in a Schema.
+  class FieldDefinition
+    BOOLEAN_MAPPING = {
+      true => true, 'true' => true, 'TRUE' => true, 'True' => true, 't' => true, 'T' => true, '1' => true, 1 => true, 1.0 => true,
+      false => false, 'false' => false, 'FALSE' => false, 'False' => false, 'f' => false, 'F' => false, '0' => false, 0 => false, 0.0 => false, nil => false
+    }
+    
+    attr_reader :name, :type
+    
+    # Define a field. Options should include :type with the class of the field. Other options available are
+    # :multivalued and :default.
+    def initialize (name, options = {})
+      @name = name.to_s
+      @type = options[:type] || String
+      @multivalued = !!options[:multivalued]
+      @default = options[:default]
+      if [Hash, Array].include?(@type) and @default.nil?
+        @default = @type.new
+      end
+    end
+    
+    # Get the default value.
+    def default
+      (@default.dup rescue @default )if @default
+    end
+    
+    # Indicates the field is multivalued.
+    def multivalued?
+      @multivalued
+    end
+    
+    # Convert a value to the proper class for storing it in the field. If the value can't be converted,
+    # the original value will be returned. Blank values are always translated to nil. Hashes will be converted
+    # to EmbeddedDocument objects if the field type extends from EmbeddedDocument.
+    def convert (val)
+      return nil if val.blank?
+      if @type == String
+        return val.to_s
+      elsif @type == Integer
+        return Kernel.Integer(val) rescue val
+      elsif @type == Float
+        return Kernel.Float(val) rescue val
+      elsif @type == Boolean
+        v = BOOLEAN_MAPPING[val]
+        v = val.to_s.downcase == 'true' if v.nil? # Check all mixed case spellings for true
+        return v
+      elsif @type == Date
+        if val.is_a?(Date)
+          return val
+        elsif val.is_a?(Time)
+          return val.to_date
+        else
+          return Date.parse(val.to_s) rescue val
+        end
+      elsif @type == Time
+        if val.is_a?(Time)
+          return Time.at((val.to_i / 60) * 60).utc
+        else
+          return Time.parse(val).utc rescue val
+        end
+      elsif @type == DateTime
+        if val.is_a?(DateTime)
+          return val.utc
+        else
+          return DateTime.parse(val).utc rescue val
+        end
+      elsif @type == Array
+        val = [val] unless val.is_a?(Array)
+        raise ArgumentError.new("#{name} must be an Array") unless val.is_a?(Array)
+        return val
+      elsif @type == Hash
+        raise ArgumentError.new("#{name} must be a Hash") unless val.is_a?(Hash)
+        return val
+      else
+        if val.is_a?(@type)
+          val
+        elsif val.is_a?(Hash) and (@type < EmbeddedDocument)
+          return @type.new(val)
+        else
+          raise ArgumentError.new("#{name} must be a #{@type}")
+        end
+      end
+    end
+    
+  end
+end

data/lib/json_record/json_field.rb

@@ -0,0 +1,59 @@
+require 'zlib'
+
+module JsonRecord
+  class JsonField
+    include AttributeMethods
+    
+    def initialize (record, name, schemas)
+      @record = record
+      @name = name
+      @schemas = schemas
+      @attributes = nil
+      @compressed = record.class.columns_hash[name].type == :binary
+    end
+    
+    def serialize
+      if @attributes
+        stripped_attributes = {}
+        @attributes.each_pair{|k, v| stripped_attributes[k] = v unless v.blank?}
+        json = stripped_attributes.to_json
+        json = Zlib::Deflate.deflate(json) if json and @compressed
+        @record[@name] = json
+      end
+    end
+    
+    def deserialize
+      @attributes = {}
+      @schemas.each do |schema|
+        schema.fields.values.each do |field|
+          @attributes[field.name] = field.multivalued? ? EmbeddedDocumentArray.new(field.type, self) : field.default
+        end
+      end
+      
+      unless @record[@name].blank?
+        json = @record[@name]
+        json = Zlib::Inflate.inflate(json) if @compressed
+        ActiveSupport::JSON.decode(json).each_pair do |attr_name, attr_value|
+          field = nil
+          @schemas.each{|schema| field = schema.fields[attr_name]; break if field}
+          field = FieldDefinition.new(attr_name, :type => attr_value.class) unless field
+          write_attribute(field, attr_value, false, @record)
+        end
+      end
+    end
+    
+    def json_attributes
+      deserialize unless @attributes
+      @attributes
+    end
+    
+    def changes
+      @record.changes
+    end
+    
+    def changed_attributes
+      @record.send(:changed_attributes)
+    end
+    
+  end
+end