perpetuity 0.4.4 → 0.4.5

data/.travis.yml

@@ -1,6 +1,5 @@
 language: ruby
 rvm:
-  - 1.9.2
   - 1.9.3
   - rbx-19mode
   - jruby-19mode

data/CHANGELOG.md

@@ -1,3 +1,12 @@
+## Version 0.4.5
+
+- Move from the `mongo` gem from 10gen to the `moped` gem for talking to MongoDB. This resulted in performance gains of 30-80%, depending on query size.
+- Make persisted objects marshalable.
+  - Previously, the `id` method was defined on individual objects that were either persisted to or retrieved from the DB. This made them unable to be marshaled with `Marshal.dump`.
+  - Now we extend the objects with `Perpetuity::PersistedObject` to keep them marshalable while still providing `id`. This keeps them marshalable (unmarshalled objects will still be extended with `Perpetuity::PersistedObject`).
+- Provided a partial fix for a [bug](https://github.com/jgaskins/perpetuity/issues/23) that kept us from being able to persist hash attributes properly. See the first comments in the linked GitHub issue for an explanation of why it is only a partial fix.
+- Stopped testing on MRI 1.9.2 with Travis CI. Moped requires 1.9.3 or higher.
+
 ## Version 0.4.4
 
 - Automatically persist all referenced objects if they are not already persisted. Previously, referenced objects were required to be persisted before persisting the referencing object.

data/README.md

@@ -23,7 +23,14 @@ gem 'perpetuity'
 The only currently supported persistence method is MongoDB. Other schemaless solutions can probably be implemented easily.
 
 ```ruby
-mongodb = Perpetuity::MongoDB.new host: 'mongodb.example.com', db: 'example_db'
+mongodb = Perpetuity::MongoDB.new(
+  db: 'example_db',            # Required
+  host: 'mongodb.example.com', # Default: 'localhost'
+  port: 27017,                 # Default: 27017
+  username: 'mongo',           # If no username/password given, do not authenticate
+  password: 'password'
+)
+
 Perpetuity.configure do 
   data_source mongodb
 end
@@ -35,7 +42,10 @@ Object mappers are generated by the following:
 
 ```ruby
 Perpetuity.generate_mapper_for MyClass do
-  # individual mapper configuration goes here
+  attribute :my_attribute
+  attribute :my_other_attribute
+
+  index :my_attribute
 end
 ```
 
@@ -68,7 +78,7 @@ You can load all persisted objects of a particular class by sending `all` to the
 Perpetuity[Article].all
 ```
 
-You can load specific objects by calling the `find` method with an ID param on that class's mapper class and passing in the criteria. You may also specify more general criteria using the `select` method with a block similar to `Enumerable#select`.
+You can load specific objects by calling the `find` method with an ID param on the mapper and passing in the criteria. You may also specify more general criteria using the `select` method with a block similar to `Enumerable#select`.
 
 ```ruby
 article  = Perpetuity[Article].find params[:id]
@@ -77,13 +87,15 @@ articles = Perpetuity[Article].select { |article| article.published_at < Time.no
 comments = Perpetuity[Comment].select { |comment| comment.article_id.in articles.map(&:id) }
 ```
 
-These methods will return a Perpetuity::Retrieval object, which will lazily retrieve the objects from the database. They will wait to hit the DB when you begin iterating over the objects so you can continue chaining methods.
+These methods will return a Perpetuity::Retrieval object, which will lazily retrieve the objects from the database. They will wait to hit the DB when you begin iterating over the objects so you can continue chaining methods, similar to ActiveRecord.
 
 ```ruby
 article_mapper = Perpetuity[Article]
 articles = article_mapper.select { |article| article.published_at < Time.now }
-articles = articles.sort(:published_at).reverse
-articles = articles.page(2).per_page(10) # built-in pagination
+                         .sort(:published_at)
+                         .reverse
+                         .page(2)
+                         .per_page(10) # built-in pagination
 
 articles.each do |article| # This is when the DB gets hit
   # Display the pretty articles
@@ -100,37 +112,52 @@ Notice that we have to use a single `&` and surround each criterion with parenth
 
 ## Associations with Other Objects
 
-The database can natively serialize some objects. For example, MongoDB can serialize `String`, `Numeric`, `Array`, `Hash`, `Time`, `nil`, `true`, `false`, and a few others. If an object references another type of object (such as an article referencing its author, a `User` object), the association is declared just as any other attribute. No special treatment is required.
+The database can natively serialize some objects. For example, MongoDB can serialize `String`, `Numeric`, `Array`, `Hash`, `Time`, `nil`, `true`, `false`, and a few others. For other data types, you must determine whether you want those attributes embedded within the same document in the database or attached as a reference. For example, a `Post` could have `Comment`s, which would likely be embedded within the post object. But these comments could have an `author` attribute that references the `Person` that wrote the comment. Embedding the author in this case is not a good idea since it would be a duplicate of the `Person` that wrote it, which would then be out of sync if the original object is modified.
 
-If the associated object's class has a mapper defined, it will be used by the parent object's mapper for serialization. Otherwise, the object will be `Marshal.dump`ed. If the object cannot be marshaled, the object cannot be serialized and an exception will be raised.
+If an object references another type of object, the association is declared just as any other attribute. No special treatment is required. For embedded relationships, make sure you use the `embedded: true` option in the attribute.
 
 ```ruby
-class User
+Perpetuity.generate_mapper_for Article do
+  attribute :title
+  attribute :body
+  attribute :author
+  attribute :comments, embedded: true
+  attribute :timestamp
 end
 
-class Article
-  attr_accessor :author
+Perpetuity.generate_mapper_for Comment do
+  attribute :body
+  attribute :author
+  attribute :timestamp
 end
+```
 
-Perpetuity.generate_mapper_for User do
-end
+In this case, the article has an array of `Comment` objects, which the serializer knows that MongoDB cannot serialize. It will then tell the `Comment` mapper to serialize it and it stores that within the array.
 
-Perpetuity.generate_mapper_for Article do
-  attribute :author
-end
+If some of the comments aren't objects of class `Comment`, it will adapt and serialize them according to their class. This works very well for objects that can have attributes of various types, such as a `User` having a profile attribute that can be either a `UserProfile` or `AdminProfile` object. You don't need to declare anything different for this case, just store the appropriate type of object into the `User`'s `profile` attribute and the mapper will take care of the details.
+
+If the associated object's class has a mapper defined, it will be used by the parent object's mapper for serialization. Otherwise, the object will be `Marshal.dump`ed. If the object cannot be marshaled, the object cannot be serialized and an exception will be raised.
+
+When you load an object that has embedded associations, the embedded attributes are loaded immediately. For referenced associations, though, only the object itself will be loaded. All referenced objects must be loaded with the `load_association!` mapper call.
+
+```ruby
+user_mapper = Perpetuity[User]
+user = user_mapper.find(params[:id])
+user_mapper.load_association! user, :profile
 ```
 
-This allows you to write the following:
+This loads up the user's profile and injects it into the profile attribute. All loading of referenced objects is explicit so that we don't load an entire object graph unnecessarily. This encourages (forces, really) you to think about all of the objects you'll be loading.
+
+If you want to load a 1:N, N:1 or M:N association, Perpetuity handles that for you.
 
 ```ruby
 article_mapper = Perpetuity[Article]
-article = article_mapper.first
-article_mapper.load_association! article, :author
-user = article.author
+articles = article_mapper.all.to_a
+article_mapper.load_association! articles.first, :tags # 1:N
+article_mapper.load_association! articles, :author     # All author objects for these articles load in a single query - N:1
+article_mapper.load_association! articles, :tags       # M:N
 ```
 
-All loading of associated objects is explicit so that we don't load an entire object graph unnecessarily.
-
 ## Customizing persistence
 
 Setting the ID of a record to a custom value rather than using the DB default.
@@ -141,6 +168,8 @@ Perpetuity.generate_mapper_for Article do
 end
 ```
 
+The block passed to the `id` macro is evaluated in the context of the object being persisted. This allows you to use the object's private methods and instance variables if you need to.
+
 ## Indexing
 
 Indexes are declared with the `index` method. The simplest way to create an index is just to pass the attribute to be indexed as a parameter:
@@ -185,4 +214,4 @@ Perpetuity[Article].reindex!
 
 ## Contributing
 
-Right now, this code is pretty bare and there are possibly some design decisions that need some more refinement. You can help. If you have ideas to build on this, send some love in the form of pull requests or issues or tweets or e-mails and I'll do what I can for them.
+Right now, this code is pretty bare and there are possibly some design decisions that need some more refinement. You can help. If you have ideas to build on this, send some love in the form of pull requests or issues or [tweets](http://twitter.com/jamie_gaskins) or e-mails and I'll do what I can for them.

data/lib/perpetuity/data_injectable.rb

@@ -1,8 +1,9 @@
+require 'perpetuity/persisted_object'
+
 module Perpetuity
   module DataInjectable
     def inject_attribute object, attribute, value
-      attribute = "@#{attribute}" unless attribute[0] == '@'
-      object.instance_variable_set(attribute, value)
+      object.instance_variable_set("@#{attribute}", value)
     end
 
     def inject_data object, data
@@ -13,9 +14,10 @@ module Perpetuity
     end
 
     def give_id_to object, *args
-      object.define_singleton_method :id do
-        args.first || object.instance_variable_get(:@id)
+      if args.any?
+        inject_attribute object, :id, args.first
       end
+      object.extend PersistedObject
     end
   end
 end

data/lib/perpetuity/mapper.rb

@@ -86,12 +86,12 @@ module Perpetuity
     end
 
     def select &block
-      query = data_source.class::Query.new(&block).to_db
+      query = data_source.query(&block).to_db
       retrieve query
     end
 
     def find id
-      retrieve(id: id).first
+      select { |object| object.id == id }.first
     end
 
     def delete object
@@ -148,13 +148,7 @@ module Perpetuity
     end
 
     def serialize object
-      Serializer.new(self, mapper_registry).serialize(object)
-    end
-
-    private
-
-    def retrieve criteria={}
-      Perpetuity::Retrieval.new mapped_class, criteria, data_source
+      Serializer.new(self).serialize(object)
     end
 
     def self.mapped_class
@@ -164,6 +158,12 @@ module Perpetuity
     def mapped_class
       self.class.mapped_class
     end
+
+    private
+
+    def retrieve criteria={}
+      Perpetuity::Retrieval.new self, criteria
+    end
   end
 end
 

data/lib/perpetuity/mongodb.rb

@@ -1,10 +1,11 @@
-require 'mongo'
+require 'moped'
 require 'perpetuity/mongodb/query'
 require 'perpetuity/mongodb/index'
+require 'set'
 
 module Perpetuity
   class MongoDB
-    attr_accessor :connection, :host, :port, :db, :pool_size, :username, :password
+    attr_accessor :host, :port, :db, :pool_size, :username, :password
 
     def initialize options
       @host       = options.fetch(:host, 'localhost')
@@ -13,47 +14,54 @@ module Perpetuity
       @pool_size  = options.fetch(:pool_size, 5)
       @username   = options[:username]
       @password   = options[:password]
-      @connection = nil
+      @session    = nil
       @indexes    = Hash.new { |hash, key| hash[key] = active_indexes(key) }
     end
 
+    def session
+      @session ||= Moped::Session.new(["#{host}:#{port}"])
+    end
+
     def connect
-      database.authenticate(@username, @password) if @username and @password
-      @connection ||= Mongo::MongoClient.new @host, @port, pool_size: @pool_size
+      session.login(@username, @password) if @username and @password
+      session
     end
 
     def connected?
-      !!@connection
+      !!@session
     end
 
     def database
       connect unless connected?
-      @connection.db(@db)
+      session.use db
     end
 
     def collection klass
-      database.collection(klass.to_s)
+      database[klass.to_s]
     end
 
     def insert klass, attributes
       if attributes.has_key? :id
         attributes[:_id] = attributes[:id]
         attributes.delete :id
+      else
+        attributes[:_id] = Moped::BSON::ObjectId.new
       end
 
       collection(klass).insert attributes
+      attributes[:_id]
     end
 
     def count klass
-      collection(klass).count
+      collection(klass).find.count
     end
 
     def delete_all klass
-      database.collection(klass.to_s).remove
+      collection(klass.to_s).find.remove_all
     end
 
     def first klass
-      document = database.collection(klass.to_s).find_one
+      document = collection(klass.to_s).find.limit(1).first
       document[:id] = document.delete("_id")
 
       document
@@ -63,46 +71,48 @@ module Perpetuity
       # MongoDB uses '_id' as its ID field.
       if criteria.has_key?(:id)
         if criteria[:id].is_a? String
-          criteria = { _id: (BSON::ObjectId.from_string(criteria[:id].to_s) rescue criteria[:id]) }
+          criteria = { _id: (Moped::BSON::ObjectId(criteria[:id].to_s) rescue criteria[:id]) }
         else
           criteria[:_id] = criteria.delete(:id)
         end
       end
 
-      other_options = { limit: options[:limit] }
-      if options[:page]
-        other_options = other_options.merge skip: (options[:page] - 1) * options[:limit]
-      end
-      cursor = database.collection(klass.to_s).find(criteria, other_options)
+      query = collection(klass.to_s).find(criteria)
+
+      skipped = options[:page] ? (options[:page] - 1) * options[:limit] : 0
+      query = query.skip skipped
+      query = query.limit(options[:limit])
+      query = sort(query, options)
 
-      sort_cursor(cursor, options).map do |document|
+      query.map do |document|
         document[:id] = document.delete("_id")
         document
       end
     end
 
-    def sort_cursor cursor, options
-      return cursor unless options.has_key?(:attribute) &&
-                           options.has_key?(:direction)
+    def sort query, options
+      return query unless options[:attribute] &&
+                          options[:direction]
 
+      sort_orders = { ascending: 1, descending: -1 }
       sort_field = options[:attribute]
       sort_direction = options[:direction]
-      sort_criteria = [[sort_field, sort_direction]]
-      cursor.sort(sort_criteria)
+      sort_criteria = { sort_field => sort_orders[sort_direction] }
+      query.sort(sort_criteria)
     end
 
     def all klass
       retrieve klass, {}, {}
     end
     
-    def delete object, klass=nil
-      id = object.respond_to?(:id) ? object.id : object
+    def delete object_or_id, klass=nil
+      id = object_or_id.is_a?(PersistedObject) ? object_or_id.id : object_or_id
       klass ||= object.class
-      collection(klass.to_s).remove "_id" => id
+      collection(klass.to_s).find("_id" => id).remove
     end
 
     def update klass, id, new_data
-      collection(klass).update({ _id: id }, new_data)
+      collection(klass).find({ _id: id }).update(new_data)
     end
 
     def can_serialize? value
@@ -110,7 +120,11 @@ module Perpetuity
     end
 
     def drop_collection to_be_dropped
-      collection(to_be_dropped).drop
+      collection(to_be_dropped.to_s).drop
+    end
+
+    def query &block
+      Query.new(&block)
     end
 
     def index klass, attribute, options={}
@@ -126,8 +140,8 @@ module Perpetuity
     end
 
     def active_indexes klass
-      indexes = collection(klass).index_information
-      indexes.map do |name, index|
+      indexes = collection(klass).indexes.to_a
+      indexes.map do |index|
         key = index['key'].keys.first
         direction = index['key'][key]
         unique = index['unique']
@@ -140,17 +154,18 @@ module Perpetuity
       order = index.order == :ascending ? 1 : -1
       unique = index.unique?
 
-      collection(index.collection).create_index [[attribute, order]], unique: unique
+      collection(index.collection).indexes.create({attribute => order}, unique: unique)
       index.activate!
     end
 
     def remove_index index
       coll = collection(index.collection)
-      db_indexes = coll.index_information.select do |name, info|
-        name =~ /#{index.attribute}/
-      end
+      db_indexes = coll.indexes.select do |db_index|
+        db_index['name'] =~ /\A#{index.attribute}/
+      end.map { |index| index['key'] }
+
       if db_indexes.any?
-        collection(index.collection).drop_index db_indexes.first.first
+        collection(index.collection).indexes.drop db_indexes.first
       end
     end
 

data/lib/perpetuity/persisted_object.rb

@@ -0,0 +1,7 @@
+module Perpetuity
+  module PersistedObject
+    def id
+      @id
+    end
+  end
+end

data/lib/perpetuity/retrieval.rb

@@ -1,16 +1,16 @@
-require 'perpetuity/data_injectable'
 require 'perpetuity/reference'
+require 'perpetuity/serializer'
 
 module Perpetuity
   class Retrieval
-    include DataInjectable
     include Enumerable
     attr_accessor :sort_attribute, :sort_direction, :result_limit, :result_page, :quantity_per_page
 
-    def initialize klass, criteria, data_source = Perpetuity.configuration.data_source
-      @class = klass
+    def initialize mapper, criteria
+      @mapper = mapper
+      @class = mapper.mapped_class
       @criteria = criteria
-      @data_source = data_source
+      @data_source = mapper.data_source
     end
     
     def sort attribute=:name
@@ -57,35 +57,7 @@ module Perpetuity
     end
 
     def unserialize(data)
-      if data.is_a?(String) && data.start_with?("\u0004") # if it's marshaled
-        Marshal.load(data)
-      elsif data.is_a? Array
-        data.map { |i| unserialize i }
-      elsif data.is_a? Hash
-        metadata = data.delete('__metadata__')
-        if metadata
-          klass = Object.const_get metadata['class']
-          id = metadata['id']
-          if id
-            object = Reference.new(klass, id)
-          else
-            object = klass.allocate
-            data.each do |attr, value|
-              inject_attribute object, attr, unserialize(value)
-            end
-          end
-        else
-          object = @class.allocate
-          data.each do |attr, value|
-            inject_attribute object, attr, unserialize(value)
-          end
-        end
-
-        give_id_to object
-        object
-      else
-        data
-      end
+      Serializer.new(@mapper).unserialize(data)
     end
 
     def [] index

data/lib/perpetuity/serializer.rb

@@ -1,10 +1,15 @@
+require 'perpetuity/data_injectable'
+
 module Perpetuity
   class Serializer
+    include DataInjectable
+
     attr_reader :mapper, :mapper_registry
 
-    def initialize(mapper, mapper_registry)
+    def initialize(mapper)
       @mapper = mapper
-      @mapper_registry = mapper_registry
+      @class = mapper.mapped_class
+      @mapper_registry = mapper.mapper_registry
     end
 
     def attribute_for object, attribute_name
@@ -22,9 +27,7 @@ module Perpetuity
         elsif mapper_registry.has_mapper?(value.class)
           serialize_with_foreign_mapper(value, attrib.embedded?)
         else
-          if attrib.embedded?
-            Marshal.dump(value)
-          end
+          Marshal.dump(value)
         end
 
         [attrib.name.to_s, serialized_value]
@@ -33,6 +36,59 @@ module Perpetuity
       Hash[attrs]
     end
 
+    def unserialize data
+      if data.is_a? Array
+        unserialize_object_array data
+      else
+        object = unserialize_object(data)
+        give_id_to object
+        object
+      end
+    end
+
+    def unserialize_object data, klass=@class
+      if data.is_a? Hash
+        object = klass.allocate
+        data.each do |attr, value|
+          inject_attribute object, attr, unserialize_attribute(value)
+        end
+        object
+      else
+        unserialize_attribute data
+      end
+    end
+
+    def unserialize_object_array objects
+      objects.map do |data|
+        object = unserialize_object data
+        give_id_to object
+        object
+      end
+    end
+
+    def unserialize_attribute data
+      if data.is_a?(String) && data.start_with?("\u0004") # if it's marshaled
+        Marshal.load(data)
+      elsif data.is_a? Array
+        data.map { |i| unserialize_attribute i }
+      elsif data.is_a? Hash
+        metadata = data.delete('__metadata__')
+        if metadata
+          klass = Object.const_get metadata['class']
+          id = metadata['id']
+          if id
+            object = Reference.new(klass, id)
+          else
+            object = unserialize_object(data, klass)
+          end
+        else
+          data
+        end
+      else
+        data
+      end
+    end
+
     def serialize_with_foreign_mapper value, embedded = false
       if embedded
         value_mapper = mapper_registry[value.class]
@@ -67,7 +123,7 @@ module Perpetuity
     end
 
     def serialize_reference value
-      unless value.respond_to? :id
+      unless value.is_a? PersistedObject
         mapper_registry[value.class].insert value
       end
       {

data/lib/perpetuity/version.rb

@@ -1,3 +1,3 @@
 module Perpetuity
-  VERSION = "0.4.4"
+  VERSION = "0.4.5"
 end

data/perpetuity.gemspec

@@ -19,6 +19,5 @@ Gem::Specification.new do |s|
   # specify any dependencies here; for example:
   s.add_development_dependency "rake"
   s.add_development_dependency "rspec", "~> 2.8.0"
-  s.add_runtime_dependency "mongo", ">= 1.8.0"
-  s.add_runtime_dependency "bson_ext" unless RUBY_PLATFORM == 'java'
+  s.add_runtime_dependency "moped"
 end

data/spec/integration/associations_spec.rb

@@ -0,0 +1,65 @@
+require 'spec_helper'
+require 'support/test_classes'
+
+describe 'associations with other objects' do
+  let(:user) { User.new }
+  let(:topic) { Topic.new }
+  let(:user_mapper) { Perpetuity[User] }
+  let(:topic_mapper) { Perpetuity[Topic] }
+
+  before do
+    user.name = 'Flump'
+    topic.creator = user
+    topic.title = 'Title'
+
+    user_mapper.insert user
+    topic_mapper.insert topic
+  end
+
+  describe 'referenced relationships' do
+    let(:creator) { topic_mapper.find(topic.id).creator }
+    subject { creator }
+
+    it { should be_a Perpetuity::Reference }
+    its(:klass) { should be User }
+    its(:id) { should be == user.id }
+  end
+
+  it 'can retrieve a one-to-one association' do
+    retrieved_topic = topic_mapper.find(topic.id)
+
+    topic_mapper.load_association! retrieved_topic, :creator
+    retrieved_topic.creator.name.should eq 'Flump'
+  end
+
+  describe 'associations with many objects' do
+    let(:pragprogs) { [User.new('Dave'), User.new('Andy')] }
+    let(:cuke_authors) { [User.new('Matt'), User.new('Aslak')] }
+    let(:pragprog_book) { Book.new("PragProg #{Time.now.to_f}", pragprogs) }
+    let(:cuke_book) { Book.new("Cucumber Book #{Time.now.to_f}", cuke_authors) }
+    let(:book_mapper) { Perpetuity[Book] }
+
+    before do
+      pragprogs.each { |author| Perpetuity[User].insert author }
+      book_mapper.insert pragprog_book
+    end
+
+    it 'can retrieve a one-to-many association' do
+      persisted_book = book_mapper.find(pragprog_book.id)
+      book_mapper.load_association! persisted_book, :authors
+
+      persisted_book.authors.first.name.should be == 'Dave'
+      persisted_book.authors.last.name.should be == 'Andy'
+    end
+
+    it 'can retrieve a many-to-many association' do
+      cuke_authors.each { |author| Perpetuity[User].insert author }
+      book_mapper.insert cuke_book
+      book_ids = [pragprog_book, cuke_book].map(&:id)
+
+      books = book_mapper.select { |book| book.id.in book_ids }.to_a
+      book_mapper.load_association! books, :authors
+      books.map(&:authors).flatten.map(&:name).should include *%w(Dave Andy Matt Aslak)
+    end
+  end
+end