association_collection_tools 0.0.1

data/README

@@ -0,0 +1,126 @@
+=Association Collection Tools
+
+== About
+
+Any time you use an ORM you need to know that you are often sacrificing
+performance for convenience and developer efficiency.  In general, this
+is a good thing.  I agree with the theory espoused by DHH that developer
+productivity is *often* more valuable than machine performance.  At least,
+I certainly agree with it in the early stages of development.  Once you
+get to a certain scale, however, there are cases where you'll need to
+write your own code that bypasses the ORM in the name of performance.
+This plugin provides some association operations that issue direct SQL
+calls to make things go faster.
+
+a. fast_copy
+A method called fast_copy is added to has_and_belongs_to_many association
+collections that makes the process of cloning HABTM associations *MUCH*
+more efficient.  Simply replace person1.items = person2.items with 
+person1.items.fast_copy(person2) and you're database, network and RAM will
+thank you.  See below for more details.
+
+b. ids
+A method called ids is added to has_many and has_and_belongs_to_many
+association collections.  It returns the list of object ids in the association
+collection without unnecessarily instantiating the objects.
+
+== Installation
+
+1. This plugin requires that the memcache-client gem is installed.
+   # gem install association_collection_tools
+
+2. Install the plugin OR the gem
+   $ script/plugin install svn://rubyforge.org/var/svn/zventstools/projects/association_collection_tools
+   - OR -
+   # gem install association_collection_tools
+
+== HABTM Fast Copy
+Copies a HABTM association collection from one object to another
+without instantiating a bunch of ActiveRecord objects.  This is
+faster than the standard assignment operation since:
+
+1. Eliminates massive number of SQL calls used in standard HABTM
+   copy by changing it from an O(n) operation to O(1) where
+   n is the number of objects in the association collection.
+2. It transfers only object IDs back and forth between the database
+   instead of all object attributes.  Resulting in less work for
+   the database, less data transferred and less memory used in ruby.
+3. It doesn't instantiate ActiveRecord objects in memory.
+
+A normal HABTM copy (e.g., person1.items = person2.items) results
+in the following SQL calls.
+
+SELECT * FROM items INNER JOIN items_people ON items.id = items_people.item_id WHERE (items_people.person_id = 1 )
+SELECT * FROM items INNER JOIN items_people ON items.id = items_people.item_id WHERE (items_people.person_id = 2 )
+DELETE FROM items_people WHERE person_id = 2 AND item_id IN (4)
+INSERT INTO items_people (`item_id`, `person_id`) VALUES (1, 2)
+INSERT INTO items_people (`item_id`, `person_id`) VALUES (2, 2)
+INSERT INTO items_people (`item_id`, `person_id`) VALUES (3, 2)
+
+Notice that:
+- items AR objects are instantiated unnecessarily (especially since
+  person2.items are about to be deleted)
+- 1 SQL call is issued for each object (item) in the association
+  collection (items_people)
+
+whereas person.items.fast_copy will result in the
+the following SQL calls greatly reducing the impact on the database
+and on ruby memory utilization.
+
+DELETE FROM items_people WHERE person_id = 2
+SELECT item_id FROM items_people WHERE person_id = 1
+REPLACE INTO items_people (person_id,item_id) VALUES (2,3),(2,2),(2,1)
+
+Here are some benchmarks:
+
+when n = 10 and 26 objects in e2.groups:
+
+Benchmark.bm do |x|
+ x.report { for i in 1..n; e1.groups.clear;e1.groups = e2.groups;end }
+ x.report { for i in 1..n; e1.groups.clear;e1.groups.fast_copy(e2);end }
+end
+
+    user     system      total        real
+1.140000   0.040000   1.180000 (  1.832122)
+0.020000   0.010000   0.030000 (  0.125368)
+
+when n = 100 and 26 objects in e2.groups:
+
+     user     system      total        real
+11.140000   0.360000  11.500000 ( 18.171410)
+ 0.140000   0.010000   0.150000 (  2.368200)
+
+This method also supports HABTM join tables with additional attributes.
+Simply pass in an attribute hash as the second argument and it will
+add the attributes to the records it creates in the join table.
+
+e.g, person1.items.fast_copy(person2, {:created_at => Time.now})
+
+REALITY CHECK: The HABTM docs refer to collection_singular_ids=ids
+which implies identical functionality, but I can't find mention of
+this method in anything other than the documentation.  Maybe this
+actually already exists and I'm just blind, but from the looks of
+http://dev.rubyonrails.org/ticket/2917, it appears that it is a
+documentation bug.
+
+== HABTM and has_many ids
+Return the list of IDs in this association collection without unnecessarily 
+instantiating a bunch of Active Record objects.  What good is the id of
+an object without the object itself?  If you think about it for a while,
+you're bound to come up with many uses, especially if you write a lot of
+SQL by hand.  For instance, the fast_copy command documented above uses
+this method to return an id list without instantiating AR objects.  The
+potential savings are enormous when you're dealing with hundreds or thousands 
+of objects at a time.
+
+== Bugs, Code and Contributing
+
+There.s a RubyForge project set up at:
+
+http://rubyforge.org/projects/zventstools/
+
+Anonymous SVN access:
+
+$ svn checkout svn://rubyforge.org/var/svn/zventstools
+
+Author: Tyler Kovacs (tyler dot kovacs at gmail dot com)

data/lib/association_collection_tools.rb

@@ -0,0 +1,138 @@
+# /usr/local/lib/ruby/gems/1.8/gems/activerecord-1.14.4/lib/active_record/associations/has_and_belongs_to_many_association.rb
+module ActiveRecord
+  module Associations
+    class HasAndBelongsToManyAssociation
+      # Copies a HABTM association collection from one object to another
+      # without instantiating a bunch of ActiveRecord objects.  This is
+      # faster than the standard assignment operation since:
+      #
+      # 1. Eliminates massive number of SQL calls used in standard HABTM
+      #    copy by changing it from an O(n) operation to O(1) where
+      #    n is the number of objects in the association collection.
+      # 2. It transfers only object IDs back and forth between the database
+      #    instead of all object attributes.  Resulting in less work for
+      #    the database, less data transferred and less memory used in ruby.
+      # 3. It doesn't instantiate ActiveRecord objects in memory.
+      # 
+      # A normal HABTM copy (e.g., person1.items = person2.items) results
+      # in the following SQL calls.
+      #
+      # SELECT * FROM items INNER JOIN items_people ON items.id = items_people.item_id WHERE (items_people.person_id = 1 )
+      # SELECT * FROM items INNER JOIN items_people ON items.id = items_people.item_id WHERE (items_people.person_id = 2 )
+      # DELETE FROM items_people WHERE person_id = 2 AND item_id IN (4)
+      # INSERT INTO items_people (`item_id`, `person_id`) VALUES (1, 2)
+      # INSERT INTO items_people (`item_id`, `person_id`) VALUES (2, 2)
+      # INSERT INTO items_people (`item_id`, `person_id`) VALUES (3, 2)
+      #
+      # Notice that:
+      # - items AR objects are instantiated unnecessarily (especially since
+      #   person2.items are about to be deleted)
+      # - 1 SQL call is issued for each object (item) in the association
+      #   collection (items_people)
+      #
+      # whereas person.items.fast_copy will result in the
+      # the following SQL calls greatly reducing the impact on the database
+      # and on ruby memory utilization.
+      #
+      # DELETE FROM items_people WHERE person_id = 2
+      # SELECT item_id FROM items_people WHERE person_id = 1
+      # REPLACE INTO items_people (person_id,item_id) VALUES (2,3),(2,2),(2,1)
+      #
+      # Here are some benchmarks:
+      #
+      # when n = 10 and 26 objects in e2.groups:
+      #
+      # Benchmark.bm do |x|
+      #  x.report { for i in 1..n; e1.groups.clear;e1.groups = e2.groups;end }
+      #  x.report { for i in 1..n; e1.groups.clear;e1.groups.fast_copy(e2);end }
+      # end
+      #
+      #     user     system      total        real
+      # 1.140000   0.040000   1.180000 (  1.832122)
+      # 0.020000   0.010000   0.030000 (  0.125368)
+      #
+      # when n = 100 and 26 objects in e2.groups:
+      #
+      #      user     system      total        real
+      # 11.140000   0.360000  11.500000 ( 18.171410)
+      #  0.140000   0.010000   0.150000 (  2.368200)
+      #
+      #
+      # This method also supports HABTM join tables with additional attributes.
+      # Simply pass in an attribute hash as the second argument and it will
+      # add the attributes to the records it creates in the join table.
+      #
+      # e.g, person1.items.fast_copy(person2, {:created_at => Time.now})
+      #
+      # REALITY CHECK: The HABTM docs refer to collection_singular_ids=ids
+      # which implies identical functionality, but I can't find mention of
+      # this method in anything other than the documentation.  Maybe this
+      # actually already exists and I'm just blind, but from the looks of
+      # http://dev.rubyonrails.org/ticket/2917, it appears that it is a 
+      # documentation bug.
+      def fast_copy(other_object,attributes = {})
+        self.fast_clear
+        other_object_assocation_ids = other_object.send(@reflection.name).ids
+        return [] if other_object_assocation_ids.empty?
+
+        column_names = [ @reflection.primary_key_name,
+          @reflection.association_foreign_key ]
+        attribute_values = []
+        attributes.keys.each{|k| 
+          column_names << k
+          attribute_values << attributes[k]
+        }
+
+        @owner.connection.execute("REPLACE INTO #{@reflection.options[:join_table]} (#{column_names.join(",")}) VALUES #{other_object_assocation_ids.map{|aid| "(#{@owner.quoted_id},#{aid}#{attributes.empty? ? "" : ("," + attribute_values.join(','))})"}.join(",")}")
+        return other_object_assocation_ids
+      end
+
+      # Return the list of IDs in this association collection without 
+      # unnecessarily instantiating a bunch of Active Record objects.
+      def ids
+        if self.loaded?
+          self.map{|x| x.id}
+        else
+          connection.select_all("SELECT #{@reflection.association_foreign_key} FROM #{@reflection.options[:join_table]} WHERE #{@reflection.primary_key_name} = #{@owner.quoted_id}").map!{|x| x[@reflection.association_foreign_key].to_i}
+        end
+      end
+
+      # Removes all records from this association.  Returns +self+ so method 
+      # calls may be chained.  If this association is not marked as :dependent,
+      # then use a faster delete_all method that doesn't instantiate a bunch
+      # of AR objects.
+      def fast_clear
+        if @reflection.options[:dependent] && @reflection.options[:dependent] == :delete_all
+          return self if length.zero? # forces load_target if hasn't happened already
+          destroy_all
+        else
+          fast_delete_all
+        end
+
+        self
+      end
+
+      def fast_delete_all
+        @owner.connection.execute("DELETE FROM #{@reflection.options[:join_table]} WHERE #{@reflection.primary_key_name} = #{@owner.quoted_id}")
+        @target = []
+      end
+    end
+  end
+end
+
+# /usr/local/lib/ruby/gems/1.8/gems/activerecord-1.14.4/lib/active_record/associations/has_many_association.rb
+module ActiveRecord
+  module Associations
+    # Return the list of IDs in this association without unnecessarily
+    # instantiating a bunch of Active Record objects.
+    class HasManyAssociation
+      def ids
+        if self.loaded?
+          self.map{|x| x.id}
+        else
+          connection.select_all("SELECT #{@reflection.klass.primary_key} FROM #{@reflection.klass.table_name} WHERE #{@reflection.primary_key_name} = #{@owner.quoted_id}").map!{|x| x[@reflection.klass.primary_key].to_i}
+        end
+      end
+    end
+  end
+end

data/test/association_tools_test.rb

@@ -0,0 +1,90 @@
+require 'test/unit'
+require File.dirname(__FILE__) + '/../../../../test/test_helper'
+
+class Person < ActiveRecord::Base
+  set_table_name 'persons'
+  has_and_belongs_to_many :items
+  has_many :pets
+end
+
+class Item < ActiveRecord::Base
+  set_table_name 'items'
+  has_and_belongs_to_many :persons
+end
+
+class Pet < ActiveRecord::Base
+  set_table_name 'pets'
+  belongs_to :person
+end
+
+class AssociationToolsTest < Test::Unit::TestCase
+  Item.connection.execute "DROP TABLE IF EXISTS items;"
+  Item.connection.execute "DROP TABLE IF EXISTS persons;"
+  Item.connection.execute "DROP TABLE IF EXISTS items_people;"
+  Item.connection.execute "DROP TABLE IF EXISTS pets;"
+  Item.connection.execute "CREATE TABLE items (id INT NOT NULL AUTO_INCREMENT, name VARCHAR(32) DEFAULT '', PRIMARY KEY(id));"
+  Item.connection.execute "CREATE TABLE persons (id INT NOT NULL AUTO_INCREMENT, name VARCHAR(32) DEFAULT '', PRIMARY KEY(id));"
+  Item.connection.execute "CREATE TABLE items_people (item_id INT NOT NULL, person_id INT NOT NULL);"
+  Item.connection.execute "CREATE TABLE pets (id INT NOT NULL AUTO_INCREMENT, name VARCHAR(32) DEFAULT '', person_id INT NOT NULL, PRIMARY KEY(id));"
+
+  Test::Unit::TestCase.fixture_path = File.dirname(__FILE__) + '/fixtures'
+  fixtures :persons, :items, :items_people, :pets
+
+  def test_has_and_belongs_to_many_ids
+    bob = persons(:bob)
+    assert_equal 3, bob.items.length
+    assert_equal [1,2,3], bob.items.map{|i| i.id}.sort
+    assert bob.items.loaded?
+
+    assert bob.reload
+    assert !bob.items.loaded?
+    assert_equal [1,2,3], bob.items.ids.sort
+    assert !bob.items.loaded?
+  end
+
+  def test_has_many_ids
+    bob = persons(:bob)
+    assert_equal 2, bob.pets.length
+    assert_equal [1,2], bob.pets.map{|p| p.id}.sort
+    assert bob.pets.loaded?
+
+    assert bob.reload
+    assert !bob.pets.loaded?
+    assert_equal [1,2], bob.pets.ids.sort
+    assert !bob.pets.loaded?
+  end
+
+  def test_has_and_belongs_to_many_fast_delete_all
+    bob = persons(:bob)
+    assert_equal 3, bob.items.length
+    assert bob.reload
+    assert !bob.items.loaded?
+
+    bob.items.fast_delete_all
+    assert !bob.items.loaded?
+    assert bob.reload
+    assert_equal 0, bob.items.length
+  end
+
+  def test_has_and_belongs_to_many_fast_copy
+    bob = persons(:bob)
+    larry = persons(:larry)
+
+    larry.items.fast_copy(bob)
+    assert !bob.items.loaded? 
+    assert !larry.items.loaded? 
+    assert_equal [1,2,3], bob.items.map{|i| i.id}.sort
+    assert_equal [1,2,3], larry.items.map{|i| i.id}.sort
+  end
+  
+  def test_has_and_belongs_to_many_standard_copy
+    bob = persons(:bob)
+    larry = persons(:larry)
+    
+    larry.items = bob.items
+    assert bob.items.loaded? 
+    assert larry.items.loaded? 
+    assert_equal [1,2,3], bob.items.map{|i| i.id}.sort
+    assert_equal [1,2,3], larry.items.map{|i| i.id}.sort
+  end
+end

metadata

@@ -0,0 +1,52 @@
+--- !ruby/object:Gem::Specification 
+rubygems_version: 0.8.10
+specification_version: 1
+name: association_collection_tools
+version: !ruby/object:Gem::Version 
+  version: 0.0.1
+date: 2006-11-02
+summary: Adds fast_copy method to has_and_belongs_to_many associations cloning associations much faster than the assignment operator.  Adds ids methods to has_many and has_and_belongs_to_many associations for retrieving object ids without instantiating ActiveRecord objects.
+require_paths: 
+- lib
+email: tyler.kovacs@zvents.com
+homepage: http://blog.zvents.com/2006/11/3/rails-plugin-association-collection-tools
+rubyforge_project: 
+description: 
+autorequire: association_collection_tools
+default_executable: 
+bindir: bin
+has_rdoc: true
+required_ruby_version: !ruby/object:Gem::Version::Requirement 
+  requirements: 
+  - - ">"
+    - !ruby/object:Gem::Version 
+      version: 0.0.0
+  version: 
+platform: ruby
+authors: 
+- Tyler Kovacs
+files: 
+- lib/association_collection_tools.rb
+- README
+test_files: 
+- test/association_tools_test.rb
+rdoc_options: []
+
+extra_rdoc_files: 
+- README
+executables: []
+
+extensions: []
+
+requirements: []
+
+dependencies: 
+- !ruby/object:Gem::Dependency 
+  name: memcache-client
+  version_requirement: 
+  version_requirements: !ruby/object:Gem::Version::Requirement 
+    requirements: 
+    - - ">="
+      - !ruby/object:Gem::Version 
+        version: 1.0.3
+    version: