simple_record 1.0.10 → 1.1.17

data/README.markdown

@@ -0,0 +1,200 @@
+# SimpleRecord - ActiveRecord for SimpleDB
+
+An ActiveRecord interface for SimpleDB.  Can be used as a drop in replacement for ActiveRecord in rails.
+
+## Discussion Group
+
+[http://groups.google.com/group/simple-record]
+
+## Issue Tracking
+
+[http://appoxy.lighthouseapp.com/projects/38366-simplerecord/overview]
+
+## Getting Started
+
+1. Install gems
+
+        ONE TIME: Be sure to add the new gemcutter source:
+
+        - gem install gemcutter
+        - gem tumble
+
+        THEN (if you already have http://gemcutter.org in your sources and it MUST be above rubyforge.org):
+
+        gem install aws uuidtools http_connection simple_record
+
+2. Create a model
+
+        require 'simple_record'
+    
+        class MyModel < SimpleRecord::Base
+           has_attributes :name
+           has_ints :age
+        end
+
+More about ModelAttributes below.
+
+3. Setup environment
+
+        AWS_ACCESS_KEY_ID='XXXX'
+        AWS_SECRET_ACCESS_KEY='YYYY'
+        SimpleRecord.establish_connection(AWS_ACCESS_KEY_ID,AWS_SECRET_ACCESS_KEY)
+
+4. Go to town
+
+        # Store a model object to SimpleDB
+        mm = MyModel.new
+        mm.name = "Travis"
+        mm.age = 32
+        mm.save
+        id = mm.id
+
+        # Get an object from SimpleDB
+        mm2 = MyModel.find(id)
+        puts 'got=' + mm2.name + ' and he/she is ' + mm.age.to_s + ' years old'
+        # Or more advanced queries? mms = MyModel?.find(:all, ["age=?", 32], :order=>"name", :limit=>10)
+
+
+## Attributes and modifiers for models
+
+NOTE: All objects will automatically have :id, :created, :updated attributes.
+
+### has_attributes
+
+Add string attributes.
+
+    class MyModel < SimpleRecord::Base
+      has_attributes :name
+    end
+
+### has_ints, has_dates and has_booleans
+
+Lets simple_record know that certain attributes defined in has_attributes should be treated as integers, dates or booleans. This is required because SimpleDB only has strings so SimpleRecord needs to know how to convert, pad, offset, etc.
+
+    class MyModel < SimpleRecord::Base
+      has_attributes :name
+      has_ints :age, :height
+      has_dates :birthday
+      has_booleans :is_nerd
+    end
+
+### belongs_to
+
+Creates a many-to-one relationship. Can only have one per belongs_to call.
+
+    class MyModel < SimpleRecord::Base
+        belongs_to :school
+        has_attributes :name
+        has_ints :age, :height
+        has_dates :birthday
+        has_booleans :is_nerd
+    end
+
+Which requires another class called 'School' or you can specify the class explicitly with:
+
+    belongs_to :school, :class_name => "Institution"
+
+### set_table_name or set_domain_name
+
+If you want to use a custom domain for a model object, you can specify it with set_table_name (or set_domain_name).
+
+    class SomeModel < SimpleRecord::Base
+        set_table_name :different_model
+    end
+
+
+## Configuration
+
+### Domain Prefix
+
+To set a global prefix across all your models, use:
+
+    SimpleRecord::Base.set_domain_prefix("myprefix_")
+
+### Connection Modes
+
+There are 4 different connection modes:
+
+* per_request (default) - opens and closes a new connection to simpledb for every simpledb request. Not the best performance, but it's safe and can handle many concurrent requests at the same time (unlike single mode).
+* single - one connection across the entire application, not recommended unless the app is used by a single person.
+* per_thread - a connection is used for every thread in the application. This is good, but the catch is that you have to ensure to close the connection.
+* pool - NOT IMPLEMENTED YET - opens a maximum number of connections and round robins them for any simpledb request.
+
+You set the mode when you call establish_connection:
+
+    SimpleRecord.establish_connection(AWS_ACCESS_KEY_ID,AWS_SECRET_ACCESS_KEY, :connection_mode=>:per_thread)
+
+We recommend per_thread with explicitly closing the connection after each Rails request (not to be mistaken for a SimpleDB request) or pool for rails apps.
+
+For rails, be sure to add this to your Application controller if using per_thread mode:
+
+    after_filter :close_sdb_connection
+
+    def close_sdb_connection
+        SimpleRecord.close_connection
+    end
+
+## SimpleRecord on Rails
+
+You don't really have to do anything except have your models extends SimpleRecord::Base instead of ActiveRecord::Base, but here are some tips you can use.
+
+### Change Connection Mode
+
+Use per_thread connection mode and close the connection after each request.
+
+    after_filter :close_sdb_connection
+
+    def close_sdb_connection
+        SimpleRecord.close_connection
+    end
+
+### Disable ActiveRecord so you don't have to setup another database
+
+This is most helpful on windows so Rails doesn't need sqlite or mysql gems/drivers installed which are painful to install on windows. In environment.rb, add 'config.frameworks -= [ :active_record ]', should look something like:
+
+    Rails::Initializer.run do |config|
+      config.frameworks -= [ :active_record ]
+      ....
+    end
+
+
+## Tips and Tricks and Things to Know
+
+### Automagic Stuff
+
+
+#### Automatic common fields
+
+Every object will automatically get the following attributes so you don't need to define them:
+
+  * id - UUID string
+  * created - set when first save
+  * updated - set every time you save/update
+
+
+#### belongs_to foreign keys/IDs are accessible without touching the database
+
+If you had the following in your model:
+
+    belongs_to :something
+
+Then in addition to being able to access the something object with:
+
+    o.something
+
+or setting it with:
+
+    o.something = someo
+
+You can also access the ID for something directly with:
+
+    o.something_id
+
+or
+
+    o.something_id = x
+
+## Kudos
+
+Special thanks to Garrett Cox for creating Activerecord2sdb which SimpleRecord is based on:
+http://activrecord2sdb.rubyforge.org/

data/lib/results_array.rb

@@ -0,0 +1,137 @@
+module SimpleRecord
+
+    #
+    # We need to make this behave as if the full set were loaded into the array.
+    class ResultsArray
+        include Enumerable
+
+        attr_reader :next_token, :clz, :params, :items, :i
+
+
+        def initialize(clz=nil, params=[], items=[], next_token=nil)
+            @clz = clz
+            #puts 'class=' + clz.inspect
+            @params = params
+            if @params.size <= 1
+                options = {}
+                @params[1] = options
+            end
+            @items = items
+            @currentset_items = items
+            @next_token = next_token
+            @i = 0
+        end
+
+        def << (val)
+            @items << val
+        end
+
+        def [](*i)
+            puts 'i.inspect=' + i.inspect
+            puts i.size.to_s
+            i.each do |x|
+                puts 'x=' + x.inspect + " -- " + x.class.name
+            end
+            if i.size == 1
+                # either fixnum or range
+                x = i[0]
+                if x.is_a?(Fixnum)
+                    load_to(x)
+                else
+                    # range
+                    end_val = x.exclude_end? ? x.end-1 : x.end
+                    load_to(end_val)
+                end
+            elsif i.size == 2
+                # two fixnums
+                end_val = i[0] + i[1]
+                load_to(end_val)
+            end
+            @items[*i]
+        end
+
+        # Will load items from SimpleDB up to i.
+        def load_to(i)
+            return if @items.size >= i
+            while @items.size < i && !@next_token.nil?
+                load_next_token_set
+            end
+        end
+
+        def first
+            @items[0]
+        end
+
+        def last
+            @items[@items.length-1]
+        end
+
+        def empty?
+            @items.empty?
+        end
+
+        def include?(obj)
+            @items.include?(obj)
+        end
+
+        def size
+            # puts 'SIZE count=' + @count.inspect
+            return @count if @count
+            params_for_count = params.dup
+            params_for_count[0] = :count
+            #puts 'params_for_count=' + params_for_count.inspect
+            @count = clz.find(*params_for_count)
+            # puts '@count=' + @count.to_s
+            @count
+        end
+
+        def length
+            return size
+        end
+
+        def each(&blk)
+            options = @params[1]
+            limit = options[:limit]
+
+            @currentset_items.each do |v|
+#                puts @i.to_s
+                yield v
+                @i += 1
+                if !limit.nil? && @i >= limit
+                    return
+                end
+            end
+            return if @clz.nil?
+
+            # no more items, but is there a next token?
+            unless @next_token.nil?
+                #puts 'finding more items...'
+                #puts 'params in block=' + params.inspect
+                #puts "i from results_array = " + @i.to_s
+
+                load_next_token_set
+                each(&blk)
+            end
+        end
+
+        def load_next_token_set
+            options = @params[1]
+            options[:next_token] = @next_token
+            res = @clz.find(*@params)
+            @currentset_items = res.items # get the real items array from the ResultsArray
+            @currentset_items.each do |item|
+                @items << item
+            end
+            @next_token = res.next_token
+        end
+
+        def delete(item)
+            @items.delete(item)
+        end
+
+        def delete_at(index)
+            @items.delete_at(index)
+        end
+
+    end
+end

data/lib/simple_record.rb

@@ -22,32 +22,194 @@
 # puts 'got=' + mm2.name + ' and he/she is ' + mm.age.to_s + ' years old'
 
 
-
 require 'right_aws'
 require 'sdb/active_sdb'
+#require 'results_array' # why the heck isn't this picking up???
+require File.expand_path(File.dirname(__FILE__) + "/results_array")
+require File.expand_path(File.dirname(__FILE__) + "/stats")
 
 module SimpleRecord
 
-    VERSION = '1.0.10'
+    @@stats = SimpleRecord::Stats.new
+
+    def self.stats
+        @@stats
+    end
+
+    # Create a new handle to an Sdb account. All handles share the same per process or per thread
+    # HTTP connection to Amazon Sdb. Each handle is for a specific account.
+    # The +params+ are passed through as-is to RightAws::SdbInterface.new
+    # Params:
+    #    { :server       => 'sdb.amazonaws.com'  # Amazon service host: 'sdb.amazonaws.com'(default)
+    #      :port         => 443                  # Amazon service port: 80(default) or 443
+    #      :protocol     => 'https'              # Amazon service protocol: 'http'(default) or 'https'
+    #      :signature_version => '0'             # The signature version : '0' or '1'(default)
+    #      :connection_mode  => :default         # options are
+    #                                                  :default (will use best known safe (as in won't need explicit close) option, may change in the future)
+    #                                                  :per_request (opens and closes a connection on every request to SDB)
+    #                                                  :single (one thread across entire app)
+    #                                                  :per_thread (one connection per thread)
+    #                                                  :pool (uses a connection pool with a maximum number of connections - NOT IMPLEMENTED YET)
+    #      :logger       => Logger Object        # Logger instance: logs to STDOUT if omitted
+    def self.establish_connection(aws_access_key=nil, aws_secret_key=nil, params={})
+        RightAws::ActiveSdb.establish_connection(aws_access_key, aws_secret_key, params)
+    end
+
+    def self.close_connection()
+        RightAws::ActiveSdb.close_connection
+    end
+
+    module Callbacks
+        #this bit of code creates a "run_blank" function for everything value in the @@callbacks array.
+        #this function can then be inserted in the appropriate place in the save, new, destroy, etc overrides
+        #basically, this is how we recreate the callback functions
+        @@callbacks=["before_validation", "before_validation_on_create", "before_validation_on_update",
+                     "after_validation", "after_validation_on_create", "after_validation_on_update",
+                     "before_save", "before_create", "before_update",
+                     "after_create", "after_update", "after_save",
+                     "after_destroy"]
+
+        def self.included(base)
+            #puts 'Callbacks included in ' + base.inspect
+
+        end
+    end
 
     class Base < RightAws::ActiveSdb::Base
 
+        include SimpleRecord::Callbacks
+
+
+        def initialize(attrs={})
+            super
+            # Need to deal with objects passed in. iterate through belongs_to perhaps and if in attrs, set the objects id rather than the object itself
+        end
+
+
+        # todo: move into Callbacks module
+        #this bit of code creates a "run_blank" function for everything value in the @@callbacks array.
+        #this function can then be inserted in the appropriate place in the save, new, destroy, etc overrides
+        #basically, this is how we recreate the callback functions
+        @@callbacks.each do |callback|
+            instance_eval <<-endofeval
+
+                #puts 'doing callback=' + callback + ' for ' + self.inspect
+                #we first have to make an initialized array for each of the callbacks, to prevent problems if they are not called
+
+                def #{callback}(*args)
+                    #puts 'callback called in ' + self.inspect + ' with ' + args.inspect
+
+                    #make_dirty(arg_s, value)
+                    #self[arg.to_s]=value
+                    #puts 'value in callback #{callback}=' + value.to_s
+                    args.each do |arg|
+                        cnames = callbacks['#{callback}']
+                        #puts '\tcnames1=' + cnames.inspect + ' for class ' + self.inspect
+                        cnames = [] if cnames.nil?
+                        cnames << arg.to_s if cnames.index(arg.to_s).nil?
+                        #puts '\tcnames2=' + cnames.inspect
+                        callbacks['#{callback}'] = cnames
+                    end
+                end
+
+            endofeval
+        end
+        #puts 'base methods=' + self.methods.inspect
+
+
+        def self.inherited(base)
+            #puts 'SimpleRecord::Base is inherited by ' + base.inspect
+            setup_callbacks(base)
+
+            base.has_dates :created, :updated
+            base.before_create :set_created, :set_updated
+            base.before_update :set_updated
+
+        end
+
+        def self.setup_callbacks(base)
+            instance_eval <<-endofeval
+
+                def callbacks
+                    @callbacks ||= {}
+                    @callbacks
+                end
+
+               def self.defined_attributes
+                    #puts 'class defined_attributes'
+                    @attributes ||= {}
+                    @attributes
+                end
+
+            endofeval
+
+            @@callbacks.each do |callback|
+                class_eval <<-endofeval
+
+             def run_#{callback}
+#                puts 'CLASS CALLBACKS for ' + self.inspect + ' = ' + self.class.callbacks.inspect
+                return true if self.class.callbacks.nil?
+                cnames = self.class.callbacks['#{callback}']
+                cnames = [] if cnames.nil?
+                #cnames += super.class.callbacks['#{callback}'] unless super.class.callbacks.nil?
+#                 puts 'cnames for #{callback} = ' + cnames.inspect
+                return true if cnames.nil?
+                cnames.each { |name|
+                    #puts 'run_  #{name}'
+                  if eval(name) == false # nil should be an ok return, only looking for false
+                    return false
+                  end
+              }
+                #super.run_#{callback}
+              return true
+            end
+
+                endofeval
+            end
+        end
+
+
+        # Holds information about an attribute
+        class Attribute
+            attr_accessor :type, :options
+
+            def initialize(type)
+                @type = type
+            end
+
+        end
+
+
+        def defined_attributes_local
+            #puts 'local defined_attributes'
+            ret = self.class.defined_attributes
+            ret.merge!(self.class.superclass.defined_attributes) if self.class.superclass.respond_to?(:defined_attributes)
+        end
+
+
         attr_accessor :errors
-        @@domain_prefix = ''
-        @domain_name_for_class = nil
+
+        @domain_prefix = ''
+        class << self;
+            attr_accessor :domain_prefix;
+        end
+
+        #@domain_name_for_class = nil
 
         @@cache_store = nil
         # Set the cache to use
         def self.cache_store=(cache)
             @@cache_store = cache
         end
+
         def self.cache_store
             return @@cache_store
         end
 
         # If you want a domain prefix for all your models, set it here.
         def self.set_domain_prefix(prefix)
-            @@domain_prefix = prefix
+            #puts 'set_domain_prefix=' + prefix
+            self.domain_prefix = prefix
         end
 
         # Same as set_table_name
@@ -58,110 +220,135 @@ module SimpleRecord
         # Sets the domain name for this class
         def self.set_domain_name(table_name)
             # puts 'setting domain name for class ' + self.inspect + '=' + table_name
-            @domain_name_for_class = table_name
+            #@domain_name_for_class = table_name
             super
         end
 
-        def self.get_domain_name
+=begin
+ def self.get_domain_name
             # puts 'returning domain_name=' + @domain_name_for_class.to_s
-            return @domain_name_for_class
+            #return @domain_name_for_class
+            return self.domain
         end
 
+=end
 
         def domain
             super # super.domain
         end
 
         def self.domain
-            return self.get_domain_name unless self.get_domain_name.nil?
+            #return self.get_domain_name unless self.get_domain_name.nil?
             d = super
-            domain_name_for_class = @@domain_prefix + d.to_s
-            self.set_domain_name(domain_name_for_class)
+            #puts 'in self.domain, d=' + d.to_s + ' domain_prefix=' + SimpleRecord::Base.domain_prefix.to_s
+            domain_name_for_class = SimpleRecord::Base.domain_prefix + d.to_s
+            #self.set_domain_name(domain_name_for_class)
             domain_name_for_class
         end
 
-        #this bit of code creates a "run_blank" function for everything value in the @@callbacks array.
-        #this function can then be inserted in the appropriate place in the save, new, destroy, etc overrides
-        #basically, this is how we recreate the callback functions
-        @@callbacks=["before_save", "before_create", "after_create", "before_update", "after_update", "after_save", "after_destroy"]
-        @@callbacks.each do |callback|
-            #we first have to make an initialized array for each of the callbacks, to prevent problems if they are not called
-            eval %{
-        @@#{callback}_names=[]
 
-        def self.#{callback}(*args)
-        args.each do |arg|
-          @@#{callback}_names << arg.to_s if @@#{callback}_names.index(arg.to_s).nil?
+        # Since SimpleDB supports multiple attributes per value, the values are an array.
+        # This method will return the value unwrapped if it's the only, otherwise it will return the array.
+        def get_attribute(arg)
+            arg = arg.to_s
+            if self[arg].class==Array
+                if self[arg].length==1
+                    ret = self[arg][0]
+                else
+                    ret = self[arg]
+                end
+            else
+                ret = self[arg]
+            end
+            ret
         end
-#      asdf    @@#{callback}_names=args.map{|arg| arg.to_s}
-      end
 
-        def run_#{callback}
-          @@#{callback}_names.each { |name|
-          unless eval(name)
-            return false
-          end
-          }
-          return true
-        end
-    }
+        def make_dirty(arg, value)
+            # todo: only set dirty if it changed
+            #puts 'making dirty arg=' + arg.to_s + ' --- ' + @dirty.inspect
+            @dirty[arg] = get_attribute(arg) # Store old value (not sure if we need it?)
+            #puts 'end making dirty ' + @dirty.inspect
         end
 
         def self.has_attributes(*args)
-            @@attributes = args
             args.each do |arg|
+                defined_attributes[arg] = SimpleRecord::Base::Attribute.new(:string) if defined_attributes[arg].nil?
                 # define reader method
-                send :define_method, arg do
+                arg_s = arg.to_s # to get rid of all the to_s calls
+                send(:define_method, arg) do
                     ret = nil
-                    if self[arg.to_s].class==Array
-                        if self[arg.to_s].length==1
-                            ret = self[arg.to_s][0]
-                        else
-                            ret = self[arg.to_s]
-                        end
-                    else
-                        ret = self[arg.to_s]
-                    end
+                    ret = get_attribute(arg)
                     return nil if ret.nil?
                     return un_offset_if_int(arg, ret)
                 end
 
                 # define writer method
-                method_name = (arg.to_s+"=")
-                send(:define_method, method_name) do |value|
-                    self[arg.to_s]=value#      end
+                send(:define_method, arg_s+"=") do |value|
+                    make_dirty(arg_s, value)
+                    self[arg_s]=value
+                end
+
+                # Now for dirty methods: http://api.rubyonrails.org/classes/ActiveRecord/Dirty.html
+                # define changed? method
+                send(:define_method, arg_s + "_changed?") do
+                    @dirty.has_key?(arg_s)
+                end
+
+                # define change method
+                send(:define_method, arg_s + "_change") do
+                    old_val = @dirty[arg_s]
+                    return nil if old_val.nil?
+                    [old_val, get_attribute(arg_s)]
+                end
+
+                # define was method
+                send(:define_method, arg_s + "_was") do
+                    old_val = @dirty[arg_s]
+                    old_val
                 end
             end
         end
 
-        @@ints = []
+        def self.has_strings(*args)
+            has_attributes(*args)
+        end
+
+        def self.has_ints(*args)
+            has_attributes(*args)
+            are_ints(*args)
+        end
+
+        def self.has_dates(*args)
+            has_attributes(*args)
+            are_dates(*args)
+        end
+
+        def self.has_booleans(*args)
+            has_attributes(*args)
+            are_booleans(*args)
+        end
+
         def self.are_ints(*args)
             #    puts 'calling are_ints: ' + args.inspect
             args.each do |arg|
-                # todo: maybe @@ints and @@dates should be maps for quicker lookups
-                @@ints << arg if @@ints.index(arg).nil?
+                defined_attributes[arg].type = :int
             end
-#    @@ints = args
-            #    puts 'ints=' + @@ints.inspect
         end
 
-        @@dates = []
         def self.are_dates(*args)
             args.each do |arg|
-                @@dates << arg if @@dates.index(arg).nil?
+                defined_attributes[arg].type = :date
             end
-#    @@dates = args
-            #    puts 'dates=' + @@dates.inspect
         end
 
-        @@booleans = []
         def self.are_booleans(*args)
             args.each do |arg|
-                @@booleans << arg if @@booleans.index(arg).nil?
+                defined_attributes[arg].type = :boolean
             end
         end
 
         @@virtuals=[]
+
         def self.has_virtuals(*args)
             @@virtuals = args
             args.each do |arg|
@@ -170,14 +357,17 @@ module SimpleRecord
             end
         end
 
-        @@belongs_to_map = {}
         # One belongs_to association per call. Call multiple times if there are more than one.
         #
         # This method will also create an {association)_id method that will return the ID of the foreign object
         # without actually materializing it.
         def self.belongs_to(association_id, options = {})
-            @@belongs_to_map[association_id] = options
+            attribute = SimpleRecord::Base::Attribute.new(:belongs_to)
+            defined_attributes[association_id] = attribute
+            attribute.options = options
+            #@@belongs_to_map[association_id] = options
             arg = association_id
+            arg_s = arg.to_s
             arg_id = arg.to_s + '_id'
 
             # todo: should also handle foreign_key http://74.125.95.132/search?q=cache:KqLkxuXiBBQJ:wiki.rubyonrails.org/rails/show/belongs_to+rails+belongs_to&hl=en&ct=clnk&cd=1&gl=us
@@ -187,8 +377,13 @@ module SimpleRecord
 
             # Define reader method
             send(:define_method, arg) do
-                options2 = @@belongs_to_map[arg]
+                attribute = defined_attributes_local[arg]
+                options2 = attribute.options # @@belongs_to_map[arg]
                 class_name = options2[:class_name] || arg.to_s[0...1].capitalize + arg.to_s[1...arg.to_s.length]
+
+                # Camelize classnames with underscores (ie my_model.rb --> MyModel)
+                class_name = class_name.camelize
+
                 #      puts "attr=" + @attributes[arg_id].inspect
                 #      puts 'val=' + @attributes[arg_id][0].inspect unless @attributes[arg_id].nil?
                 ret = nil
@@ -202,7 +397,7 @@ module SimpleRecord
 #          puts 'belongs_to incache=' + ret.inspect
                     end
                     if ret.nil?
-                        to_eval = "#{class_name}.find(@attributes['#{arg_id}'][0], :auto_load=>true)"
+                        to_eval = "#{class_name}.find(@attributes['#{arg_id}'][0])"
 #      puts 'to eval=' + to_eval
                         begin
                             ret = eval(to_eval) # (defined? #{arg}_id)
@@ -220,7 +415,21 @@ module SimpleRecord
                 return ret
             end
 
-            # Define reader ID method
+
+            # Define writer method
+            send(:define_method, arg.to_s + "=") do |value|
+                arg_id = arg.to_s + '_id'
+                if value.nil?
+                    make_dirty(arg_id, nil)
+                    self[arg_id]=nil unless self[arg_id].nil? # if it went from something to nil, then we have to remember and remove attribute on save
+                else
+                    make_dirty(arg_id, value.id)
+                    self[arg_id]=value.id
+                end
+            end
+
+
+            # Define ID reader method for reading the associated objects id without getting the entire object
             send(:define_method, arg_id) do
                 if !@attributes[arg_id].nil? && @attributes[arg_id].size > 0 && @attributes[arg_id][0] != nil && @attributes[arg_id][0] != ''
                     return @attributes[arg_id][0]
@@ -228,13 +437,12 @@ module SimpleRecord
                 return nil
             end
 
-            # Define writer method
-            send(:define_method, arg.to_s + "=") do |value|
-                arg_id = arg.to_s + '_id'
+            # Define writer method for setting the _id directly without the associated object
+            send(:define_method, arg_id + "=") do |value|
                 if value.nil?
-                    self[arg_id]=nil unless self[arg_id].nil? # if it went from something to nil, then we have to remember and remove attribute on save
+                    self[arg_id] = nil unless self[arg_id].nil? # if it went from something to nil, then we have to remember and remove attribute on save
                 else
-                    self[arg_id]=value.id
+                    self[arg_id] = value
                 end
             end
 
@@ -266,27 +474,8 @@ module SimpleRecord
 
         end
 
-        has_attributes :created, :updated
-        before_create :set_created, :set_updated
-        before_update :set_updated
-        are_dates :created, :updated
-
-        def set_created
-#    puts 'SETTING CREATED'
-            #    @created = DateTime.now
-            self[:created] = DateTime.now
-#    @tester = 'some test value'
-            #    self[:tester] = 'some test value'
-        end
-
-        def set_updated
-#    puts 'SETTING UPDATED'
-            #    @updated = DateTime.now
-            self[:updated] = DateTime.now
-#    @tester = 'some test value updated'
-        end
-
         def initialize(*params)
+
             if params[0]
                 #we have to handle the virtuals. Right now, this assumes that all parameters are passed from inside an array
                 #this is the usually the case when the parameters are passed passed via POST and obtained from the params array
@@ -301,6 +490,34 @@ module SimpleRecord
                 super()
             end
             @errors=SimpleRecord_errors.new
+            @dirty = {}
+
+
+        end
+
+        def clear_errors
+            @errors=SimpleRecord_errors.new
+        end
+
+        def []=(attribute, values)
+            make_dirty(attribute, values)
+            super
+        end
+
+
+        def set_created
+#    puts 'SETTING CREATED'
+            #    @created = DateTime.now
+            self[:created] = DateTime.now
+#    @tester = 'some test value'
+            #    self[:tester] = 'some test value'
+        end
+
+        def set_updated
+#    puts 'SETTING UPDATED'
+            #    @updated = DateTime.now
+            self[:updated] = DateTime.now
+#    @tester = 'some test value updated'
         end
 
 
@@ -310,7 +527,7 @@ module SimpleRecord
 
         def self.pad_and_offset(x)
             # todo: add Float, etc
-        #    puts 'padding=' + x.class.name + " -- " + x.inspect
+            #    puts 'padding=' + x.class.name + " -- " + x.inspect
             if x.kind_of? Integer
                 x += @@offset
                 x_str = x.to_s
@@ -318,7 +535,7 @@ module SimpleRecord
                 x_str = '0' + x_str while x_str.size < 20
                 return x_str
             elsif x.respond_to?(:iso8601)
-               #  puts x.class.name + ' responds to iso8601'
+                #  puts x.class.name + ' responds to iso8601'
                 #
                 # There is an issue here where Time.iso8601 on an incomparable value to DateTime.iso8601.
                 # Amazon suggests: 2008-02-10T16:52:01.000-05:00
@@ -329,7 +546,7 @@ module SimpleRecord
                 else
                     x_str = x.getutc.strftime(@@date_format)
                 end
-               #  puts 'utc=' + x_str
+                #  puts 'utc=' + x_str
                 return x_str
             else
                 return x
@@ -345,24 +562,71 @@ module SimpleRecord
             return false
         end
 
+        def valid?
+            errors.clear
+
+#            run_callbacks(:validate)
+            validate
+
+            if new_record?
+#                run_callbacks(:validate_on_create)
+                validate_on_create
+            else
+#                run_callbacks(:validate_on_update)
+                validate_on_update
+            end
+
+            errors.empty?
+        end
+
+        def new_record?
+            # todo: new_record in activesdb should align with how we're defining a new record here, ie: if id is nil
+            super
+        end
+
+        def invalid?
+            !valid?
+        end
+
+        def validate
+            true
+        end
+
+        def validate_on_create
+            true
+        end
+
+        def validate_on_update
+            true
+        end
+
         @create_domain_called = false
 
-        def save(*params)
+        # Options:
+        #   - :except => Array of attributes to NOT save
+        #   - :dirty => true - Will only store attributes that were modified
+        #
+        def save(options={})
             #    puts 'SAVING: ' + self.inspect
+            clear_errors
             is_create = self[:id].nil?
-            ok = pre_save(*params)
+            ok = pre_save(options)
             if ok
                 begin
                     #        puts 'is frozen? ' + self.frozen?.to_s + ' - ' + self.inspect
-                    to_delete = get_atts_to_delete
-                    if super(*params)
+                    if options[:dirty] # Only used in simple_record right now
+                        options[:dirty_atts] = @dirty
+                    end
+                    to_delete = get_atts_to_delete # todo: this should use the @dirty hash now
+#                    puts 'saving'
+                    if super(options)
 #          puts 'SAVED super'
                         self.class.cache_results(self)
                         delete_niled(to_delete)
-                        if run_after_save && is_create ? run_after_create : run_after_update
+                        if (is_create ? run_after_create : run_after_update) && run_after_save
+#                            puts 'all good?'
                             return true
                         else
-                            #I thought about calling destroy here, but rails doesn't behave that way, so neither will I
                             return false
                         end
                     else
@@ -373,7 +637,7 @@ module SimpleRecord
                     if (domain_ok($!))
                         if !@create_domain_called
                             @create_domain_called = true
-                            save(*params)
+                            save(options)
                         else
                             raise $!
                         end
@@ -388,54 +652,57 @@ module SimpleRecord
         end
 
         def pad_and_offset_ints_to_sdb()
-            if !@@ints.nil?
-                for i in @@ints
+
+            defined_attributes_local.each_pair do |name, att_meta|
 #          puts 'int encoding: ' + i.to_s
-                    if !self[i.to_s].nil?
+                if att_meta.type == :int && !self[name.to_s].nil?
 #            puts 'before: ' + self[i.to_s].inspect
-                        #            puts @attributes.inspect
-                        #            puts @attributes[i.to_s].inspect
-                        arr = @attributes[i.to_s]
-                        arr.collect!{ |x| self.class.pad_and_offset(x) }
-                        @attributes[i.to_s] = arr
+                    #            puts @attributes.inspect
+                    #            puts @attributes[i.to_s].inspect
+                    arr = @attributes[name.to_s]
+                    arr.collect!{ |x| self.class.pad_and_offset(x) }
+                    @attributes[name.to_s] = arr
 #            puts 'after: ' + @attributes[i.to_s].inspect
-                    else
-                        #            puts 'was nil'
-                    end
                 end
             end
         end
-         def convert_dates_to_sdb()
-            if !@@dates.nil?
-                for i in @@dates
+
+        def convert_dates_to_sdb()
+
+            defined_attributes_local.each_pair do |name, att_meta|
 #          puts 'int encoding: ' + i.to_s
-                    if !self[i.to_s].nil?
+                if att_meta.type == :date && !self[name.to_s].nil?
 #            puts 'before: ' + self[i.to_s].inspect
-                        #            puts @attributes.inspect
-                        #            puts @attributes[i.to_s].inspect
-                        arr = @attributes[i.to_s]
-                        #puts 'padding date=' + i.to_s
-                        arr.collect!{ |x| self.class.pad_and_offset(x) }
-                        @attributes[i.to_s] = arr
+                    #            puts @attributes.inspect
+                    #            puts @attributes[i.to_s].inspect
+                    arr = @attributes[name.to_s]
+                    #puts 'padding date=' + i.to_s
+                    arr.collect!{ |x| self.class.pad_and_offset(x) }
+                    @attributes[name.to_s] = arr
 #            puts 'after: ' + @attributes[i.to_s].inspect
-                    else
-                        #            puts 'was nil'
-                    end
+                else
+                    #            puts 'was nil'
                 end
             end
         end
 
-        def pre_save(*params)
-            if respond_to?('validate')
-                validate
+        def pre_save(options)
+
+            is_create = self[:id].nil?
+            ok = run_before_validation && (is_create ? run_before_validation_on_create : run_before_validation_on_update)
+            return false unless ok
+
+            validate
+            is_create ? validate_on_create : validate_on_update
 #      puts 'AFTER VALIDATIONS, ERRORS=' + errors.inspect
-                if (!@errors.nil? && @errors.length > 0 )
+            if (!@errors.nil? && @errors.length > 0 )
 #        puts 'THERE ARE ERRORS, returning false'
-                    return false
-                end
+                return false
             end
 
-            is_create = self[:id].nil?
+            ok = run_after_validation && (is_create ? run_after_validation_on_create : run_after_validation_on_update)
+            return false unless ok
+
             ok = respond_to?('before_save') ? before_save : true
             if ok
                 if is_create && respond_to?('before_create')
@@ -445,7 +712,7 @@ module SimpleRecord
                 end
             end
             if ok
-                ok = run_before_save && is_create ? run_before_create : run_before_update
+                ok = run_before_save && (is_create ? run_before_create : run_before_update)
             end
             if ok
 #      puts 'ABOUT TO SAVE: ' + self.inspect
@@ -465,6 +732,7 @@ module SimpleRecord
         end
 
         def get_atts_to_delete
+            # todo: this should use the @dirty hash now
             to_delete = []
             @attributes.each do |key, value|
                 #      puts 'value=' + value.inspect
@@ -477,20 +745,24 @@ module SimpleRecord
 
         # Run pre_save on each object, then runs batch_put_attributes
         # Returns
-        def self.batch_save(objects)
+        def self.batch_save(objects, options={})
             results = []
             to_save = []
             if objects && objects.size > 0
                 objects.each do |o|
-                    ok = o.pre_save
+                    ok = o.pre_save(options)
                     raise "Pre save failed on object [" + o.inspect + "]" if !ok
                     results << ok
-                    next if !ok
+                    next if !ok # todo: this shouldn't be here should it?  raises above
                     o.pre_save2
                     to_save << RightAws::SdbInterface::Item.new(o.id, o.attributes, true)
+                    if to_save.size == 25 # Max amount SDB will accept
+                        connection.batch_put_attributes(domain, to_save)
+                        to_save.clear
+                    end
                 end
             end
-            connection.batch_put_attributes(domain, to_save)
+            connection.batch_put_attributes(domain, to_save) if to_save.size > 0
             results
         end
 
@@ -510,51 +782,14 @@ module SimpleRecord
         end
 
         def un_offset_if_int(arg, x)
-            if !@@ints.nil?
-                for i in @@ints
-#        puts 'unpadding: ' + i.to_s
-                    # unpad and unoffset
-                    if i == arg
-#          puts 'unoffsetting ' + x.to_s
-                        x = un_offset_int(x)
-                    end
-                end
-            end
-            if !@@dates.nil?
-                for d in @@dates
-#        puts 'converting created: ' + self['created'].inspect
-                    if d == arg
-                        x = to_date(x)
-                    end
-#          if !self[d].nil?
-#            self[d].collect!{ |d2|
-#              if d2.is_a?(String)
-#                DateTime.parse(d2)
-#              else
-#                d2
-#              end
-#            }
-#          end
-#        puts 'after=' + self['created'].inspect
-                end
-            end
-            if !@@booleans.nil?
-                for b in @@booleans
-#        puts 'converting created: ' + self['created'].inspect
-                    if b == arg
-                        x = to_bool(x)
-                    end
-#          if !self[d].nil?
-#            self[d].collect!{ |d2|
-#              if d2.is_a?(String)
-#                DateTime.parse(d2)
-#              else
-#                d2
-#              end
-#            }
-#          end
-#        puts 'after=' + self['created'].inspect
-                end
+            att_meta = defined_attributes_local[arg]
+#          puts 'int encoding: ' + i.to_s
+            if att_meta.type == :int
+                x = un_offset_int(x)
+            elsif att_meta.type == :date
+                x = to_date(x)
+            elsif att_meta.type == :boolean
+                x = to_bool(x)
             end
             x
         end
@@ -606,40 +841,15 @@ module SimpleRecord
         end
 
         def unpad_self
-            if !@@ints.nil?
-                for i in @@ints
-#        puts 'unpadding: ' + i.to_s
-                    # unpad and unoffset
-
-                    unpad(i, @attributes)
+            defined_attributes_local.each_pair do |name, att_meta|
+                if att_meta.type == :int
+                    unpad(name, @attributes)
                 end
             end
         end
 
         def reload
             super()
-#    puts 'decoding...'
-
-=begin
-This is done on getters now
- if !@@dates.nil?
-        for d in @@dates
-#        puts 'converting created: ' + self['created'].inspect
-          if !self[d].nil?
-            self[d].collect!{ |d2|
-              if d2.is_a?(String)
-                DateTime.parse(d2)
-              else
-                d2
-              end
-            }
-          end
-#        puts 'after=' + self['created'].inspect
-        end
-      end
-=end
-
-#      unpad_self
         end
 
         def update_attributes(*params)
@@ -678,31 +888,48 @@ This is done on getters now
             end
         end
 
+        @@regex_no_id = /.*Couldn't find.*with ID.*/
+
+        #
+        # Usage:
+        # Find by ID:
+        #   MyModel.find(ID)
+        #
+        # Query:
+        #   MyModel.find(:all, ["name = ?", name], :order=>"created desc", :limit=>10)
+        #
         def self.find(*params)
-            reload=true
-            first=false
-            all=false
-            select=false
+            #puts 'params=' + params.inspect
+            q_type = :all
             select_attributes=[]
 
             if params.size > 0
-                all = params[0] == :all
-                first = params[0] == :first
+                q_type = params[0]
             end
 
-
             # Pad and Offset number attributes
-            options = params[1]
-#    puts 'options=' + options.inspect
-            convert_condition_params(options)
-#    puts 'after collect=' + options.inspect
+            options = {}
+            if params.size > 1
+                options = params[1]
+                #puts 'options=' + options.inspect
+                #puts 'after collect=' + options.inspect
+                convert_condition_params(options)
+            end
+            #puts 'params2=' + params.inspect
 
-            results = all ? [] : nil
+            results = q_type == :all ? [] : nil
             begin
                 results=super(*params)
-                cache_results(results)
+                #puts 'params3=' + params.inspect
+                SimpleRecord.stats.selects += 1
+                if q_type != :count
+                    cache_results(results)
+                    if results.is_a?(Array)
+                        results = SimpleRecord::ResultsArray.new(self, params, results, next_token)
+                    end
+                end
             rescue RightAws::AwsError, RightAws::ActiveSdb::ActiveSdbError
-                puts "RESCUED: " + $!
+                puts "RESCUED: " + $!.message
                 if ($!.message().index("NoSuchDomain") != nil)
                     # this is ok
                 elsif ($!.message() =~ @@regex_no_id)
@@ -714,48 +941,20 @@ This is done on getters now
             return results
         end
 
-        @@regex_no_id = /.*Couldn't find.*with ID.*/
         def self.select(*params)
-            first=false
-            all=false
-            select=false
-            select_attributes=[]
-
-            if params.size > 0
-                all = params[0] == :all
-                first = params[0] == :first
-            end
-
-            options = params[1]
-            convert_condition_params(options)
-
-            results = all ? [] : nil
-            begin
-                results=super(*params)
-                cache_results(results)
-            rescue RightAws::AwsError, RightAws::ActiveSdb::ActiveSdbError
-                if ($!.message().index("NoSuchDomain") != nil)
-                    # this is ok
-                elsif ($!.message() =~ @@regex_no_id)
-                    results = nil
-                else
-                    raise $!
-                end
-            end
-            return results
-
+            return find(*params)
         end
 
         def self.convert_condition_params(options)
-            if !options.nil? && options.size > 0
-                conditions = options[:conditions]
-                if !conditions.nil? && conditions.size > 1
-                    # all after first are values
-                    conditions.collect! { |x|
-                        self.pad_and_offset(x)
-                    }
-                end
+            return if options.nil?
+            conditions = options[:conditions]
+            if !conditions.nil? && conditions.size > 1
+                # all after first are values
+                conditions.collect! { |x|
+                    self.pad_and_offset(x)
+                }
             end
+
         end
 
         def self.cache_results(results)
@@ -766,8 +965,8 @@ This is done on getters now
                     class_name = results.class.name
                     id = results.id
                     cache_key = self.cache_key(class_name, id)
-#        puts 'caching result at ' + cache_key + ': ' + results.inspect
-                    @@cache_store.write(cache_key, results, :expires_in =>10*60)
+                    #puts 'caching result at ' + cache_key + ': ' + results.inspect
+                    @@cache_store.write(cache_key, results, :expires_in =>30)
                 end
             end
         end
@@ -776,9 +975,8 @@ This is done on getters now
             return class_name + "/" + id.to_s
         end
 
-
-
         @@debug=""
+
         def self.debug
             @@debug
         end
@@ -788,7 +986,27 @@ This is done on getters now
         end
 
         def self.table_name
-            return @@domain_prefix + self.class.name.tableize
+            return domain
+        end
+
+        def changed
+            return @dirty.keys
+        end
+
+        def changed?
+            return @dirty.size > 0
+        end
+
+        def changes
+            ret = {}
+            #puts 'in CHANGES=' + @dirty.inspect
+            @dirty.each_pair {|key, value| ret[key] = [value, get_attribute(key)]}
+            return ret
+        end
+
+        def mark_as_old
+            super
+            @dirty = {}
         end
 
     end
@@ -822,6 +1040,14 @@ This is done on getters now
         def full_messages
             return @errors
         end
+
+        def clear
+            @errors.clear
+        end
+
+        def empty?
+            @errors.empty?
+        end
     end
 
     class Activerecordtosdb_subrecord_array
@@ -852,6 +1078,14 @@ This is done on getters now
             return load << ob
         end
 
+        def count
+            return load.count
+        end
+
+        def size
+            return count
+        end
+
         def each(*params, &block)
             return load.each(*params){|record| block.call(record)}
         end
@@ -899,3 +1133,4 @@ This is done on getters now
 
     end
 end
+