zermelo 1.1.0 → 1.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: c9908061576a4c3d173358524eaf988a6d16bd33
-  data.tar.gz: 843acdcd8b10b1b13407a7c1440f30f87412ac71
+  metadata.gz: bc193e2da0a7a65f2a3643520bd99663172a3d2e
+  data.tar.gz: ee336568cf9e99aae87ac54bbe6ce0db00500e42
 SHA512:
-  metadata.gz: 5365e1259edae7641187444a6dce2f903be37ae3787f47c5d12006d8f4dd7cf308e6874e54509dabd1ac336a62c39e166839b8ffd76d8126530e43fcc6488a22
-  data.tar.gz: c405762c072ab4f1fe5d366e9997448ba6f35d4b202a6be33535481abc20a1fdf6cbb1d14f87cd4bae37a082cbd9dbe9be3010785c453dd22622064bf5c3bea7
+  metadata.gz: 46fea7c0eee8e85846323a42ce4a83caee6761e48ad672c7ba70fa0a2ddd862b5d5aac849a8ce7d7d0b9c4c7deebec9bf90d1c918878c9ec73fe8b08a130cb9d
+  data.tar.gz: 0257355f3e92135da783b321db21850adb806758ba1089bf6a3837bf661b93925a66771ad760ad6796914fbeb269a3495720f604dad5977c1d103144143d5756

data/CHANGELOG.md

@@ -1,5 +1,15 @@
 ## Zermelo Changelog
 
+# 1.2.0 - 2015-06-24
+
+* spec cleanup, apply to multiple backends if not backend-specific
+* fix sorted_sets to compose with other queries
+* range queries against sorted sets
+* improve setting association from its inverse side; all callbacks now called properly
+* renamed .delete on associations to .remove
+* also support .remove_ids on multiple associations, so record need not be loaded
+* stub record type
+
 # 1.1.0 - 2015-04-09
 
 * refactored query builder to improve composability

data/README.md

@@ -38,11 +38,11 @@ You can optionally set `Zermelo.logger` to an instance of a Ruby `Logger` class,
 
 ### Class ids
 
-Include **zermelo**'s Record module in the class you want to persist data from:
+Include **zermelo**'s `Zermelo::Records::Redis` module in the class you want to persist data from:
 
 ```ruby
 class Post
-  include Zermelo::Record
+  include Zermelo::Records::Redis
 end
 ```
 
@@ -67,7 +67,7 @@ A data record without any actual data isn't very useful, so let's add a few simp
 
 ```ruby
 class Post
-  include Zermelo::Record
+  include Zermelo::Records::Redis
   define_attributes :title     => :string,
                     :score     => :integer,
                     :timestamp => :timestamp,
@@ -96,8 +96,7 @@ which can then be verified by inspection of the object's attributes, e.g.:
 post.attributes.inpsect # == {:id => '03c839ac-24af-432e-aa58-fd1d4bf73f24', :title => 'Introduction to Zermelo', :score => 100, :timestamp => '2000-01-01 00:00:00 UTC', :published => false}
 ```
 
-Zermelo supports the following simple attribute types, and automatically
-validates that the values are of the correct class, casting if possible:
+Zermelo supports the following simple attribute types, and automatically validates that the values are of the correct class, casting if possible:
 
 | Type       |  Ruby class                        | Notes |
 |------------|-------------------------------|-------|
@@ -116,7 +115,7 @@ So if we add tags to the Post data definition:
 
 ```ruby
 class Post
-  include Zermelo::Record
+  include Zermelo::Records::Redis
   define_attributes :title     => :string,
                     :score     => :integer,
                     :timestamp => :timestamp,
@@ -125,7 +124,7 @@ class Post
 end
 ```
 
-and then create another
+and then create another `Post` instance:
 
 ```ruby
 post = Post.new(:id => 1, :tags => Set.new(['database', 'ORM']))
@@ -139,19 +138,18 @@ SADD post:1:attrs:tags 'database' 'ORM'
 SADD post::attrs:ids 1
 ```
 
-Zermelo supports the following complex attribute types, and automatically
-validates that the values are of the correct class, casting if possible:
+Zermelo supports the following complex attribute types, and automatically validates that the values are of the correct class, casting if possible:
 
-| Type       |  Ruby class   | Notes                                                   |
-|------------|---------------|---------------------------------------------------------|
-| :list      |  Enumerable   | Stored as a Redis [LIST](http://redis.io/commands#list) |
-| :set       |  Array or Set | Stored as a Redis [SET](http://redis.io/commands#set)   |
-| :hash      |  Hash         | Stored as a Redis [HASH](http://redis.io/commands#hash) |
+| Type        |  Ruby class   | Notes                                                   |
+|-------------|---------------|---------------------------------------------------------|
+| :list       |  Enumerable   | Stored as a Redis [LIST](http://redis.io/commands#list) |
+| :set        |  Array or Set | Stored as a Redis [SET](http://redis.io/commands#set)   |
+| :hash       |  Hash         | Stored as a Redis [HASH](http://redis.io/commands#hash) |
+| :sorted_set |  Enumerable   | Stored as a Redis [ZSET](http://redis.io/commands#zset) |
 
-Structure data members must be primitives that will cast OK to and from Redis via the
-driver, thus String, Integer and Float.
+Structure data members must be primitives that will cast OK to and from Redis via the driver, thus String, Integer and Float.
 
-Redis [sorted sets](http://redis.io/commands#sorted_set) are only supported through associations, for which see later on.
+Redis [sorted sets](http://redis.io/commands#sorted_set) are also supported through **zermelo**'s associations (recommended due to the fact that queries can be constructed against them).
 
 ### Validations
 
@@ -161,9 +159,9 @@ So an attribute which should be present:
 
 ```ruby
 class Post
-  include Zermelo::Record
-  define_attributes :title     => :string,
-                    :score     => :integer
+  include Zermelo::Records::Redis
+  define_attributes :title    => :string,
+                    :score    => :integer
   validates :title, :presence => true
 end
 ```
@@ -202,15 +200,15 @@ Another feature added by ActiveModel is the ability to detect changed data in re
 
 ```ruby
 class Author
-  include Zermelo::Record
+  include Zermelo::Records::Redis
 end
 
 class Post
-  include Zermelo::Record
+  include Zermelo::Records::Redis
 end
 
 class Comment
-  include Zermelo::Record
+  include Zermelo::Records::Redis
 end
 
 Author.lock(Post, Comment) do
@@ -224,7 +222,7 @@ Assuming a saved `Post` instance has been created:
 
 ```ruby
 class Post
-  include Zermelo::Record
+  include Zermelo::Records::Redis
   define_attributes :title     => :string,
                     :score     => :integer,
                     :timestamp => :timestamp,
@@ -296,19 +294,19 @@ Instances also have attribute accessors and the various methods included from th
 |Name                       | Type                      | Redis data structure | Notes |
 |---------------------------|---------------------------|----------------------|-------|
 | `has_many`                | one-to-many               | [SET](http://redis.io/commands#set) | |
-| `has_sorted_set`          | one-to-many               | [ZSET](http://redis.io/commands#sorted_set) | |
+| `has_sorted_set`          | one-to-many               | [ZSET](http://redis.io/commands#sorted_set) | Arguments: `:key` (required), `:order` (optional, `:asc` or `:desc`) |
 | `has_one`                 | one-to-one                | [HASH](http://redis.io/commands#hash) | |
 | `belongs_to`              | many-to-one or one-to-one | [HASH](http://redis.io/commands#hash) or [STRING](http://redis.io/commands#string)  | Inverse of any of the above three |
 | `has_and_belongs_to_many` | many-to-many              | 2 [SET](http://redis.io/commands#set)s | Mirrored by an inverse HaBtM association on the other side. |
 
 ```ruby
 class Post
-  include Zermelo::Record
+  include Zermelo::Records::Redis
   has_many :comments, :class_name => 'Comment', :inverse_of => :post
 end
 
 class Comment
-  include Zermelo::Record
+  include Zermelo::Records::Redis
   belongs_to :post, :class_name => 'Post', :inverse_of => :comments
 end
 ```
@@ -319,24 +317,35 @@ Records are added and removed from their parent one-to-many or many-to-many asso
 
 ```ruby
 post.comments.add(comment) # or post.comments << comment
+post.comments.remove(comment)
 ```
 
-Associations' `.add` can also take more than one argument:
+Associations' `.add`/`.remove` can also take more than one argument:
 
 ```ruby
 post.comments.add(comment1, comment2, comment3)
+post.comments.remove(comment1, comment2, comment3)
+```
+
+If you only have ids available, you don't need to `.load` the respective objects, you can instead use `.add_ids`/`.remove_ids`:
+
+```ruby
+post.comments.add_ids("comment_id")
+post.comments.remove_ids("comment_id")
+post.comments.add_ids("comment1_id", "comment2_id", "comment3_id")
+post.comments.remove_ids("comment1_id", "comment2_id", "comment3_id")
 ```
 
 `has_one` associations are simply set with an `=` method on the association:
 
 ```ruby
 class User
-  include Zermelo::Record
+  include Zermelo::Records::Redis
   has_one :preferences, :class_name => 'Preferences', :inverse_of => :user
 end
 
 class Preferences
-  include Zermelo::Record
+  include Zermelo::Records::Redis
   belongs_to :user, :class_name => 'User', :inverse_of => :preferences
 end
 
@@ -348,10 +357,16 @@ prefs.save
 user.preferences = prefs
 ```
 
+and cleared by assigning the association to nil:
+
+```ruby
+user.preferences = nil
+```
+
 The class methods defined above can be applied to associations references as well, so the resulting data will be filtered by the data relationships applying in the association, e.g.
 
 ```ruby
-post     = Post.new(:id => 'a')
+post = Post.new(:id => 'a')
 post.save
 comment1 = Comment.new(:id => '1')
 comment1.save
@@ -364,12 +379,12 @@ post.comments << comment1
 p post.comments.ids # == [1]
 ```
 
-`associated_ids_for` is somewhat of a special case; it uses the smallest/simplest queries possible to get the ids of the associated records of a set of records, e.g. for the data directly above:
+`.associated_ids_for` is somewhat of a special case; it uses the simplest queries possible to get the ids of the associated records of a set of records, e.g. for the data directly above:
 
 ```ruby
 Post.associated_ids_for(:comments)                       # => {'a' => ['1']}
 
-post_b     = Post.new(:id => 'b')
+post_b = Post.new(:id => 'b')
 post_b.save
 post_b.comments << comment2
 comment3 = Comment.new(:id => '3')
@@ -377,7 +392,6 @@ comment3.save
 post.comments << comment3
 
 Post.associated_ids_for(:comments)                       # => {'a' => ['1', '3'], 'b' => ['2']}
-Post.intersect(:id => 'a').associated_ids_for(:comments) # => {'a' => ['1', '3']}
 ```
 
 For `belongs to` associations, you may pass an extra option to `associated_ids_for`, `:inversed => true`, and you'll get the data back as if it were applied from the inverse side; however the data will only cover that used as the query root. Again, assuming the data from the last two code blocks, e.g.
@@ -385,9 +399,6 @@ For `belongs to` associations, you may pass an extra option to `associated_ids_f
 ```ruby
 Comment.associated_ids_for(:post)                    # => {'1' => 'a', '2' => 'b', '3' => 'a'}
 Comment.associated_ids_for(:post, :inversed => true) # => {'a' => ['1', '3'], 'b' => ['2']}
-
-Comment.intersect(:id => ['1', '2']).associated_ids_for(:post) # => {'1' => 'a', '2' => 'b'}
-Comment.intersect(:id => ['1', '2']).associated_ids_for(:post, :inversed => true) # => {'a' => ['1'], 'b' => ['2']}
 ```
 
 ### Class data indexing
@@ -398,7 +409,7 @@ Using the code from the instance attributes section, and adding indexing:
 
 ```ruby
 class Post
-  include Zermelo::Record
+  include Zermelo::Records::Redis
   define_attributes :title     => :string,
                     :score     => :integer,
                     :timestamp => :timestamp,
@@ -436,15 +447,12 @@ SADD post::indices:by_published:boolean:false 03c839ac-24af-432e-aa58-fd1d4bf73f
 
 | Name            | Input                 | Output       | Arguments                             | Options                                  |
 |-----------------|-----------------------|--------------|---------------------------------------|------------------------------------------|
-| intersect       | `set` or `sorted_set` | `set`        | Query hash                            |                                          |
-| union           | `set` or `sorted_set` | `set`        | Query hash                            |                                          |
-| diff            | `set` or `sorted_set` | `set`        | Query hash                            |                                          |
-| intersect_range | `sorted_set`          | `sorted_set` | start (`Integer`), finish (`Integer`) | :desc (`Boolean`), :by_score (`Boolean`) |
-| union_range     | `sorted_set`          | `sorted_set` | start (`Integer`), finish (`Integer`) | :desc (`Boolean`), :by_score (`Boolean`) |
-| diff_range      | `sorted_set`          | `sorted_set` | start (`Integer`), finish (`Integer`) | :desc (`Boolean`), :by_score (`Boolean`) |
+| intersect       | `set` / `sorted_set`  | (as input)   | Query hash                            |                                          |
+| union           | `set` / `sorted_set`  | (as input)   | Query hash                            |                                          |
+| diff            | `set` / `sorted_set`  | (as input)   | Query hash                            |                                          |
 | sort            | `set` or `sorted_set` | `list`       | keys (Symbol or Array of Symbols)     | :limit (`Integer`), :offset (`Integer`)  |
-| offset          | `list`                | `list`       | amount (`Integer`)                    |                                          |
-| limit           | `list`                | `list`       | amount (`Integer`)                    |                                          |
+| offset          | `list` / `sorted_set` | `list`       | amount (`Integer`)                    | :limit (`Integer`)                                        |
+| page            | `list` / `sorted_set` | `list`       | page_number (`Integer`)               | :per_page (`Integer`)                                    |
 
 These queries can be applied against all instances of a class, or against associations belonging to an instance, e.g.
 
@@ -481,21 +489,37 @@ DEL comment::tmp:fe8dd59e4a1197f62d19c8aa942c4ff9
 
 (where the name of the temporary Redis `SET` will of course change every time)
 
-The current implementation of the filtering is somewhat ad-hoc, and has these limitations:
+---
+
+`has_sorted_set` queries can take exact values, or a range bounded in no, one or both directions. (Regular Ruby `Range` objects can't be used as they don't easily support timestamps, so there's a `Zermelo::Filters::IndexRange` class which can be used as a query value instead.)
 
-* no conversion of `list`s back into `set`s is allowed
-* `sort`/`offset`/`limit` can only be used once in a filter chain
+```ruby
+class Comment
+  include Zermelo::Records::Redis
+  define_attributes :created_at => :timestamp
+end
 
-I plan to fix these as soon as I possibly can.
+t = Time.now
+
+comment1 = Comment.new(:id => '1', :created_at => t - 120)
+comment1.save
+comment2 = Comment.new(:id => '2', :created_at => t - 60)
+comment2.save
+
+range = Zermelo::Filters::IndexRange.new(t - 90, t, :by_score => true)
+Comment.ids # ['1', '2']
+Comment.intersect(:created_at => range).ids # ['2']
+
+```
 
 ### Future
 
 Some possible changes:
 
-* pluggable key naming strategies
 * pluggable id generation strategies
+* pluggable key naming strategies
 * instrumentation for benchmarking etc.
-* multiple data backends; there's currently an experimental InfluxDB backend, and more are planned.
+* multiple data backends; there's currently an experimental InfluxDB 0.8 backend (waiting for the Ruby driver to update for 0.9 support).
 
 ## License
 

data/lib/zermelo/associations/association_data.rb

@@ -2,11 +2,12 @@ module Zermelo
   module Associations
     class AssociationData
       attr_writer   :data_klass_name, :related_klass_names
-      attr_accessor :name, :type_klass, :inverse, :sort_key, :callbacks
+      attr_accessor :name, :type_klass, :data_type, :inverse, :sort_key,
+        :sort_order, :callbacks
 
       def initialize(opts = {})
-        [:name, :type_klass, :inverse, :sort_key, :callbacks, :data_klass_name,
-         :related_klass_names].each do |a|
+        [:name, :type_klass, :data_type, :inverse, :sort_key, :sort_order,
+         :callbacks, :data_klass_name, :related_klass_names].each do |a|
 
           send("#{a}=".to_sym, opts[a])
         end

data/lib/zermelo/associations/class_methods.rb

@@ -1,12 +1,10 @@
 require 'zermelo/associations/association_data'
 require 'zermelo/associations/index_data'
 
-require 'zermelo/associations/belongs_to'
-require 'zermelo/associations/has_and_belongs_to_many'
-require 'zermelo/associations/has_many'
-require 'zermelo/associations/has_one'
-require 'zermelo/associations/has_sorted_set'
+require 'zermelo/associations/singular'
+require 'zermelo/associations/multiple'
 require 'zermelo/associations/index'
+require 'zermelo/associations/range_index'
 require 'zermelo/associations/unique_index'
 
 # NB: this module gets mixed in to Zermelo::Record as class methods
@@ -30,6 +28,14 @@ module Zermelo
         nil
       end
 
+      def range_index_by(*args)
+        att_types = attribute_types
+        args.each do |arg|
+          index(::Zermelo::Associations::RangeIndex, arg, :type => att_types[arg])
+        end
+        nil
+      end
+
       def unique_index_by(*args)
         att_types = attribute_types
         args.each do |arg|
@@ -39,31 +45,33 @@ module Zermelo
       end
 
       def has_many(name, args = {})
-        associate(::Zermelo::Associations::HasMany, name, args)
+        associate(::Zermelo::Associations::Multiple, :has_many, name, args)
         nil
       end
 
-      def has_one(name, args = {})
-        associate(::Zermelo::Associations::HasOne, name, args)
+      def has_sorted_set(name, args = {})
+        associate(::Zermelo::Associations::Multiple, :has_sorted_set, name, args)
         nil
       end
 
-      def has_sorted_set(name, args = {})
-        associate(::Zermelo::Associations::HasSortedSet, name, args)
+      def has_and_belongs_to_many(name, args = {})
+        associate(::Zermelo::Associations::Multiple, :has_and_belongs_to_many, name, args)
         nil
       end
 
-      def has_and_belongs_to_many(name, args = {})
-        associate(::Zermelo::Associations::HasAndBelongsToMany, name, args)
+      def has_one(name, args = {})
+        associate(::Zermelo::Associations::Singular, :has_one, name, args)
         nil
       end
 
       def belongs_to(name, args = {})
-        associate(::Zermelo::Associations::BelongsTo, name, args)
+        associate(::Zermelo::Associations::Singular, :belongs_to, name, args)
         nil
       end
       # end used by client classes
 
+      private
+
       # used internally by other parts of Zermelo to implement the above
       # configuration
 
@@ -77,15 +85,12 @@ module Zermelo
           @association_data.values.each do |data|
             klass = data.data_klass
             next if visited.include?(klass)
-            visited |= klass.associated_classes(visited, false)
+            visited |= klass.send(:associated_classes, visited, false)
           end
         end
         visited
       end
 
-      # TODO for each association: check whether it has changed
-      # would need an instance-level hash with association name as key,
-      #   boolean 'changed' value
       def with_associations(record)
         @lock.synchronize do
           @association_data ||= {}
@@ -112,14 +117,6 @@ module Zermelo
        end
       # end used internally within Zermelo
 
-      # # TODO  can remove need for some of the inverse mapping
-      # # was inverse_of(source, klass)
-      # with_association_data do |d|
-      #   d.detect {|name, data| data.klass == klass && data.inverse == source}
-      # end
-
-      private
-
       def add_index_data(klass, name, args = {})
         return if name.nil?
 
@@ -151,12 +148,7 @@ module Zermelo
         instance_eval idx, __FILE__, __LINE__
       end
 
-      def add_association_data(klass, name, args = {})
-
-        # TODO have inverse be a reference (or copy?) of the association data
-        # record for that inverse association; would need to defer lookup until
-        # all data in place for all assocs, so might be best if looked up and
-        # cached on first use
+      def add_association_data(klass, type, name, args = {})
         inverse = if args[:inverse_of].nil? || args[:inverse_of].to_s.empty?
           nil
         else
@@ -164,13 +156,10 @@ module Zermelo
         end
 
         callbacks = case klass.name
-        when ::Zermelo::Associations::HasMany.name,
-             ::Zermelo::Associations::HasSortedSet.name,
-             ::Zermelo::Associations::HasAndBelongsToMany.name
-          [:before_add, :after_add, :before_remove, :after_remove]
-        when ::Zermelo::Associations::HasOne.name,
-             ::Zermelo::Associations::BelongsTo.name
-          [:before_set, :after_set, :before_clear, :after_clear]
+        when ::Zermelo::Associations::Multiple.name
+          [:before_add, :after_add, :before_remove, :after_remove, :before_read, :after_read]
+        when ::Zermelo::Associations::Singular.name
+          [:before_set, :after_set, :before_clear, :after_clear, :before_read, :after_read]
         else
           []
         end
@@ -178,6 +167,7 @@ module Zermelo
         data = Zermelo::Associations::AssociationData.new(
           :name                => name,
           :data_klass_name     => args[:class_name],
+          :data_type           => type,
           :type_klass          => klass,
           :inverse             => inverse,
           :related_klass_names => args[:related_class_names],
@@ -186,8 +176,10 @@ module Zermelo
                                   }
         )
 
-        if klass.name == Zermelo::Associations::HasSortedSet.name
-          data.sort_key = (args[:key] || :id)
+        if :has_sorted_set.eql?(type)
+          data.sort_key   = args[:key]
+          data.sort_order =
+            !args[:order].nil? && :desc.eql?(args[:order].to_sym) ? :desc : :asc
         end
 
         @lock.synchronize do
@@ -196,25 +188,20 @@ module Zermelo
         end
       end
 
-      def associate(klass, name, args = {})
+      def associate(klass, type, name, args = {})
         return if name.nil?
 
-        add_association_data(klass, name, args)
+        add_association_data(klass, type, name, args)
 
         assoc = case klass.name
-        when ::Zermelo::Associations::HasMany.name,
-             ::Zermelo::Associations::HasSortedSet.name,
-             ::Zermelo::Associations::HasAndBelongsToMany.name
-
+        when ::Zermelo::Associations::Multiple.name
           %Q{
             def #{name}
               #{name}_proxy
             end
           }
 
-        when ::Zermelo::Associations::HasOne.name,
-             ::Zermelo::Associations::BelongsTo.name
-
+        when ::Zermelo::Associations::Singular.name
           %Q{
             def #{name}
               #{name}_proxy.value
@@ -232,7 +219,7 @@ module Zermelo
           def #{name}_proxy
             raise "Associations cannot be invoked for records without an id" if self.id.nil?
 
-            @#{name}_proxy ||= #{klass.name}.new(self, '#{name}')
+            @#{name}_proxy ||= #{klass.name}.new(:#{type}, self.class, self.id, '#{name}')
           end
           private :#{name}_proxy
         }

data/lib/zermelo/associations/index.rb

@@ -1,5 +1,7 @@
 # NB index instances are all internal to zermelo, not user-accessible
 
+require 'zermelo/records/key'
+
 module Zermelo
   module Associations
     class Index
@@ -29,7 +31,7 @@ module Zermelo
 
       def move_id(id, value_from, indexer_to, value_to)
         return unless indexer = key(value_from)
-        @backend.move(indexer, id, indexer_to.key(value_to))
+        @backend.move(indexer, id, indexer_to.key(value_to), id)
       end
 
       def key(value)