perobs 2.0.1 → 2.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: a2fdb476312688171e078295e7b128663885249e
-  data.tar.gz: 26b0422a60677e6120ded2ff43943a8b805304ca
+  metadata.gz: 50292da8f34e68427887d224c56dc2070242fc27
+  data.tar.gz: f3f41eca90b32ebefeeae05eaf655e99130a3af8
 SHA512:
-  metadata.gz: b415f4d0e12b9fd5e62ec6bc801911651970b70fcb52609a2f24c2d13b86bdcaa668207ac4723589ca8538950da9ddea23aae221e0d1b321c07574c2d211e619
-  data.tar.gz: e52498a83b9a21bc6514ec6e61d6ca310d9054ec68801042e4ed65cda8f6a4276bfd3e6b325daf22d6fa79084b787b72cabc77cddda6a0bf6679f1a83148d3c6
+  metadata.gz: 2ed683b2e8ae0cd9a6bc4531a8f70f6d2f0d98dc5f3f4947ea64f24d715e3c5d050317d6c29ab481820f25cb302e4d8c1e67ca5cf8048c0e1354d732489aef94
+  data.tar.gz: 2dce8dff389bc974126a88e9fdd5d042f5071e13c6560a2eeab63b05ae536ac209dfa4cddbcccac0e4a2332d46c82ea27894cc0e3a7de1c32b016f515920fbd6

data/README.md

@@ -3,48 +3,77 @@
 PEROBS is a library that provides a persistent object store for Ruby
 objects. Objects of your classes can be made persistent by deriving
 them from PEROBS::Object. They will be in memory when needed and
-transparently stored into a persistent storage. Currently only
-filesystem based storage is supported, but back-ends for key/value
-databases can be easily added.
+transparently stored into a persistent storage.
 
 This library is ideal for Ruby applications that work on huge, mostly
 constant data sets and usually handle a small subset of the data at a
 time. To ensure data consistency of a larger data set, you can use
-transactions to make modifications of multiple objects atomic.
+transactions to make modifications of multiple objects atomicaly.
 Transactions can be nested and are aborted when an exception is
 raised.
 
 ## Usage
 
-It features a garbage collector that removes all objects that are no
-longer in use. A build-in cache keeps access latencies to recently
-used objects low and lazily flushes modified objects into the
-persistend back-end when not using transactions.
-
-Persistent objects must be created by deriving your class from
-PEROBS::Object. Only instance variables that are declared via
-po_attr will be persistent. All objects that are stored in persistent
-instance variables must provide a to_json() method that generates JSON
-syntax that can be also parsed into their original object again. It is
-required that references to other objects are all going to persistent
-objects again.
-
-There are currently 3 kinds of persistent objects available:
+The objects that you want to persist must be of a class that has been
+derived from PEROBS::BaseObject. PEROBS already provides 3 such
+classes:
 
 * PEROBS::Object is the base class for all your classes that should be
-  persistent.
+  persistent. You can determine which instance variables should be
+  persisted and what default values should be used.
 
 * PEROBS::Array provides an interface similar to the built-in Array class
-  but its objects are automatically stored.
+  but its objects are automatically persisted.
 
 * PEROBS::Hash provides an interface similar to the built-in Hash
-  class but its objects are automatically stored.
-
-You must create at least one PEROBS::Store object that owns your
-persistent objects. The store provides the persistent database. If you
-are using the default serializer (JSON), you can only use the subset
-of Ruby types that JSON supports.  Alternatively, you can use Marshal
-or YAML which support almost every Ruby data type.
+  class but its objects are automatically persisted.
+
+When you derive your own class from PEROBS::Object you need to
+specify which instance variables should be persistent. By using
+po_attr you can provide a list of symbols that describe the instance
+variables to persist. This will also create getter and setter methods
+for these instance varables.  You can use attr_init in the constructor
+to define default values for your persistent objects.
+
+To start off you must create at least one PEROBS::Store object that
+owns your persistent objects. The store provides the persistent
+database. A persistent object is tied to the creating store for its
+whole lifetime. By default, PEROBS::Store uses an on-disk database in the
+directory you specify. But you can use key/value databases as well.
+Currently only Amazon DynamoDB is supported. You can create your own
+key/value database wrapper with little effort.
+
+When creating the store you can also specify the serializer to use.
+The serializer controls how your data is converted to be stored in the
+database.  The default serializer (JSON), you can only use the subset
+of Ruby types that JSON supports. See http://www.json.org/ for
+details. Alternatively, you can use Marshal or YAML which support
+almost every Ruby data type. YAML is much slower than JSON and Marshal
+is not guaranteed to be compatible between Ruby versions.
+
+Once you have created a store you can assign objects to it. All
+persistent objects must be created with @store.new(). This is
+necessary as you will only deal with proxy objects in your code.
+Except for the member methods, you will never deal with the objects
+directly. Instead @store.new() returns a POXReference object that acts
+as a transparent proxy. This proxy is needed as your code never knows
+if the actual object is really loaded into the memory or not. PEROBS
+will handle this transparently for you.
+
+A build-in cache keeps access latencies to recently used objects low
+and lazily flushes modified objects into the persistend back-end when
+not using transactions.  It also features a garbage collector that
+removes all objects that are no longer in use. 
+
+So what does 'in use' mean? You can assign a few objects to the store
+directly. The store acts like a hash. These root objects can then
+reference other persistent objects and so on. The garbage collector
+will find all objects that are reachable from the root objects and
+discards all other from the database. You have to invoke the garbage
+collector manually with Store.gc(). Depending on the size of your
+database it can take some time. It is recommended that you don't use
+persistend objects for temporary objects in your code. Every created
+object will end up in the database end needs to be garbage collected.
 
 Here is an example how to use PEROBS. Let's define a class that models
 a person with their family relations.
@@ -106,6 +135,19 @@ accesses, you must manually mark the instance as modified by calling
 Object::mark_as_modified(). If that is forgotten, the change will
 reside in memory but might not be persisted into the database.
 
+### Use of proxy objects
+
+Your code should never deal with the persistent objects directly. The
+PEROBS API takes care that you will always get a proxy object. The
+only exception to this rule is the code in the instance methods. By
+design, this code operates on the real object. The only caveat here is
+the use of self(). If you pass the result of self() to another object
+you will leak a PEROBS::ObjectBase derived object into your data
+structures.  PEROBS will watch for this and will throw an exception
+when it detects such objects. Just remember to use myself() instead of
+self() if you want to pass a reference to the current persistent
+object to another object.
+
 ## Installation
 
 Add this line to your application's Gemfile:

data/lib/perobs/Array.rb

@@ -119,7 +119,21 @@ module PEROBS
 
     def _serialize
       @data.map do |v|
-        v.respond_to?(:is_poxreference?) ? POReference.new(v.id) : v
+        if v.respond_to?(:is_poxreference?)
+          POReference.new(v.id)
+        else
+          # Outside of the PEROBS library all PEROBS::ObjectBase derived
+          # objects should not be used directly. The library only exposes them
+          # via POXReference proxy objects.
+          if v.is_a?(ObjectBase)
+            raise RuntimeError, 'A PEROBS::ObjectBase object escaped! ' +
+              "It is stored in a PEROBS::Array at index #{@data.index(v)}. " +
+              'Have you used self() instead of myself() to' +
+              "get the reference of this PEROBS object?\n" +
+              v.inspect
+          end
+          v
+        end
       end
     end
 

data/lib/perobs/BTreeBlob.rb

@@ -2,7 +2,7 @@
 #
 # = BTreeBlob.rb -- Persistent Ruby Object Store
 #
-# Copyright (c) 2015 by Chris Schlaeger <chris@taskjuggler.org>
+# Copyright (c) 2015, 2016 by Chris Schlaeger <chris@taskjuggler.org>
 #
 # MIT License
 #
@@ -137,12 +137,23 @@ module PEROBS
     end
 
     # Remove all entries from the index that have not been marked.
+    # @return [Array] List of deleted object IDs.
     def delete_unmarked_entries
+      deleted_ids = []
       # First remove the entry from the hash table.
-      @entries_by_id.delete_if { |id, e| e[MARKED] == 0 }
+      @entries_by_id.delete_if do |id, e|
+        if e[MARKED] == 0
+          deleted_ids << id
+          true
+        else
+          false
+        end
+      end
       # Then delete the entry itself.
       @entries.delete_if { |e| e[MARKED] == 0 }
       write_index
+
+      deleted_ids
     end
 
     # Run a basic consistency check.

data/lib/perobs/BTreeDB.rb

@@ -2,7 +2,7 @@
 #
 # = BTreeDB.rb -- Persistent Ruby Object Store
 #
-# Copyright (c) 2015 by Chris Schlaeger <chris@taskjuggler.org>
+# Copyright (c) 2015, 2016 by Chris Schlaeger <chris@taskjuggler.org>
 #
 # MIT License
 #
@@ -158,8 +158,11 @@ module PEROBS
 
     # Permanently delete all objects that have not been marked. Those are
     # orphaned and are no longer referenced by any actively used object.
+    # @return [Array] List of IDs that have been removed from the DB.
     def delete_unmarked_objects
-      each_blob { |blob| blob.delete_unmarked_entries }
+      deleted_ids = []
+      each_blob { |blob| deleted_ids += blob.delete_unmarked_entries }
+      deleted_ids
     end
 
     # Mark an object.

data/lib/perobs/Cache.rb

@@ -2,7 +2,7 @@
 #
 # = Cache.rb -- Persistent Ruby Object Store
 #
-# Copyright (c) 2015 by Chris Schlaeger <chris@taskjuggler.org>
+# Copyright (c) 2015, 2016 by Chris Schlaeger <chris@taskjuggler.org>
 #
 # MIT License
 #
@@ -31,7 +31,7 @@ module PEROBS
 
   # The Cache provides two functions for the PEROBS Store. It keeps some
   # amount of objects in memory to substantially reduce read access latencies.
-  # It # also stores a list of objects that haven't been synced to the
+  # It also stores a list of objects that haven't been synced to the
   # permanent store yet to accelerate object writes.
   class Cache
 

data/lib/perobs/DynamoDB.rb

@@ -2,7 +2,7 @@
 #
 # = DynamoDB.rb -- Persistent Ruby Object Store
 #
-# Copyright (c) 2015 by Chris Schlaeger <chris@taskjuggler.org>
+# Copyright (c) 2015, 2016 by Chris Schlaeger <chris@taskjuggler.org>
 #
 # MIT License
 #
@@ -134,10 +134,17 @@ module PEROBS
 
     # Permanently delete all objects that have not been marked. Those are
     # orphaned and are no longer referenced by any actively used object.
+    # @return [Array] List of object IDs of the deleted objects.
     def delete_unmarked_objects
+      deleted_ids = []
       each_item do |id|
-        dynamo_delete_item(id) unless dynamo_is_marked?(id)
+        unless dynamo_is_marked?(id)
+          dynamo_delete_item(id)
+          deleted_ids << id
+        end
       end
+
+      deleted_ids
     end
 
     # Mark an object.

data/lib/perobs/Hash.rb

@@ -118,8 +118,25 @@ module PEROBS
 
     def _serialize
       data = {}
-      @data.each { |k, v| data[k] = v.respond_to?(:is_poxreference?) ?
-                                    POReference.new(v.id) : v }
+
+      @data.each do |k, v|
+        if v.respond_to?(:is_poxreference?)
+          data[k] = POReference.new(v.id)
+        else
+          # Outside of the PEROBS library all PEROBS::ObjectBase derived
+          # objects should not be used directly. The library only exposes them
+          # via POXReference proxy objects.
+          if v.is_a?(ObjectBase)
+            raise RuntimeError, 'A PEROBS::ObjectBase object escaped! ' +
+              "It is stored in a PEROBS::Hash with key #{k.inspect}. " +
+              'Have you used self() instead of myself() to' +
+              "get the reference of this PEROBS object?\n" +
+              v.inspect
+          end
+          data[k] = v
+        end
+      end
+
       data
     end
 

data/lib/perobs/Object.rb

@@ -175,13 +175,6 @@ module PEROBS
       _all_attributes.each do |attr|
         ivar = ('@' + attr.to_s).to_sym
         value = instance_variable_get(ivar)
-        #if (value = instance_variable_get(ivar)).is_a?(ObjectBase)
-        #  raise ArgumentError, "The instance variable #{ivar} contains a " +
-        #                       "reference to a PEROBS::ObjectBase object! " +
-        #                       "This is not allowed. You must use the " +
-        #                       "accessor method to assign a reference to " +
-        #                       "another PEROBS object."
-        #end
         attributes[attr.to_s] = value.respond_to?(:is_poxreference?) ?
           POReference.new(value.id) : value
       end
@@ -189,33 +182,28 @@ module PEROBS
     end
 
     def _set(attr, val)
-      ivar = ('@' + attr.to_s).to_sym
-      if !val.respond_to?(:is_poxreference?) && val.is_a?(ObjectBase)
+      if val.is_a?(ObjectBase)
         # References to other PEROBS::Objects must be handled somewhat
         # special.
         if @store != val.store
           raise ArgumentError, 'The referenced object is not part of this store'
         end
-        # To release the object from the Ruby object list later, we store the
-        # PEROBS::Store ID of the referenced object instead of the actual
-        # reference.
-        instance_variable_set(ivar, POXReference.new(@store, val._id))
-      else
-        instance_variable_set(ivar, val)
+        unless val.respond_to?(:is_poxreference?)
+          raise ArgumentError, 'A PEROBS::ObjectBase object escaped! ' +
+                               'Have you used self() instead of myself() to' +
+                               'get the reference of the PEROBS object that ' +
+                               'you are trying to assign here?'
+        end
       end
+      instance_variable_set(('@' + attr.to_s).to_sym, val)
       # Let the store know that we have a modified object.
-      @store.cache.cache_write(self)
+      mark_as_modified
 
       val
     end
 
     def _get(attr)
-      value = instance_variable_get(('@' + attr.to_s).to_sym)
-      if value.respond_to?(:is_poxreference?)
-        @store.object_by_id(value.id)
-      else
-        value
-      end
+      instance_variable_get(('@' + attr.to_s).to_sym)
     end
 
     def _all_attributes

data/lib/perobs/ObjectBase.rb

@@ -2,7 +2,7 @@
 #
 # = ObjectBase.rb -- Persistent Ruby Object Store
 #
-# Copyright (c) 2015 by Chris Schlaeger <chris@taskjuggler.org>
+# Copyright (c) 2015, 2016 by Chris Schlaeger <chris@taskjuggler.org>
 #
 # MIT License
 #
@@ -34,7 +34,7 @@ module PEROBS
   # since it's no longer referenced once it has been evicted from the
   # PEROBS::Store cache. The POXReference objects function as a transparent
   # proxy for the objects they are referencing.
-  class POXReference  < BasicObject
+  class POXReference < BasicObject
 
     attr_reader :store, :id
 
@@ -47,12 +47,13 @@ module PEROBS
     # Proxy all calls to unknown methods to the referenced object.
     def method_missing(method_sym, *args, &block)
       unless (obj = _referenced_object)
-        raise ::RuntimeError, "Internal consistency error. No object with " +
-          "ID #{@id} found in the store"
+        ::Kernel.raise ::RuntimeError,
+          "Internal consistency error. No object with ID #{@id} found in " +
+          'the store.'
       end
       if obj.respond_to?(:is_poxreference?)
-        raise ::RuntimeError,
-          "POXReference that references a POXReference found"
+        ::Kernel.raise ::RuntimeError,
+          "POXReference that references a POXReference found."
       end
       obj.send(method_sym, *args, &block)
     end
@@ -86,6 +87,18 @@ module PEROBS
       _referenced_object == obj
     end
 
+    # BasicObject provides a equal?() method that prevents method_missing from
+    # being called. So we have to pass the call manually to the referenced
+    # object.
+    # @param obj object to compare this object with.
+    def equal?(obj)
+      if obj.respond_to?(:is_poxreference?)
+        _referenced_object.equal?(obj._referenced_object)
+      else
+        _referenced_object.equal?(obj)
+      end
+    end
+
     # Shortcut to access the _id() method of the referenced object.
     def _id
       @id
@@ -102,7 +115,7 @@ module PEROBS
   # common to all classes of persistent objects.
   class ObjectBase
 
-    attr_reader :_id, :store
+    attr_reader :_id, :store, :myself
 
     # New PEROBS objects must always be created by calling # Store.new().
     # PEROBS users should never call this method or equivalents of derived
@@ -110,17 +123,29 @@ module PEROBS
     def initialize(store)
       @store = store
       unless @store.object_creation_in_progress
-        raise ::RuntimeError,
+        ::Kernel.raise ::RuntimeError,
           "All PEROBS objects must exclusively be created by calling " +
           "Store.new(). Never call the object constructor directly."
       end
       @_id = @store.db.new_id
+      ObjectSpace.define_finalizer(self, ObjectBase._finalize(@store, @_id))
       @_stash_map = nil
+      # Allocate a proxy object for this object. User code should only operate
+      # on this proxy, never on self.
+      @myself = POXReference.new(@store, @_id)
 
       # Let the store know that we have a modified object.
       @store.cache.cache_write(self)
     end
 
+    # This method generates the destructor for the objects of this class. It
+    # is done this way to prevent the Proc object hanging on to a reference to
+    # self which would prevent the object from being collected. This internal
+    # method is not intended for users to call.
+    def ObjectBase._finalize(store, id)
+      proc { store._collect(id) }
+    end
+
     public
 
     # This method can be overloaded by derived classes to do some massaging on
@@ -157,10 +182,7 @@ module PEROBS
 
       klass = store.class_map.id_to_class(db_obj['class_id'])
       # Call the constructor of the specified class.
-      obj = store.construct_po(Object.const_get(klass))
-      # The object gets created with a new ID by default. We need to restore
-      # the old one.
-      obj._change_id(id)
+      obj = store._construct_po(Object.const_get(klass), id)
       obj._deserialize(db_obj['data'])
       obj.post_restore
 

data/lib/perobs/Store.rb

@@ -2,7 +2,7 @@
 #
 # = Store.rb -- Persistent Ruby Object Store
 #
-# Copyright (c) 2015 by Chris Schlaeger <chris@taskjuggler.org>
+# Copyright (c) 2015, 2016 by Chris Schlaeger <chris@taskjuggler.org>
 #
 # MIT License
 #
@@ -26,6 +26,7 @@
 # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 
 require 'set'
+require 'weakref'
 
 require 'perobs/Cache'
 require 'perobs/ClassMap'
@@ -128,16 +129,18 @@ module PEROBS
       # the Store.new() call by PEROBS users.
       @object_creation_in_progress = false
 
+      # List of PEROBS objects that are currently available as Ruby objects
+      # hashed by their ID.
+      @in_memory_objects = {}
+
       # The Cache reduces read and write latencies by keeping a subset of the
       # objects in memory.
       @cache = Cache.new(options[:cache_bits] || 16)
 
       # The named (global) objects IDs hashed by their name
       unless (@root_objects = object_by_id(0))
-        @root_objects = construct_po(Hash)
-
         # The root object hash always has the object ID 0.
-        @root_objects._change_id(0)
+        @root_objects = _construct_po(Hash, 0)
         # The ID change removes it from the write cache. We need to add it
         # again.
         @cache.cache_write(@root_objects)
@@ -152,26 +155,39 @@ module PEROBS
     #        constructor of the specified class.
     # @return [POXReference] A reference to the newly created object.
     def new(klass, *args)
-      POXReference.new(self, construct_po(klass, *args)._id)
+      _construct_po(klass, nil, *args).myself
     end
 
     # For library internal use only!
-    def construct_po(klass, *args)
+    # This method will create a new PEROBS object.
+    # @param klass [BasicObject] Class of the object to create
+    # @param id [Fixnum, Bignum or nil] Requested object ID or nil
+    # @param *args [Array] Arguments to pass to the object constructor.
+    # @return [BasicObject] Newly constructed PEROBS object
+    def _construct_po(klass, id, *args)
+      unless klass.is_a?(BasicObject)
+        raise ArgumentError, "#{klass} is not a BasicObject derivative"
+      end
       @object_creation_in_progress = true
       obj = klass.new(self, *args)
       @object_creation_in_progress = false
+      # If a specific object ID was requested we need to set it now.
+      obj._change_id(id) if id
+      # Add the new object to the in-memory list. We only store a weak
+      # reference to the object so it can be garbage collected. When this
+      # happens the object finalizer is triggered and calls _forget() to
+      # remove the object from this hash again.
+      @in_memory_objects[obj._id] = WeakRef.new(obj)
       obj
     end
 
-
     # Delete the entire store. The store is no longer usable after this
     # method was called.
     def delete_store
       @db.delete_database
-      @class_map = @cache = @root_objects = nil
+      @db = @class_map = @cache = @root_objects = nil
     end
 
-
     # Store the provided object under the given name. Use this to make the
     # object a root or top-level object (think global variable). Each store
     # should have at least one root object. Objects that are not directly or
@@ -187,13 +203,10 @@ module PEROBS
         return nil
       end
 
-      if obj.respond_to?(:is_poxreference?)
-        obj = obj._referenced_object
-      end
       # We only allow derivatives of PEROBS::Object to be stored in the
       # store.
       unless obj.is_a?(ObjectBase)
-        raise ArgumentError, "Object must be of class PEROBS::Object but "
+        raise ArgumentError, 'Object must be of class PEROBS::Object but ' +
                              "is of class #{obj.class}"
       end
 
@@ -203,8 +216,6 @@ module PEROBS
 
       # Store the name and mark the name list as modified.
       @root_objects[name] = obj._id
-      # Add the object to the in-memory storage list.
-      @cache.cache_write(obj)
 
       obj
     end
@@ -217,7 +228,7 @@ module PEROBS
       # Return nil if there is no object with that name.
       return nil unless (id = @root_objects[name])
 
-      object_by_id(id)
+      POXReference.new(self, id)
     end
 
     # Flush out all modified objects to disk and shrink the in-memory list if
@@ -233,6 +244,7 @@ module PEROBS
     # from the back-end storage. The garbage collector is not invoked
     # automatically. Depending on your usage pattern, you need to call this
     # method periodically.
+    # @return [Fixnum] The number of collected objects
     def gc
       sync
       mark
@@ -243,21 +255,27 @@ module PEROBS
     # public API and should never be called by outside users. It's purely
     # intended for internal use.
     def object_by_id(id)
-      if (obj = @cache.object_by_id(id))
+      if (obj = @in_memory_objects[id])
         # We have the object in memory so we can just return it.
-        return obj
-      else
-        # We don't have the object in memory. Let's find it in the storage.
-        if @db.include?(id)
-          # Great, object found. Read it into memory and return it.
-          obj = ObjectBase::read(self, id)
-          # Add the object to the in-memory storage list.
-          @cache.cache_read(obj)
-
-          return obj
+        begin
+          return obj.__getobj__
+        rescue WeakRef::RefError
+          # Due to a race condition the object can still be in the
+          # @in_memory_objects list but has been collected already by the Ruby
+          # GC. In that case we need to load it again.
         end
       end
 
+      # We don't have the object in memory. Let's find it in the storage.
+      if @db.include?(id)
+        # Great, object found. Read it into memory and return it.
+        obj = ObjectBase::read(self, id)
+        # Add the object to the in-memory storage list.
+        @cache.cache_read(obj)
+
+        return obj
+      end
+
       # The requested object does not exist. Return nil.
       nil
     end
@@ -268,47 +286,24 @@ module PEROBS
     # unreadable object is found, the reference will simply be deleted.
     # @param repair [TrueClass/FalseClass] true if a repair attempt should be
     #        made.
+    # @return [Fixnum] The number of references to bad objects found.
     def check(repair = true)
+      # All objects must have in-db version.
+      sync
       # Run basic consistency checks first.
       @db.check_db(repair)
 
+      # We will use the mark to mark all objects that we have checked already.
+      # Before we start, we need to clear all marks.
       @db.clear_marks
-      # A buffer to hold a working set of object IDs.
-      stack = []
-      # First we check the top-level objects. They are only added to the
-      # working set if they are OK.
+
+      errors = 0
       @root_objects.each do |name, id|
-        unless @db.check(id, repair)
-          stack << id
-        end
+        errors += check_object(id, repair)
       end
-      if repair
-        # Delete any top-level object that is defective.
-        stack.each { |id| @root_objects.delete(id) }
-        # The remaining top-level objects are the initial working set.
-        stack = @root_objects.values
-      else
-        # The initial working set must only be OK objects.
-        stack = @root_objects.values - stack
-      end
-      stack.each { |id| @db.mark(id) }
+      @root_objects.delete_if { |name, id| !@db.check(id, false) }
 
-      while !stack.empty?
-        id = stack.pop
-        (obj = object_by_id(id))._referenced_object_ids.each do |id|
-          # Add all found references that have passed the check to the working
-          # list for the next iterations.
-          if @db.check(id, repair)
-            unless @db.is_marked?(id)
-              stack << id
-              @db.mark(id)
-            end
-          elsif repair
-            # Remove references to bad objects.
-            obj._delete_reference_to_id(id)
-          end
-        end
-      end
+      errors
     end
 
     # This method will execute the provided block as an atomic transaction
@@ -339,7 +334,7 @@ module PEROBS
       while !stack.empty?
         # Get an object index from the stack.
         obj = object_by_id(id = stack.pop)
-        yield(obj) if block_given?
+        yield(POXReference.new(self, id)) if block_given?
         obj._referenced_object_ids.each do |id|
           unless @db.is_marked?(id)
             @db.mark(id)
@@ -355,6 +350,24 @@ module PEROBS
       @class_map.rename(rename_map)
     end
 
+    # Remove the object from the in-memory list. This is an internal method
+    # and should never be called from user code.
+    # @param id [Fixnum or Bignum] Object ID of object to remove from the list
+    def _collect(id, ignore_errors = false)
+      unless ignore_errors || @in_memory_objects.include?(id)
+        raise RuntimeError, "Object with id #{id} is currently not in memory"
+      end
+      @in_memory_objects.delete(id)
+    end
+
+    # This method returns a Hash with some statistics about this store.
+    def statistics
+      {
+        :in_memory_objects => @in_memory_objects.length,
+        :root_objects => 0 #@root_objects.length
+      }
+    end
+
     private
 
     # Mark phase of a mark-and-sweep garbage collector. It will mark all
@@ -368,8 +381,46 @@ module PEROBS
     # Sweep phase of a mark-and-sweep garbage collector. It will remove all
     # unmarked objects from the store.
     def sweep
-      @db.delete_unmarked_objects
+      cntr = @db.delete_unmarked_objects.length
       @cache.reset
+      cntr
+    end
+
+    # Check the object with the given start_id and all other objects that are
+    # somehow reachable from the start object.
+    # @param start_id [Fixnum or Bignum] ID of the top-level object to start
+    #        with
+    # @param repair [Boolean] Delete refernces to broken objects if true
+    # @return [Fixnum] The number of references to bad objects.
+    def check_object(start_id, repair)
+      errors = 0
+      @db.mark(start_id)
+      # The todo list holds a touple for each object that still needs to be
+      # checked. The first item is the referring object and the second is the
+      # ID of the object to check.
+      todo_list = [ [ nil, start_id ] ]
+
+      while !todo_list.empty?
+        # Get the next PEROBS object to check
+        ref_obj, id = todo_list.pop
+
+        if (obj = object_by_id(id)) && @db.check(id, repair)
+          # The object exists and is OK. Mark is as checked.
+          @db.mark(id)
+          # Now look at all other objects referenced by this object.
+          obj._referenced_object_ids.each do |refd_id|
+            # Push them onto the todo list unless they have been marked
+            # already.
+            todo_list << [ obj, refd_id ] unless @db.is_marked?(refd_id)
+          end
+        else
+          # Remove references to bad objects.
+          ref_obj._delete_reference_to_id(id) if ref_obj && repair
+          errors += 1
+        end
+      end
+
+      errors
     end
 
   end

data/lib/perobs/version.rb

@@ -1,4 +1,4 @@
 module PEROBS
   # The version number
-  VERSION = "2.0.1"
+  VERSION = "2.1.0"
 end

data/tasks/changelog.rake

@@ -155,7 +155,7 @@ task :changelog do
     def getReleaseVersions
       # Get list of release tags from Git repository
       releaseVersions = `git tag`.split("\n").map { |r| r.chomp }.
-        delete_if { |r| ! (/release-\d+\.\d+\.\d+/ =~ r) }.
+        delete_if { |r| ! (/v\d+\.\d+\.\d+/ =~ r) }.
         sort{ |a, b| compareTags(a, b) }
       releaseVersions << 'HEAD'
     end

data/test/Array_spec.rb

@@ -40,6 +40,10 @@ class PO < PEROBS::Object
     _set(:name, name)
   end
 
+  def get_self
+    self # Never do this in real user code!
+  end
+
 end
 
 describe PEROBS::Array do
@@ -205,4 +209,19 @@ describe PEROBS::Array do
     pcheck { expect(a).to eq([ 1, 2, 3, 4, 5, nil ]) }
   end
 
+  it 'should only provide POXReference objects' do
+    a = cpa([ @store.new(PO), @store.new(PO) ])
+    expect(a[0].respond_to?(:is_poxreference?)).to be true
+    a.each do |a|
+      expect(a.respond_to?(:is_poxreference?)).to be true
+    end
+  end
+
+  it 'should catch a leaked PEROBS::ObjectBase object' do
+    @store['a'] = a = @store.new(PEROBS::Array)
+    o = @store.new(PO)
+    a[0] = o.get_self
+    expect { @store.sync }.to raise_error(RuntimeError)
+  end
+
 end

data/test/Hash_spec.rb

@@ -41,6 +41,10 @@ class PO < PEROBS::Object
     @name = name
   end
 
+  def get_self
+    self # Never do this in real user code!
+  end
+
 end
 
 describe PEROBS::Hash do
@@ -154,4 +158,11 @@ describe PEROBS::Hash do
     pcheck { expect(h2).to eq(hb) }
   end
 
+  it 'should catch a leaked PEROBS::ObjectBase object' do
+    @store['a'] = a = @store.new(PEROBS::Hash)
+    o = @store.new(PO)
+    a['a'] = o.get_self
+    expect { @store.sync }.to raise_error(RuntimeError)
+  end
+
 end

data/test/Object_spec.rb

@@ -51,6 +51,10 @@ class O2 < PEROBS::Object
     @a3.a1
   end
 
+  def get_self
+    self # Never do this in real user code!
+  end
+
 end
 
 class O3 < PEROBS::Object
@@ -125,6 +129,19 @@ describe PEROBS::Store do
     expect(o2.a3_deref).to eq('a1')
   end
 
+  it 'should always return a POXReference for a PEROBS object' do
+    @store['o1'] = o1 = @store.new(O1)
+    o1.a1 = @store.new(O2)
+    expect(@store['o1'].respond_to?(:is_poxreference?)).to be true
+    expect(o1.a1.respond_to?(:is_poxreference?)).to be true
+  end
+
+  it 'should catch a leaked PEROBS::ObjectBase object' do
+    @store['a'] = a = @store.new(O1)
+    o = @store.new(O2)
+    expect { a.a1 = o.get_self }.to raise_error(ArgumentError)
+  end
+
   it 'should raise an error when no attributes are defined' do
     @store['o3'] = @store.new(O3)
     expect { @store.sync }.to raise_error(StandardError)

data/test/Store_spec.rb

@@ -1,6 +1,6 @@
 # encoding: UTF-8
 #
-# Copyright (c) 2015 by Chris Schlaeger <chris@taskjuggler.org>
+# Copyright (c) 2015, 2016 by Chris Schlaeger <chris@taskjuggler.org>
 #
 # MIT License
 #
@@ -218,19 +218,23 @@ describe PEROBS::Store do
     p1.related = p2
     p2.related = p1
     p0.related = p1
-    @store.sync
-    @store.gc
+    expect(@store.check).to eq(0)
+    expect(@store.gc).to eq(0)
+    p0 = p1 = p2 = nil
+    GC.start
     @store = PEROBS::Store.new(@db_file)
     expect(@store['person0']._id).to eq(id0)
     expect(@store['person0'].related._id).to eq(id1)
     expect(@store['person0'].related.related._id).to eq(id2)
 
     @store['person0'].related = nil
-    @store.gc
+    expect(@store.gc).to eq(2)
+    GC.start
     expect(@store.object_by_id(id1)).to be_nil
     expect(@store.object_by_id(id2)).to be_nil
 
     @store = PEROBS::Store.new(@db_file)
+    expect(@store.check).to eq(0)
     expect(@store.object_by_id(id1)).to be_nil
     expect(@store.object_by_id(id2)).to be_nil
   end
@@ -392,6 +396,17 @@ describe PEROBS::Store do
     expect(@store['person2']).to be_nil
   end
 
+  it 'should track in-memory objects properly' do
+    @store = PEROBS::Store.new(@db_file)
+    @store['person'] = @store.new(Person)
+    # We have the root hash and the Person object.
+    expect(@store.statistics[:in_memory_objects]).to eq(2)
+    @store.sync
+    GC.start
+    # Now the Person should be gone from memory.
+    expect(@store.statistics[:in_memory_objects]).to eq(1)
+  end
+
   it 'should survive a real world usage test' do
     options = { :engine => PEROBS::BTreeDB, :dir_bits => 4 }
     @store = PEROBS::Store.new(@db_file, options)
@@ -437,7 +452,7 @@ describe PEROBS::Store do
       when 6
         if rand(50) == 0
           @store.sync
-          @store.check(false)
+          expect(@store.check(false)).to eq(0)
         end
       when 7
         index = rand(i)

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: perobs
 version: !ruby/object:Gem::Version
-  version: 2.0.1
+  version: 2.1.0
 platform: ruby
 authors:
 - Chris Schlaeger
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2016-01-01 00:00:00.000000000 Z
+date: 2016-01-09 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler