dbd 0.0.1 → 0.0.2

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 6c1d692a3ce886178a32b2dd624c2a69b4f380ed
+  data.tar.gz: 2ec48ccfd39bb4bab04e626eabc002b0b7c18105
+SHA512:
+  metadata.gz: 1fc7a48c2f7243febf3d8f8e34a6acb658dfc321ee81bae513fbed3635c9a07289f2b3a93a6f2f2c8fa745e367b6581f2afc965d96a3613b901dcb46096c13d9
+  data.tar.gz: 0130fe663c07bf96352c36ab6e90a6d2bae8cd0731625c71ff2bb49f96c9870ec5daf9dea680aac4946f9044669e0a792e1f452955a49e4f4dd31ef6aed74d3e

data/HISTORY.txt

@@ -0,0 +1,13 @@
+0.0.1 (12 May 2013)
+=====
+
+* Initial release showing the basic Fact format
+* a description with features and rationale
+
+0.0.2 (22 May 2013)
+=====
+
+* Better TimeStamp management (better randomness)
+* Adding a (Provenance)Resource directly to a Graph with <<
+* Simplification and cleaner implementations
+* Adding Fact to a Resource now sets (provenance_)subject

data/README.md

@@ -17,7 +17,7 @@ This is facts based data store, inspired by [RDF] concepts, but adding a log bas
   * 1 data source has _all_ my data : never more loose stuff :-)
   * facts can be invalidated (and replaced) later on
 * Privacy
-  * a "hard delete" is possible: all downstream readers of the fact stream
+  * a "hard delete" is possible: all downstream readers of the fact stream  
     must remove this fact and replace the back-up
   * since one single back-up file suffices, replacing the *single* back-up  
     file will actually remove the hard deleted fact(s) for good
@@ -74,23 +74,22 @@ Open Source [MIT]
 
     graph = Dbd::Graph.new
 
-    provenance.each {|provenance_fact| graph << provenance_fact}
-    nobel_peace_2012.each {|fact| graph << fact}
+    graph << provenance
+    graph << nobel_peace_2012
 
     puts graph.to_CSV
 
 results in
 
-
     $ ruby test.rb
-    "611dbc31-6961-4a86-9259-4a2700add783","2013-05-12 21:50:19 UTC","","98b7bb17-9921-4d52-a08a-39667c2abb4c","provenance:context","public"
-    "79e9c0e7-b6fd-4735-817c-8c21c97c9575","2013-05-12 21:50:19 UTC","","98b7bb17-9921-4d52-a08a-39667c2abb4c","dcterms:creator","@peter_v"
-    "7d143a50-8a63-4637-8ab8-c2aa7fc6e12e","2013-05-12 21:50:19 UTC","","98b7bb17-9921-4d52-a08a-39667c2abb4c","provenance:created_at","2013-05-12 21:50:19 UTC"
-    "fd121b00-7934-4e22-81c8-8e810760c686","2013-05-12 21:50:19 UTC","98b7bb17-9921-4d52-a08a-39667c2abb4c","477a2e10-5e34-434d-8fc1-969277f61f9f","base:nobelPeacePriceWinner","2012"
-    "2d852fe1-911f-497d-9485-6c24a6000fbb","2013-05-12 21:50:19 UTC","98b7bb17-9921-4d52-a08a-39667c2abb4c","477a2e10-5e34-434d-8fc1-969277f61f9f","rdfs:label","EU"
-    "ab00b092-65a3-47c0-b10b-837cb0a5ad81","2013-05-12 21:50:19 UTC","98b7bb17-9921-4d52-a08a-39667c2abb4c","477a2e10-5e34-434d-8fc1-969277f61f9f","rdfs:comment","European Union"
-    "a8d6b34b-6e02-4a5e-8529-4785f090866a","2013-05-12 21:50:19 UTC","98b7bb17-9921-4d52-a08a-39667c2abb4c","477a2e10-5e34-434d-8fc1-969277f61f9f","base:story","A long period of peace,
-     that is a ""bliss""."
+    "aaf11676-d016-4e74-a502-2db042ea8c67","2013-05-22 21:30:08.830374243 UTC","","3fe37986-0c00-45fb-a574-ed2d0374b3fc","provenance:context","public"
+    "1fd25f59-b838-4872-a290-4857e783a12c","2013-05-22 21:30:08.830416859 UTC","","3fe37986-0c00-45fb-a574-ed2d0374b3fc","dcterms:creator","@peter_v"
+    "f118e66a-aa96-4523-ae47-4f9ceff11916","2013-05-22 21:30:08.830434360 UTC","","3fe37986-0c00-45fb-a574-ed2d0374b3fc","provenance:created_at","2013-05-22 21:30:08 UTC"
+    "c2d29b70-7135-4434-829b-f0640475aeb5","2013-05-22 21:30:08.830450090 UTC","3fe37986-0c00-45fb-a574-ed2d0374b3fc","f628f608-27c3-4eb6-a687-cb121f793a4d","base:nobelPeacePriceWinner","2012"
+    "9c8048a5-b9e3-459c-9e82-6ae195ce22e6","2013-05-22 21:30:08.830465012 UTC","3fe37986-0c00-45fb-a574-ed2d0374b3fc","f628f608-27c3-4eb6-a687-cb121f793a4d","rdfs:label","EU"
+    "5e06472d-2146-4933-a31c-90873fa9ed26","2013-05-22 21:30:08.830478065 UTC","3fe37986-0c00-45fb-a574-ed2d0374b3fc","f628f608-27c3-4eb6-a687-cb121f793a4d","rdfs:comment","European Union"
+    "d984e5c3-8acd-4c50-b40f-4cacf9f8f5c7","2013-05-22 21:30:08.830489061 UTC","3fe37986-0c00-45fb-a574-ed2d0374b3fc","f628f608-27c3-4eb6-a687-cb121f793a4d","base:story","A long period of peace,
+ that is a ""bliss""."
 
 [RDF]:              http://www.w3.org/RDF/
 [Rationale]:        http://github.com/petervandenabeele/dbd/blob/master/docs/rationale.md

data/dbd.gemspec

@@ -26,5 +26,5 @@ Gem::Specification.new do |spec|
   spec.add_development_dependency 'yard'
   spec.add_runtime_dependency 'neography'
   spec.add_runtime_dependency 'rdf', '~> 1.0.6'
-  spec.add_runtime_dependency 'ruby_peter_v'
+  spec.add_runtime_dependency 'ruby_peter_v', '>= 0.0.4'
 end

data/docs/stories/009_build_and_store_resources_with_provenance.txt

@@ -33,6 +33,9 @@ I can build and store a group of resources with provenance
 
 * change arguments for (Provenance)Fact to an options hash
 
-* add a store method on Graph
-  * that will store a (Provenance)Resource instance
-  * this will now set the time_stamps (enforcing the strictly monotic order)
+* add a << method on Graph
+  * that stores a fact
+  * this will now set the time_stamp (enforcing the strictly monotic order)
+  * the time_stamp is a set_once property
+    (a soft form of immutable behavior that does not require the creation of
+     new objects (garbage collection cost).

data/docs/stories/010_time_class.txt

@@ -0,0 +1,23 @@
+010_time_class
+
+As a client
+I can add facts to a fact stream and the time_stamp is set
+
+* change valid? to errors
+  * returns array of error messages (errors.empty? indicates valid?)
+
+* add a performance bench mark to the test set (this will also
+  act as a collision test for the randomization approaches).
+
+* upgrade ruby 1.9.3 and 2.0.0
+
+* use a TimeStamp class
+
+* the to_s of this class shows the time_stamp to ns precision
+
+* the TimeStamp.new accepts a larger_than: option that
+  works out the offsetting for being strictly larger than the
+  newest_time_stamp in a Fact collection in a graph
+
+* the time_stamp class adds some random offset time
+  (e.g. between 1 and 999 ns) to a new time_stamp

data/docs/stories/011_store_resource_in_graph.txt

@@ -0,0 +1,10 @@
+011_store_resource_in_graph.txt
+
+* allow setting the subject and provenance_subject of a fact with set_once
+  (a soft form of immutable behavior that does not require the creation of
+   new objects (garbage collection cost) and may clean up the way too complex
+   check_or_set_subject_and_provenance behavior in Resource)
+
+* add a << method for a resource on Graph
+  * that will store a (Provenance)Resource instance
+  * this will now set the time_stamps (enforcing the strictly monotic order)

data/docs/stories/{010_provenance_fact_properties_from_provenance_ontology.txt → 012_provenance_fact_properties_from_provenance_ontology.txt}

@@ -1,4 +1,4 @@
-010_provenance_fact_predicates_from_provenance_ontology
+011_provenance_fact_predicates_from_provenance_ontology
 
 As a client
 I can read the predicates for a provenance_fact from provenance ontology

data/docs/test.rb

@@ -26,7 +26,17 @@ nobel_peace_2012 << fact_EU_story
 
 graph = Dbd::Graph.new
 
-provenance.each {|provenance_fact| graph << provenance_fact}
-nobel_peace_2012.each {|fact| graph << fact}
+graph << provenance
+graph << nobel_peace_2012
 
 puts graph.to_CSV
+
+# "9f868d99-af27-4d83-86ae-ea5f4a1fa654","2013-05-22 21:25:48.136527770 UTC","","a6e028dd-a340-49ce-b3f8-2f158e257a87","provenance:context","public"
+# "28496b40-1891-4bd0-9ee1-0c6c2a878cc1","2013-05-22 21:25:48.136596276 UTC","","a6e028dd-a340-49ce-b3f8-2f158e257a87","dcterms:creator","@peter_v"
+# "98b9dd72-3473-4500-814d-d955eec2c5ee","2013-05-22 21:25:48.136621174 UTC","","a6e028dd-a340-49ce-b3f8-2f158e257a87","provenance:created_at","2013-05-22 21:25:40 UTC"
+# "a0482b46-414d-40df-b436-41142728fda6","2013-05-22 21:25:55.367834295 UTC","a6e028dd-a340-49ce-b3f8-2f158e257a87","cd66aece-0b21-4e3e-8286-4191efb3aea1","base:nobelPeacePriceWinner","2012"
+# "c1af381d-800f-4279-a2cc-ccccf31f5134","2013-05-22 21:25:55.367891996 UTC","a6e028dd-a340-49ce-b3f8-2f158e257a87","cd66aece-0b21-4e3e-8286-4191efb3aea1","rdfs:label","EU"
+# "ac08843a-baae-49ea-a725-81b7e199e8f9","2013-05-22 21:25:55.367910018 UTC","a6e028dd-a340-49ce-b3f8-2f158e257a87","cd66aece-0b21-4e3e-8286-4191efb3aea1","rdfs:comment","European Union"
+# "6e91fa40-daa8-45d1-916e-f9b243d01f2c","2013-05-22 21:25:55.367928936 UTC","a6e028dd-a340-49ce-b3f8-2f158e257a87","cd66aece-0b21-4e3e-8286-4191efb3aea1","base:story","A long period of peace,
+# that is a ""bliss""."
+

data/lib/dbd.rb

@@ -3,6 +3,7 @@ require 'ruby_peter_v'
 require 'dbd/version'
 
 require 'dbd/errors'
+require 'dbd/time_stamp'
 require 'dbd/fact'
 require 'dbd/provenance_fact'
 require 'dbd/resource'

data/lib/dbd/fact.rb

@@ -7,7 +7,7 @@ module Dbd
   ##
   # Basic Fact of knowledge.
   #
-  # The database is built as an ordered sequence of facts, the "fact stream".
+  # The database is built as an ordered sequence of facts, a "fact stream".
   #
   # This is somewhat similar to a "triple" in the RDF (Resource Description
   # Framework) concept, but with different and extended functionality.
@@ -15,22 +15,22 @@ module Dbd
   # Each basic fact has:
   # * a unique and invariant *id* (a uuid)
   #
-  #   To allow referencing back to it (e.g. to invalidate it later in the fact stream).
+  #   To allow referencing back to it (e.g. to invalidate it later in a fact stream).
   #
   # * a *time_stamp*  (time with nanosecond granularity)
   #
-  #   To allow verifying that the order in the fact stream is correct.
+  #   To allow verifying that the order in a fact stream is correct.
   #
   #   A time_stamp does not need to represent the exact time of the
   #   creation of the fact, but it has to increase in strictly monotic
-  #   order in the fact stream.
+  #   order in a fact stream.
   #
   # * a *provenance_subject* (a uuid)
   #
   #   The subject of the ProvenanceResource (a set of ProvenanceFacts with
   #   the same subject) about this fact. Each Fact, points *back* to a
   #   ProvenanceResource (the ProvenanceResource must have been fully
-  #   defined, earlier in the fact stream).
+  #   defined, earlier in a fact stream).
   #
   # * a *subject* (a uuid)
   #
@@ -72,9 +72,35 @@ module Dbd
       attr_reader attribute
     end
 
+    ##
+    # A set_once setter for time_stamp.
+    #
+    # This implements a "form" of immutable behavior. The value can
+    # be set once (possibly after creation the object), but can
+    # never be changed after that.
     def time_stamp=(time_stamp)
-      raise RuntimeError if @time_stamp
-      @time_stamp = time_stamp
+      raise ArgumentError unless time_stamp.is_a?(TimeStamp)
+      set_once(:time_stamp, time_stamp)
+    end
+
+    ##
+    # A set_once setter for subject.
+    #
+    # This implements a "form" of immutable behavior. The value can
+    # be set once (possibly after creation the object), but can
+    # never be changed after that.
+    def subject=(subject)
+      set_once(:subject, subject)
+    end
+
+    ##
+    # A set_once setter for provenance_subject.
+    #
+    # This implements a "form" of immutable behavior. The value can
+    # be set once (possibly after creation the object), but can
+    # never be changed after that.
+    def provenance_subject=(provenance_subject)
+      set_once(:provenance_subject, provenance_subject)
     end
 
     ##
@@ -128,15 +154,18 @@ module Dbd
     end
 
     ##
-    # Checks if a fact is valid for storing in the graph.
+    # Checks if a fact has errors for storing in the graph.
     #
-    # @return [#true?] not nil if valid
-    def valid?
-      # id not validated, is set automatically
-      # predicate not validated, is validated in initialize
-      # object not validated, is validated in initialize
-      provenance_subject_valid?(provenance_subject) &&
-      subject
+    # @return [Array] an Array of error messages
+    def errors
+      # * id not validated, is set automatically upon creation
+      # * time_stamp not validated, is set automatically later
+      # * predicate not validated, is validated in initialize
+      # * object not validated, is validated in initialize
+      [].tap do |a|
+        a << provenance_subject_error(provenance_subject)
+        a << "Subject is missing" unless subject
+      end.compact
     end
 
     ##
@@ -147,35 +176,9 @@ module Dbd
     # This is how the difference is encoded between Fact and
     # ProvenanceFact in the fact stream.
     # @param [#nil?] provenance_subject
-    # Return [Boolean]
-    def provenance_subject_valid?(provenance_subject)
-      provenance_subject
-    end
-
-    ##
-    # Builds duplicate with the subject set.
-    #
-    # @param [Subject] subject_arg
-    # @return [Fact] the duplicate fact
-    def dup_with_subject(subject_arg)
-      self.class.new(
-       provenance_subject: provenance_subject,
-       subject: subject_arg, # from arg
-       predicate: predicate,
-       object: object)
-    end
-
-    ##
-    # Builds duplicate with the provenance_subject set.
-    #
-    # @param [Subject] provenance_subject_arg
-    # @return [Fact] the duplicate fact
-    def dup_with_provenance_subject(provenance_subject_arg)
-      self.class.new(
-       provenance_subject: provenance_subject_arg, # from arg
-       subject: subject,
-       predicate: predicate,
-       object: object)
+    # Return [nil, String] nil or an error message
+    def provenance_subject_error(provenance_subject)
+      "Provenance subject is missing" unless provenance_subject
     end
 
   end

data/lib/dbd/fact/collection.rb

@@ -29,7 +29,7 @@ module Dbd
       #
       # @return [self] for chaining
       #
-      # Validates that added fact is valid.
+      # Validates that added fact is valid (has no errors).
       #
       # Validates that added fact is newer.
       #
@@ -41,8 +41,7 @@ module Dbd
       #
       # Mark the fact in the list of used provenance_subjects (for [A]).
       def <<(fact)
-        # TODO Add a more descriptive Exception message
-        raise FactError unless fact.valid?
+        raise FactError, "#{fact.errors.join(', ')}." unless fact.errors.empty?
         raise OutOfOrderError if (self.newest_time_stamp && fact.time_stamp <= self.newest_time_stamp)
         raise OutOfOrderError if (@used_provenance_subjects[fact.subject])
         index = Helpers::OrderedSetCollection.add_and_return_index(fact, @internal_collection)

data/lib/dbd/graph.rb

@@ -9,9 +9,15 @@ module Dbd
 
     include Fact::Collection
 
-    def <<(fact)
-      enforce_strictly_monotonic_time(fact)
-      super(fact)
+    ##
+    # Add a Fact or Resource to the Graph.
+    #
+    # This will add a time_stamp to the Facts.
+    def <<(fact_or_resource)
+      Array(fact_or_resource).each do |fact|
+        enforce_strictly_monotonic_time(fact)
+        super(fact)
+      end
     end
 
     ##
@@ -29,18 +35,11 @@ module Dbd
   private
 
     ##
-    # The system mmust enforce that the time_stamps are strictly monotonic.
-    #
-    # This has been detected because on Java (JRuby) the the Wall time has
-    # a resolution of only 1 ms so sometimes, the exact same value for
-    # Time.now was reported.
+    # Setting a strictly monotonically increasing time_stamp (if not yet set).
+    # The time_stamp also has some randomness (1 .. 999 ns) to reduce the
+    # chance on collisions when merging fact streams from different sources.
     def enforce_strictly_monotonic_time(fact)
-      new_time = Time.now.utc
-      newest_time_stamp = newest_time_stamp()
-      if newest_time_stamp && new_time <= newest_time_stamp
-        new_time = newest_time_stamp + 0.000_000_002 # Add approx. 2 nanoseconds
-      end
-      fact.time_stamp = new_time
+      fact.time_stamp = TimeStamp.new(larger_than: newest_time_stamp) unless fact.time_stamp
     end
 
   end

data/lib/dbd/provenance_fact.rb

@@ -12,7 +12,7 @@ module Dbd
   # usage of provenance_subject is not recursive on this level (this
   # allows efficient single pass loading in an underlying database).
   #
-  # In the serialisation of the fact stream, the presence or absence of a
+  # In the serialisation of a fact stream, the presence or absence of a
   # provenance_subject marks the difference between a (base) Fact and a
   # ProvenanceFact.
   #
@@ -47,9 +47,9 @@ module Dbd
     #
     # Here, in the derived ProvenanceFact, it must not be present.
     # @param [#nil?] provenance_subject
-    # Return [Boolean]
-    def provenance_subject_valid?(provenance_subject)
-      provenance_subject.nil?
+    # Return [nil, String] nil if valid, an error message if not
+    def provenance_subject_error(provenance_subject)
+      "Provenance subject should not be present in Provenance Fact" if provenance_subject
     end
 
     ##

data/lib/dbd/provenance_resource.rb

@@ -44,10 +44,11 @@ module Dbd
     ##
     # Check provenance_subject, which should be nil here
     # @param [ProvenanceFact] provenance_fact
-    # @return [ProvenanceFact] with validated nil on provenance_subject
-    def check_or_set_provenance(provenance_fact)
-      raise ProvenanceError if provenance_fact.provenance_subject
-      provenance_fact
+    def set_provenance!(provenance_fact)
+      if provenance_fact.provenance_subject
+        raise ProvenanceError,
+          "trying to set provenance_subject to#{provenance_fact.provenance_subject}"
+      end
     end
 
   end

data/lib/dbd/resource.rb

@@ -19,14 +19,22 @@ module Dbd
   # Resources that are associated with it.
   #
   # During build-up of a Fact, the subject and the provenance_subject
-  # can be nil. These will then be set in a local duplicate when the
-  # Fact is added (with '<<') to a resource.
+  # can be nil. These will then be set when the Fact is added
+  # (with '<<') to a resource.
   class Resource
 
     include Helpers::OrderedSetCollection
 
     attr_reader :subject
 
+    ##
+    # Getter for provenance_subject.
+    #
+    # Will be overridden in the ProvenanceResource subclass.
+    def provenance_subject
+      @provenance_subject
+    end
+
     ##
     # @return [Fact::Subject] a new (random) Resource subject
     def self.new_subject
@@ -56,57 +64,28 @@ module Dbd
     ##
     # Add a fact.
     #
-    # * if it has no subject, the subject is set in a duplicate fact
+    # * if it has no subject, the subject is set (this modifies the fact !)
     # * if is has the same subject as the resource, added unchanged.
     # * if it has a different subject, a SubjectError is raised.
-    # * if it has no provenance_subject, the provenance_subject is set in a duplicate fact
+    # * if it has no provenance_subject, the provenance_subject is set (this modifies the fact !)
     # * if is has the same provenance_subject as the resource, added unchanged.
     # * if it has a different provenance_subject, a ProvenanceError is raised.
     def <<(fact)
       # TODO: check the type of the fact (Fact)
-      super(check_or_set_subject_and_provenance(fact))
-    end
-
-    ##
-    # Getter for provenance_subject.
-    #
-    # Will be overridden in the ProvenanceResource subclass.
-    def provenance_subject
-      @provenance_subject
+      set_subject!(fact)
+      set_provenance!(fact)
+      super(fact)
     end
 
   private
 
-    def check_or_set_subject_and_provenance(element)
-      with_subject = check_or_set_subject(element)
-      check_or_set_provenance(with_subject)
-    end
-
-    def check_or_set_subject(element)
-      if element.subject
-        if element.subject == @subject
-          return element
-        else
-          raise SubjectError,
-            "self.subject is #{subject} and element.subject is #{element.subject}"
-        end
-      else
-        element.dup_with_subject(@subject)
-      end
+    def set_subject!(fact)
+      fact.subject = subject
     end
 
     # this will be overriden in the ProvenanceResource sub_class
-    def check_or_set_provenance(element)
-      if element.provenance_subject
-        if element.provenance_subject == @provenance_subject
-          return element
-        else
-          raise ProvenanceError,
-            "self.provenance_subject is #{provenance_subject} and element.provenance_subject is #{element.provenance_subject}"
-        end
-      else
-        element.dup_with_provenance_subject(@provenance_subject)
-      end
+    def set_provenance!(fact)
+      fact.provenance_subject = provenance_subject
     end
 
     def validate_provenance_subject