omf_rete 0.5 → 0.6.1

data/README.md

@@ -1,5 +1,8 @@
 
-= Introduction
+# Introduction
+
+__Warning: This is embarrassingly out of date. Check tests for a more accurate 
+reflection of what's implemented__
 
 This library implements a tuple store with a query and subscribe mechanism. 
 A subscribe is effectively a standing query which executes a block whenever
@@ -10,65 +13,69 @@ The store holds same sized tuples with each value being assigned a name and
 type at creation to support varous convenience functions to create and retrieve
 tuples.
 
-The following code snippet creates a simple RDF store and adds a few triplets
+The following code snippet creates a simple RDF store (tuple_length: 3) and adds a  triplet
 to it.
 
-  store = OMF::Rete::Store.new(3)
-  store.add('myFridge', 'contains', 'milk')
+    eng = OMF::Rete.create_engine(tuple_length: 3)
+    eng.add_fact('myFridge', 'contains', 'milk')
 
-A filter consists of an array of tuple +patterns+ and a +block+ to be called when the store
+A rule consists of an array of tuple +patterns+ and a +block+ to be called when the store
 contains a set of tuples matching the +pattern+.
 
 The following filter only looks for a single, specific tuple. The supplied block is called
 immediately if the tuple already exists in the store, or when such a tuple would be added at a later
 stage.
 
-  store.subscribe(:report_problem, [
-    ['myFridge', 'status', 'broken']
-  ]) do |m|
-  	puts "My fridge is broken"
-  end
+    eng = OMF::Rete.create_engine(tuple_length: 3)
+    eng.add_rule(:report_problem, [
+      ['myFridge', 'status', 'broken']
+    ]) do |m|
+    	puts "My fridge is broken"
+    end
+    eng.add_fact('myFridge', 'status', 'ok')
+    eng.add_fact('myFridge', 'status', 'broken')    
   
 The following filter contains two +patterns+ and therefore both need to be matched at the same
 time in order for the block to fire. Note, that the order these tuples are added to the store
 or the interval between is irrelevant.
 
-  store.subscribe(:save_milk, [
-    ['myFridge', 'status', 'broken'],
-    ['myFridge', 'contains', 'milk'],
-  ]) do |m|
-  	puts "Save the milk from my fridge"
-  end
-  
+    eng.subscribe(:save_milk, [
+      [:fridge?, 'status', 'broken'],
+      [:fridge?, 'contains', 'milk'],
+    ]) do |m|
+      puts "Save the milk from #{m.fridge?}"
+    end
+    eng.add_fact('myFridge', 'status', 'broken')
   
-So far the filter pattern were fully specified. The <tt>:_</tt> symbol can be used as a wildcard identifier.
+So far the filter pattern were fully specified. The <tt>nil</tt> value can be used as a wildcard identifier.
 The following code snippet reports anything which is broken.
 
-  store.subscribe(:something_broken, [
-    [:_, 'status', 'broken']
-  ]) do |m|
-    puts "Something is broken"
-  end
+    eng.subscribe(:something_broken, [
+      [nil, 'status', 'broken']
+    ]) do |m|
+      puts "Something is broken"
+    end
+    eng.add_fact('myFridge', 'status', 'broken')
 
 _Not implemented yet_
 Similar to OMF::Rete::Store#addNamed we can describe a pattern with a hash. Any value not named is automatically 
 wildcarded. Therefore, an alternative represenation of the previous filter is as follows:
 
-  store.subscribe(:something_broken, [
-    {:pred => 'status', :obj => 'broken'}
-  ]) do |m|
-    puts "Something is broken"
-  end
+    store.subscribe(:something_broken, [
+      {:pred => 'status', :obj => 'broken'}
+    ]) do |m|
+      puts "Something is broken"
+    end
   
 The +match+ argument to the block holds the context of the match and specifically, the tuples involved 
 in the match.
 
-  store.subscribe(:something_broken, [
-    [:_, 'status', 'broken']
-  ]) do |match|
-    what = match.tuples[0][:subject]
-    puts "#{what} is broken"
-  end
+    store.subscribe(:something_broken, [
+      [:_, 'status', 'broken']
+    ]) do |match|
+      what = match.tuples[0][:subject]
+      puts "#{what} is broken"
+    end
   
 <tt>match.tuples</tt> returns an area of tuples one for each pattern. The matched tuple for the first pattern is at index 0,
 the second one at index 1, and so on. Individual values of a tuple can be retrieved through the initially declared 
@@ -78,18 +85,19 @@ Let us assume we are monitoring many fridges, so if we want to report broken one
 that the +subject+ in both patterns in our second example are identical. Or in more technical terms, we need to +bind+ or +join+
 values across patterns. A binding variable is identified by a symbol with a trailing <b>?</b>.
 
-  store.subscribe(:save_milk, [
-    [:fridge?, 'status', 'broken'],
-    [:fridge?, 'contains', 'milk'],
-  ]) do |match|
-    fridge = match[:fridge]
-    puts "Save the milk from #{fridge}"
-  end
+    store.subscribe(:save_milk, [
+      [:fridge?, 'status', 'broken'],
+      [:fridge?, 'contains', 'milk'],
+    ]) do |match|
+      fridge = match[:fridge]
+      puts "Save the milk from #{fridge}"
+    end
 
 <tt>match[bindingName]</tt> (without the '?') returns the value bound to <tt>:fridge?</tt> for this match. 
 Obviously <tt>match.tuples[0][:subject]</tt> will return the same value.
 
-== Functions
+## Functions
+
 
 Pattern matches alone are not always sufficient. For instance, let us assume that we have also stored the age in years
 of each monitored fridge and want to replace each broken one which is older than 10 years. To describe such a filter
@@ -99,38 +107,38 @@ Functions are identified by the <tt>:PROC</tt> symbol in the first position of a
 name, and the list of parameters. Effectively, a function filters the values previosuly bound to a variable to those
 for which the function returns true.
 
-  store.subscribe(:replace_old_ones, [
-    [:fridge?, 'status', 'broken'],
-    [:fridge?, 'age', :age?],
-    [:PROC, :greater, :age?, 10]
-  ]) do |match|
-    puts "Replace #{match[:fridge]}"
-  end
+    store.subscribe(:replace_old_ones, [
+      [:fridge?, 'status', 'broken'],
+      [:fridge?, 'age', :age?],
+      [:PROC, :greater, :age?, 10]
+    ]) do |match|
+      puts "Replace #{match[:fridge]}"
+    end
 
 <b>Design Note:</b> A more generic solution based on a 'lambda' is most likely cleaner. This is effectively
 identical to the final block, except that the block should return +true+ for tuples passing the filter,
 and +false+ for all others. To further simplify this and also reduce the search space, we can define a
 +filter+ function which takes a list of bound variables and calls the associated block with specific bindings.
 
-  store.subscribe(:replace_old_ones, [
-    [:fridge?, 'status', 'broken'],
-    [:fridge?, 'age', :age?],
-    filter(:age?) { |age| age > 10 }
-  ]) do |match|
-    puts "Replace #{match[:fridge]}"
-  end
+    store.subscribe(:replace_old_ones, [
+      [:fridge?, 'status', 'broken'],
+      [:fridge?, 'age', :age?],
+      filter(:age?) { |age| age > 10 }
+    ]) do |match|
+      puts "Replace #{match[:fridge]}"
+    end
   
-== Set Operators
+### Set Operators
 
 Let us assume we want the store to not only reflect the current facts but the entire history of a system. We
 can achieve that by adding a timestamp to each fact and never retract facts.
 
-  store = OMF::Rete::Store.new(:subj => String, :pred => String, :obj => Object, :tstamp => Time)
+    store = OMF::Rete::Store.new(:subj => String, :pred => String, :obj => Object, :tstamp => Time)
   
 This now allows us to capture that a fridge broke on a specific date and was fixed some times later.
 
-  store.add('myFridge', 'status', 'broken', '2008-12-20')
-  store.add('myFridge', 'status', 'ok', '2008-12-22')
+    store.add('myFridge', 'status', 'broken', '2008-12-20')
+    store.add('myFridge', 'status', 'ok', '2008-12-22')
   
 However, how can we now determine that a specific fridge is CURRENTLY broken? The pattern
 <tt>[:f?, 'status' 'broken']</tt> will identify all fridges which are currently broken, as well as those
@@ -141,21 +149,21 @@ the filter picks the one with the most recent timestamp.
 The current syntax achieves this through special match values. For instance, <tt>:LATEST</tt> for <tt>Time</tt>
 types picks the most recent fact.
   
-  [:fridge?, 'status', :_, :LATEST]
+    [:fridge?, 'status', :_, :LATEST]
   
 To find all currently broken fridges we need to bind this to all broken status facts.
 
-  store.subscribe(:broken_lately, [
-    [:fridge?, 'status', :_, :LATEST],
-    [:fridge?, 'status', 'broken']
-  ]) do |match|
-    puts "#{match[:fridge]} is broken"
-  end
+    store.subscribe(:broken_lately, [
+      [:fridge?, 'status', :_, :LATEST],
+      [:fridge?, 'status', 'broken']
+    ]) do |match|
+      puts "#{match[:fridge]} is broken"
+    end
   
 <b>Design Note:</b> This seems to be a fairly ad-hoc syntax. Is there a better one? This assumes that there is no join 
 on any of the bound variables, they are simply keys for the sets. But overloading functionality always adds complexity.
 
-== Negated Conditions
+## Negated Conditions
 
 Now let us consider we know that our fridge is broken and we want to monitor any future status updates. 
 There may be many different status types and we are interested in all of them as long as they are
@@ -163,12 +171,12 @@ different to 'broken'. In other words, we need a way to describe what is refered
 condition' and is defined by a leading <tt>:NOT</tt>, followed by one or multiple patterns describing
 what should NOT be in the store.
 
-  store.subscribe(:find_latest, [
-    ['My Fridge', :status, :_, :LATEST],
-    [:NOT, ['My Fridge', 'status', 'broken']]
-  ]) do |match|
-    puts "Status for my fridge changed to '#{match.tuples[0][:obj]}."
-  end
+    store.subscribe(:find_latest, [
+      ['My Fridge', :status, :_, :LATEST],
+      [:NOT, ['My Fridge', 'status', 'broken']]
+    ]) do |match|
+      puts "Status for my fridge changed to '#{match.tuples[0][:obj]}."
+    end
   
 Please note that the above example fails to report when my fridge is reported as broken again.
 

data/lib/omf_rete/abstract_tuple_set.rb

@@ -27,7 +27,11 @@ module OMF::Rete
     def addTuple(tuple)
       raise 'Abstract class'
     end
-    
+ 
+    def removeTuple(tuple)
+      raise 'Abstract class'
+    end
+   
     # Call block for every tuple stored in this set currently and
     # in the future. In other words, the block may be called even after this
     # method returns. 

data/lib/omf_rete/indexed_tuple_set.rb

@@ -8,25 +8,25 @@ module OMF::Rete
   # removed.
   #
   # The IndexedTupleSet is defined by a +description+ and an
-  # +indexPattern+. 
+  # +indexPattern+.
   #
   # The +description+ is an array of the
   # same length as the tuples maintained. Each element,
   # if not nil, names the binding variable associated with it.
-  # The position of a binding can be retrieved with 
+  # The position of a binding can be retrieved with
   # +index_for_binding+.
   #
   # The +indexPattern+ describes which elements of the inserted
-  # tuple are being combined in an array to form the index 
-  # key for each internal tuple. The elements in the +indexPattern+ 
+  # tuple are being combined in an array to form the index
+  # key for each internal tuple. The elements in the +indexPattern+
   # are described by the binding name.
   #
   #
   class IndexedTupleSet < AbstractTupleSet
-    
+
     attr_reader :indexPattern
     attr_writer :transient # if true only process tuple but don't store it
-    
+
     def initialize(description, indexPattern, source = nil, opts = {})
       super description, source
       if (indexPattern.length == 0)
@@ -39,7 +39,7 @@ module OMF::Rete
 
       @index = {}
     end
-    
+
     def addTuple(tuple)
       key = @indexMap.collect do |ii|
         tuple[ii]
@@ -58,12 +58,38 @@ module OMF::Rete
       end
       tuple # return added tuple
     end
-    
+
+    def removeTuple(tuple)
+      key = @indexMap.collect do |ii|
+        tuple[ii]
+      end
+
+      if @transient
+        @onRemoveBlockWithIndex.call(key, tuple) if @onRemoveBlockWithIndex
+        @onRemoveBlock.call(tuple) if @onRemoveBlock
+      else
+        vset = @index[key]
+        if vset
+          vset.delete(tuple)
+          @onRemoveBlockWithIndex.call(key, tuple) if @onRemoveBlockWithIndex
+          @onRemoveBlock.call(tuple) if @onRemoveBlock
+        end
+      end
+      tuple # return removed tuple
+    end
+
+    # Clear index
+    def clear()
+      @onRemoveBlockWithIndex.call(nil, nil) if @onRemoveBlockWithIndex
+      @onRemoveBlock.call(nil) if @onRemoveBlock
+      @index = {}
+    end
+
     # Call block for every tuple stored in this set currently and
     # in the future. In other words, the block may be called even after this
-    # method returns. 
+    # method returns.
     #
-    # The block will be called with one parameters, the 
+    # The block will be called with one parameters, the
     # tuple added.
     #
     # Note: Only one +block+ can be registered at a time
@@ -80,9 +106,9 @@ module OMF::Rete
 
     # Call block for every tuple stored in this set currently and
     # in the future. In other words, the block may be called even after this
-    # method returns. 
+    # method returns.
     #
-    # The block will be called with two parameters, the index of the tuple followed by the 
+    # The block will be called with two parameters, the index of the tuple followed by the
     # tuple itself.
     #
     # Note: Only one +block+ can be registered at a time
@@ -96,14 +122,44 @@ module OMF::Rete
       @onAddBlockWithIndex = block
     end
 
-    # Return the set of tuples index by +key+. 
+    # Call block for every tuple removed from this set in the future.
+    # In other words, the block may be called after this
+    # method returns.
+    #
+    # The block will be called with one parameters, the
+    # tuple removed. If the parameter is nil, everything has
+    # been removed (cleared)
+    #
+    # Note: Only one +block+ can be registered at a time
+    #
+    def on_remove(&block)
+      @onRemoveBlock = block
+    end
+
+
+    # Call block for every tuple removed from this set
+    # in the future. In other words, the block may be called even after this
+    # method returns.
+    #
+    # The block will be called with two parameters, the index of the tuple followed by the
+    # tuple itself. If both parameters are nil, everything has
+    # been removed (cleared)
+    #
+    # Note: Only one +block+ can be registered at a time
+    #
+    def on_remove_with_index(&block)
+      @onRemoveBlockWithIndex = block
+    end
+
+
+    # Return the set of tuples index by +key+.
     # Will return nil if nothing is stored for +key+
     #
     def [](key)
       res = @index[key]
       res
     end
-    
+
     # Return all stored tuples in an array.
     def to_a
       a = []
@@ -114,16 +170,27 @@ module OMF::Rete
       end
       a
     end
-    
+
+    # Return all stored tuples in a set.
+    def to_set
+      a = Set.new
+      @index.each_value do |s|
+        s.each do |t|
+          a << t
+        end
+      end
+      a
+    end
+
     def describe(out = STDOUT, offset = 0, incr = 2, sep = "\n")
       out.write(" " * offset)
       desc = @description.collect do |e| e || '*' end
       out.write("ts: [#{desc.join(', ')}]")
       ind = @indexMap.collect do |i| @description[i] end
       out.write("  (index: [#{ind.sort.join(', ')}])#{sep}")
-      @source.describe(out, offset + incr, incr, sep) if @source 
+      @source.describe(out, offset + incr, incr, sep) if @source
     end
-    
+
   end # class IndexedTupleSet
 end # module
 

data/lib/omf_rete/join_op.rb

@@ -2,27 +2,27 @@ require 'omf_rete/indexed_tuple_set'
 
 module OMF::Rete
 
-  # This class implements the join operation between two 
+  # This class implements the join operation between two
   # +IndexedTupleSets+ feeding into a third, result tuple set.
   # The size of both incoming tuple sets needs to be identical and they
   # are supposed to be indexed on the same list of variables as this is
   # what they wil be joined at.
-  # 
-  # Implementation Note: We first calculate a +combinePattern+ 
+  #
+  # Implementation Note: We first calculate a +combinePattern+
   # from the +description+ of the result set.
   # The +combinePattern+ describes how to create a joined tuple to insert
-  # into the result tuple set. The +combinePattern+ is an array of 
-  # the same size as the result tuple. Each element is a 2-array 
+  # into the result tuple set. The +combinePattern+ is an array of
+  # the same size as the result tuple. Each element is a 2-array
   # with the first element describing the input set (0 .. left, 1 .. right)
   # and the second one the index from which to take the value.
-  # 
+  #
   #
   class JoinOP
     def initialize(leftSet, rightSet, resultSet)
       @resultSet = resultSet
       @left = leftSet
       @right = rightSet
-      
+
       @combinePattern = resultSet.description.collect do |bname|
         side = 0
         unless (i = leftSet.index_for_binding(bname))
@@ -35,7 +35,8 @@ module OMF::Rete
         [side, i]
       end
       @resultLength = @combinePattern.length
-      
+      @results = {} # Keep track of how may input tuples create the same result - necessary for 'removeTuple'
+
       leftSet.on_add_with_index do |index, ltuple|
         if (rs = rightSet[index])
           rs.each do |rtuple|
@@ -50,7 +51,30 @@ module OMF::Rete
           end
         end
       end
-      
+
+      leftSet.on_remove_with_index do |index, ltuple|
+        if (index.nil?)
+          clear_result()
+        else
+          if (rs = rightSet[index])
+            rs.each do |rtuple|
+              remove_result(ltuple, rtuple)
+            end
+          end
+        end
+      end
+      rightSet.on_remove_with_index do |index, rtuple|
+        if (index.nil?)
+          clear_result()
+        else
+          if (ls = leftSet[index])
+            ls.each do |ltuple|
+              remove_result(ltuple, rtuple)
+            end
+          end
+        end
+      end
+
       # Supporting 'check_for_tuple'
       @left_pattern = @left.description.map do |bname|
         @resultSet.index_for_binding(bname)
@@ -60,7 +84,7 @@ module OMF::Rete
       end
 
     end
-    
+
     # Check if +tuple+ can be produced by this join op. We first
     # check if we can find a match on one side and then request
     # from the other side all the tuples which would lead to full
@@ -76,19 +100,19 @@ module OMF::Rete
       end
       return false
     end
-    
+
     def description()
       @resultSet.description
     end
-    
+
     def describe(out = STDOUT, offset = 0, incr = 2, sep = "\n")
       out.write(" " * offset)
       result = @combinePattern.collect do |side, index|
         (side == 0) ? @left.binding_at(index) : @right.binding_at(index)
       end
       out.write("join: [#{@left.indexPattern.sort.join(', ')}] => [#{result.sort.join(', ')}]#{sep}")
-      @left.describe(out, offset + incr, incr, sep) 
-      @right.describe(out, offset + incr, incr, sep)         
+      @left.describe(out, offset + incr, incr, sep)
+      @right.describe(out, offset + incr, incr, sep)
     end
 
     private
@@ -104,10 +128,46 @@ module OMF::Rete
         result[i] = t[index]
         i += 1
       end
-      @resultSet.addTuple(result)
+
+      if @results.key? result
+        @results[result] = @results[result] + 1
+      else
+        @results[result] = 1
+        @resultSet.addTuple(result)
+      end
+      #puts "add: #{result.inspect} (#{@results.inspect})"
     end
-    
-    
-    
+
+    def remove_result(ltuple, rtuple)
+      unless @resultLength
+        i = 2
+      end
+      result = Array.new(@resultLength)
+      i = 0
+      @combinePattern.each do |setId, index|
+        t = setId == 0 ? ltuple : rtuple
+        result[i] = t[index]
+        i += 1
+      end
+      # Remove it from result set if it has only been added once
+      #puts "remove: #{result.inspect} (#{@results.inspect})"
+      if count = @results[result]
+        if count == 1
+          @results.delete(result)
+          @resultSet.removeTuple(result)
+        else
+          @results[result] = count - 1
+        end
+      else
+        raise "Should never happen"
+      end
+    end
+
+    def clear_result()
+      @results = {}
+      @resultSet.clear
+    end
+
+
   end # class
 end # module

data/lib/omf_rete/planner/join_plan.rb

@@ -1,15 +1,16 @@
 
 require 'omf_rete/tuple_stream'
+require 'omf_rete/join_op'
 
 module OMF::Rete
   module Planner
-    
-    
+
+
     # This class represents a planned join op.
-    # 
+    #
     #
     class JoinPlan < AbstractPlan
-      
+
       # stream1 - first stream to join
       # stream2 - second stream to join
       # joinSet - set of bindings to join on
@@ -17,8 +18,8 @@ module OMF::Rete
       # coverSet - set of leaf nodes contributing to this result
       #
       def initialize(stream1, stream2, joinSet, resultSet, coverSet, planBuilder)
-        super coverSet, resultSet 
-        
+        super coverSet, resultSet
+
         @planBuilder = planBuilder
         @left = stream1
         @right = stream2
@@ -34,7 +35,7 @@ module OMF::Rete
           description = @result_set.to_a.sort
           resultSet = IndexedTupleSet.new(description, indexPattern, nil, opts)
         end
-          
+
         indexPattern = @join_set.to_a
         leftSet = @left.materialize(indexPattern, nil, opts)
         rightSet = @right.materialize(indexPattern, nil, opts)
@@ -46,7 +47,7 @@ module OMF::Rete
       # Create a hash for this plan which allows us to
       # to identify identical plans.
       #
-      # Please note, that there is most likely a mroe efficient way to 
+      # Please note, that there is most likely a mroe efficient way to
       # calculate a hash with the above properties
       #
       def hash()
@@ -58,7 +59,7 @@ module OMF::Rete
         end
         @hash
       end
-      
+
       # Return the cost of this plan.
       #
       # TODO: Some more meaningful heuristic will be nice
@@ -69,25 +70,25 @@ module OMF::Rete
           rcost = @right.cost()
           #@cost = 1 + 1.2 * (lcost > rcost ? lcost : rcost)
           @cost = 1 + 1.2 * (lcost + rcost)
-          
+
         end
         @cost
       end
-      
+
       def describe(out = STDOUT, offset = 0, incr = 2, sep = "\n")
         out.write(" " * offset)
         result = @result_set.to_a.sort
         join = @join_set.to_a.sort
         out.write("join: [#{join.join(', ')}] => [#{result.join(', ')}] cost: #{cost}#{sep}")
-        @left.describe(out, offset + incr, incr, sep) 
-        @right.describe(out, offset + incr, incr, sep) 
+        @left.describe(out, offset + incr, incr, sep)
+        @right.describe(out, offset + incr, incr, sep)
       end
-      
+
       def to_s
         result = @result_set.to_a.sort
         join = @join_set.to_a.sort
         "JoinPlan [#{join.join(', ')}] out: [#{result.join(', ')}]"
-      end        
+      end
     end # PlanBuilder
 
   end # Planner