omf_rete 0.5

data/.gitignore

@@ -0,0 +1,4 @@
+.project
+Rakefile-back
+examples/of.rb
+lib/omf_rete/UNUSED/tuple_set.rb

data/README.md

@@ -0,0 +1,182 @@
+
+= Introduction
+
+This library implements a tuple store with a query and subscribe mechanism. 
+A subscribe is effectively a standing query which executes a block whenever
+a newly added tuple together with the store's content fullfills the filter
+specification.
+
+The store holds same sized tuples with each value being assigned a name and 
+type at creation to support varous convenience functions to create and retrieve
+tuples.
+
+The following code snippet creates a simple RDF store and adds a few triplets
+to it.
+
+  store = OMF::Rete::Store.new(3)
+  store.add('myFridge', 'contains', 'milk')
+
+A filter consists of an array of tuple +patterns+ and a +block+ to be called when the store
+contains a set of tuples matching the +pattern+.
+
+The following filter only looks for a single, specific tuple. The supplied block is called
+immediately if the tuple already exists in the store, or when such a tuple would be added at a later
+stage.
+
+  store.subscribe(:report_problem, [
+    ['myFridge', 'status', 'broken']
+  ]) do |m|
+  	puts "My fridge is broken"
+  end
+  
+The following filter contains two +patterns+ and therefore both need to be matched at the same
+time in order for the block to fire. Note, that the order these tuples are added to the store
+or the interval between is irrelevant.
+
+  store.subscribe(:save_milk, [
+    ['myFridge', 'status', 'broken'],
+    ['myFridge', 'contains', 'milk'],
+  ]) do |m|
+  	puts "Save the milk from my fridge"
+  end
+  
+  
+So far the filter pattern were fully specified. The <tt>:_</tt> symbol can be used as a wildcard identifier.
+The following code snippet reports anything which is broken.
+
+  store.subscribe(:something_broken, [
+    [:_, 'status', 'broken']
+  ]) do |m|
+    puts "Something is broken"
+  end
+
+_Not implemented yet_
+Similar to OMF::Rete::Store#addNamed we can describe a pattern with a hash. Any value not named is automatically 
+wildcarded. Therefore, an alternative represenation of the previous filter is as follows:
+
+  store.subscribe(:something_broken, [
+    {:pred => 'status', :obj => 'broken'}
+  ]) do |m|
+    puts "Something is broken"
+  end
+  
+The +match+ argument to the block holds the context of the match and specifically, the tuples involved 
+in the match.
+
+  store.subscribe(:something_broken, [
+    [:_, 'status', 'broken']
+  ]) do |match|
+    what = match.tuples[0][:subject]
+    puts "#{what} is broken"
+  end
+  
+<tt>match.tuples</tt> returns an area of tuples one for each pattern. The matched tuple for the first pattern is at index 0,
+the second one at index 1, and so on. Individual values of a tuple can be retrieved through the initially declared 
+value name (see OMF::Rete::Tuple#[]).
+
+Let us assume we are monitoring many fridges, so if we want to report broken ones with milk inside, we need to ensure
+that the +subject+ in both patterns in our second example are identical. Or in more technical terms, we need to +bind+ or +join+
+values across patterns. A binding variable is identified by a symbol with a trailing <b>?</b>.
+
+  store.subscribe(:save_milk, [
+    [:fridge?, 'status', 'broken'],
+    [:fridge?, 'contains', 'milk'],
+  ]) do |match|
+    fridge = match[:fridge]
+    puts "Save the milk from #{fridge}"
+  end
+
+<tt>match[bindingName]</tt> (without the '?') returns the value bound to <tt>:fridge?</tt> for this match. 
+Obviously <tt>match.tuples[0][:subject]</tt> will return the same value.
+
+== Functions
+
+Pattern matches alone are not always sufficient. For instance, let us assume that we have also stored the age in years
+of each monitored fridge and want to replace each broken one which is older than 10 years. To describe such a filter
+we introduce functions (or what in SPARQL is refered to as a FILTER) which allow us to restrict bound values.
+
+Functions are identified by the <tt>:PROC</tt> symbol in the first position of a pattern, followed by the function 
+name, and the list of parameters. Effectively, a function filters the values previosuly bound to a variable to those
+for which the function returns true.
+
+  store.subscribe(:replace_old_ones, [
+    [:fridge?, 'status', 'broken'],
+    [:fridge?, 'age', :age?],
+    [:PROC, :greater, :age?, 10]
+  ]) do |match|
+    puts "Replace #{match[:fridge]}"
+  end
+
+<b>Design Note:</b> A more generic solution based on a 'lambda' is most likely cleaner. This is effectively
+identical to the final block, except that the block should return +true+ for tuples passing the filter,
+and +false+ for all others. To further simplify this and also reduce the search space, we can define a
++filter+ function which takes a list of bound variables and calls the associated block with specific bindings.
+
+  store.subscribe(:replace_old_ones, [
+    [:fridge?, 'status', 'broken'],
+    [:fridge?, 'age', :age?],
+    filter(:age?) { |age| age > 10 }
+  ]) do |match|
+    puts "Replace #{match[:fridge]}"
+  end
+  
+== Set Operators
+
+Let us assume we want the store to not only reflect the current facts but the entire history of a system. We
+can achieve that by adding a timestamp to each fact and never retract facts.
+
+  store = OMF::Rete::Store.new(:subj => String, :pred => String, :obj => Object, :tstamp => Time)
+  
+This now allows us to capture that a fridge broke on a specific date and was fixed some times later.
+
+  store.add('myFridge', 'status', 'broken', '2008-12-20')
+  store.add('myFridge', 'status', 'ok', '2008-12-22')
+  
+However, how can we now determine that a specific fridge is CURRENTLY broken? The pattern
+<tt>[:f?, 'status' 'broken']</tt> will identify all fridges which are currently broken, as well as those
+which broke in the past but are ok now. What we need is a way to describe sets and a filter to select a single tuple 
+from each set. In our example, each set would contain all the status messages for a specific fridge, while
+the filter picks the one with the most recent timestamp. 
+
+The current syntax achieves this through special match values. For instance, <tt>:LATEST</tt> for <tt>Time</tt>
+types picks the most recent fact.
+  
+  [:fridge?, 'status', :_, :LATEST]
+  
+To find all currently broken fridges we need to bind this to all broken status facts.
+
+  store.subscribe(:broken_lately, [
+    [:fridge?, 'status', :_, :LATEST],
+    [:fridge?, 'status', 'broken']
+  ]) do |match|
+    puts "#{match[:fridge]} is broken"
+  end
+  
+<b>Design Note:</b> This seems to be a fairly ad-hoc syntax. Is there a better one? This assumes that there is no join 
+on any of the bound variables, they are simply keys for the sets. But overloading functionality always adds complexity.
+
+== Negated Conditions
+
+Now let us consider we know that our fridge is broken and we want to monitor any future status updates. 
+There may be many different status types and we are interested in all of them as long as they are
+different to 'broken'. In other words, we need a way to describe what is refered to as a 'negated
+condition' and is defined by a leading <tt>:NOT</tt>, followed by one or multiple patterns describing
+what should NOT be in the store.
+
+  store.subscribe(:find_latest, [
+    ['My Fridge', :status, :_, :LATEST],
+    [:NOT, ['My Fridge', 'status', 'broken']]
+  ]) do |match|
+    puts "Status for my fridge changed to '#{match.tuples[0][:obj]}."
+  end
+  
+Please note that the above example fails to report when my fridge is reported as broken again.
+
+= Implementation
+
+
+
+
+
+
+

data/Rakefile

@@ -0,0 +1,14 @@
+require 'rake/testtask'
+require "bundler/gem_tasks"
+
+task :default => :test
+
+
+#
+# TESTING
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << "tests"
+  t.test_files = FileList['tests/test.rb']
+  t.verbose = true
+end

data/lib/omf_rete/abstract_tuple_set.rb

@@ -0,0 +1,68 @@
+
+module OMF::Rete
+  #
+  # This class maintains a set of tuples and
+  # supports a block being attached which is
+  # being called whenever a tuple is added or
+  # removed.
+  #
+  # The TupleSet is defined by a +description+. 
+  #
+  # The +description+ is an array of the
+  # same length as the tuples maintained. Each element,
+  # if not nil, names the binding variable associated with it.
+  # The position of a binding can be retrieved with 
+  # +index_for_binding+.
+  #
+  class AbstractTupleSet
+    
+    attr_reader :description
+    attr_accessor :source
+    
+    def initialize(description, source = nil)
+      @description = description
+      @source = source
+    end
+    
+    def addTuple(tuple)
+      raise 'Abstract class'
+    end
+    
+    # Call block for every tuple stored in this set currently and
+    # in the future. In other words, the block may be called even after this
+    # method returns. 
+    #
+    # The block will be called with one parameters, the 
+    # tuple added.
+    #
+    def on_add(&block)
+      raise 'Abstract class'
+    end
+
+    # Return all stored tuples in an array.
+    def to_a
+      raise 'Abstract class'
+    end
+    
+    # Retunr the index into the tuple for the binding variable +bname+.
+    #
+    # Note: This index is different to the set index used in +IndexedTupleSet+
+    #
+    def index_for_binding(bname)
+      @description.find_index do |el|
+        el == bname
+      end
+    end
+    
+    def binding_at(index)
+      @description[index]
+    end
+
+    def describe(out = STDOUT, offset = 0, incr = 2, sep = "\n")
+      raise 'Abstract class'
+    end
+    
+  
+  end # class 
+end # module
+

data/lib/omf_rete/indexed_tuple_set.rb

@@ -0,0 +1,129 @@
+require 'omf_rete/abstract_tuple_set'
+
+module OMF::Rete
+  #
+  # This class maintains a set of tuples and
+  # supports a block being attached which is
+  # being called whenever a tuple is added or
+  # removed.
+  #
+  # The IndexedTupleSet is defined by a +description+ and an
+  # +indexPattern+. 
+  #
+  # The +description+ is an array of the
+  # same length as the tuples maintained. Each element,
+  # if not nil, names the binding variable associated with it.
+  # The position of a binding can be retrieved with 
+  # +index_for_binding+.
+  #
+  # The +indexPattern+ describes which elements of the inserted
+  # tuple are being combined in an array to form the index 
+  # key for each internal tuple. The elements in the +indexPattern+ 
+  # are described by the binding name.
+  #
+  #
+  class IndexedTupleSet < AbstractTupleSet
+    
+    attr_reader :indexPattern
+    attr_writer :transient # if true only process tuple but don't store it
+    
+    def initialize(description, indexPattern, source = nil, opts = {})
+      super description, source
+      if (indexPattern.length == 0)
+        raise "Expected index to be non-nil (#{description.join(', ')})"
+      end
+      @indexPattern = indexPattern
+      @indexMap = indexPattern.collect do |bname|
+        index_for_binding(bname)
+      end
+
+      @index = {}
+    end
+    
+    def addTuple(tuple)
+      key = @indexMap.collect do |ii|
+        tuple[ii]
+      end
+
+      if @transient
+        @onAddBlockWithIndex.call(key, tuple) if @onAddBlockWithIndex
+        @onAddBlock.call(tuple) if @onAddBlock
+      else
+        vset = (@index[key] ||= Set.new)
+        if vset.add?(tuple)
+          # new value
+          @onAddBlockWithIndex.call(key, tuple) if @onAddBlockWithIndex
+          @onAddBlock.call(tuple) if @onAddBlock
+        end
+      end
+      tuple # return added tuple
+    end
+    
+    # Call block for every tuple stored in this set currently and
+    # in the future. In other words, the block may be called even after this
+    # method returns. 
+    #
+    # The block will be called with one parameters, the 
+    # tuple added.
+    #
+    # Note: Only one +block+ can be registered at a time
+    #
+    def on_add(&block)
+      @index.each do |index, values|
+        values.each do |v|
+          block.call(v)
+        end
+      end
+      @onAddBlock = block
+    end
+
+
+    # Call block for every tuple stored in this set currently and
+    # in the future. In other words, the block may be called even after this
+    # method returns. 
+    #
+    # The block will be called with two parameters, the index of the tuple followed by the 
+    # tuple itself.
+    #
+    # Note: Only one +block+ can be registered at a time
+    #
+    def on_add_with_index(&block)
+      @index.each do |index, values|
+        values.each do |v|
+          block.call(index, v)
+        end
+      end
+      @onAddBlockWithIndex = block
+    end
+
+    # Return the set of tuples index by +key+. 
+    # Will return nil if nothing is stored for +key+
+    #
+    def [](key)
+      res = @index[key]
+      res
+    end
+    
+    # Return all stored tuples in an array.
+    def to_a
+      a = []
+      @index.each_value do |s|
+        s.each do |t|
+          a << t
+        end
+      end
+      a
+    end
+    
+    def describe(out = STDOUT, offset = 0, incr = 2, sep = "\n")
+      out.write(" " * offset)
+      desc = @description.collect do |e| e || '*' end
+      out.write("ts: [#{desc.join(', ')}]")
+      ind = @indexMap.collect do |i| @description[i] end
+      out.write("  (index: [#{ind.sort.join(', ')}])#{sep}")
+      @source.describe(out, offset + incr, incr, sep) if @source 
+    end
+    
+  end # class IndexedTupleSet
+end # module
+

data/lib/omf_rete/join_op.rb

@@ -0,0 +1,113 @@
+require 'omf_rete/indexed_tuple_set'
+
+module OMF::Rete
+
+  # This class implements the join operation between two 
+  # +IndexedTupleSets+ feeding into a third, result tuple set.
+  # The size of both incoming tuple sets needs to be identical and they
+  # are supposed to be indexed on the same list of variables as this is
+  # what they wil be joined at.
+  # 
+  # Implementation Note: We first calculate a +combinePattern+ 
+  # from the +description+ of the result set.
+  # The +combinePattern+ describes how to create a joined tuple to insert
+  # into the result tuple set. The +combinePattern+ is an array of 
+  # the same size as the result tuple. Each element is a 2-array 
+  # with the first element describing the input set (0 .. left, 1 .. right)
+  # and the second one the index from which to take the value.
+  # 
+  #
+  class JoinOP
+    def initialize(leftSet, rightSet, resultSet)
+      @resultSet = resultSet
+      @left = leftSet
+      @right = rightSet
+      
+      @combinePattern = resultSet.description.collect do |bname|
+        side = 0
+        unless (i = leftSet.index_for_binding(bname))
+          side = 1
+          unless (i = rightSet.index_for_binding(bname))
+            raise "Can't find binding '#{bname}' in either streams. Should never happen"
+          end
+        end
+        #description << bname
+        [side, i]
+      end
+      @resultLength = @combinePattern.length
+      
+      leftSet.on_add_with_index do |index, ltuple|
+        if (rs = rightSet[index])
+          rs.each do |rtuple|
+            add_result(ltuple, rtuple)
+          end
+        end
+      end
+      rightSet.on_add_with_index do |index, rtuple|
+        if (ls = leftSet[index])
+          ls.each do |ltuple|
+            add_result(ltuple, rtuple)
+          end
+        end
+      end
+      
+      # Supporting 'check_for_tuple'
+      @left_pattern = @left.description.map do |bname|
+        @resultSet.index_for_binding(bname)
+      end
+      @right_pattern = @right.description.map do |bname|
+        @resultSet.index_for_binding(bname)
+      end
+
+    end
+    
+    # Check if +tuple+ can be produced by this join op. We first
+    # check if we can find a match on one side and then request
+    # from the other side all the tuples which would lead to full
+    # join.
+    #
+    def check_for_tuple(tuple)
+      ltuple = @left_pattern.map {|i| tuple[i]}
+      if @left.check_for_tuple(ltuple)
+        rtuple = @right_pattern.map {|i| tuple[i]}
+        if @right.check_for_tuple(rtuple)
+          return true
+        end
+      end
+      return false
+    end
+    
+    def description()
+      @resultSet.description
+    end
+    
+    def describe(out = STDOUT, offset = 0, incr = 2, sep = "\n")
+      out.write(" " * offset)
+      result = @combinePattern.collect do |side, index|
+        (side == 0) ? @left.binding_at(index) : @right.binding_at(index)
+      end
+      out.write("join: [#{@left.indexPattern.sort.join(', ')}] => [#{result.sort.join(', ')}]#{sep}")
+      @left.describe(out, offset + incr, incr, sep) 
+      @right.describe(out, offset + incr, incr, sep)         
+    end
+
+    private
+
+    def add_result(ltuple, rtuple)
+      unless @resultLength
+        i = 2
+      end
+      result = Array.new(@resultLength)
+      i = 0
+      @combinePattern.each do |setId, index|
+        t = setId == 0 ? ltuple : rtuple
+        result[i] = t[index]
+        i += 1
+      end
+      @resultSet.addTuple(result)
+    end
+    
+    
+    
+  end # class
+end # module

data/lib/omf_rete/planner/abstract_plan.rb

@@ -0,0 +1,57 @@
+module OMF::Rete
+  module Planner
+    
+    
+    # This class is the super class for all plans
+    # 
+    #
+    class AbstractPlan
+
+      attr_reader :cover_set, :result_set
+      
+      #
+      # coverSet -- set of source plans covered by this plan
+      # resultSet -- set of bindings provided by this source
+      #
+      def initialize(coverSet, resultSet)
+        @cover_set = coverSet
+        @result_set = resultSet
+        @is_used = false
+        @is_complete = false
+      end
+      
+      def result_description
+        @result_set.to_a.sort
+      end
+      
+
+      
+      # Return true if this plan is a complete one.
+      #
+      # A complete plan covers (@coverSet) all leaf plans.
+      #
+      def complete?()
+        @is_complete
+      end
+      
+      # Set this plan to be complete
+      #
+      def complete()
+        @is_complete = true
+      end
+      
+      # Return true if used by some higher plan
+      #
+      def used?()
+        @is_used
+      end
+      
+      # Informs the plan that it is used by some higher plan
+      #
+      def used()
+        @is_used = true
+      end
+    end # PlanBuilder
+
+  end # Planner
+end # module

data/lib/omf_rete/planner/filter_plan.rb

@@ -0,0 +1,49 @@
+
+require 'omf_rete/planner/plan_builder'
+require 'omf_rete/planner/abstract_plan'
+require 'set'
+
+module OMF::Rete
+  module Planner
+    
+    # This class represents a filter operation on a binding stream.
+    # 
+    #
+    class FilterPlan
+      attr_reader :description
+      
+      #
+      # resultSet - set of bindings provided by this source
+      #
+      def initialize(projectPattern, outDescription = nil, &block)
+        @projectPattern = projectPattern
+        @description = outDescription #|| projectPattern.sort
+        @block = block
+      end
+      
+      
+      def materialize(description, source, opts)
+        # A filter has the same in as well as out description as it doesn't change
+        # the tuple just potentially drop it.
+        #  
+        pts = FilterTupleStream.new(@projectPattern, description, &@block)
+        pts.source = source
+#          if (in_description == @projectPattern)
+#            pts.on_add &@block
+#          else
+#            projectIndex = @projectPattern.collect do |bname|
+#              pts.index_for_binding(bname)
+#            end
+#            pts.on_add do |*t|
+#              pt = projectIndex.collect do |index|
+#                t[index]
+#              end
+#              @block.call(*pt)
+#            end
+#          end
+        pts
+      end
+    end # FilterPlan
+
+  end # Planner
+end # module