rdf-normalize 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: d7076dcfeccdbfc0b35ec046d0b338a6ad41d776
+  data.tar.gz: cd5f278797b575a3a6cced04890b9014c2350f42
+SHA512:
+  metadata.gz: 2510cec72f19af6eef55678382f688a5948b59fad4c6be18465c53a66d16b25bb543dadbd5a5148675d291afac133c8cc7d5399650ad9043b858c1a1f6165291
+  data.tar.gz: f785bd00b4abacf7da181daae96e2101aa449b19791a08742654451cf9a3b25abdf368dd77d99a86c4f74a49084b2d9af6464ea30bbed45589f87713e899ee63

data/AUTHORS

@@ -0,0 +1 @@
+* Gregg Kellogg <gregg@greggkellogg.net>

data/LICENSE

@@ -0,0 +1,25 @@
+This is free and unencumbered software released into the public domain.
+
+Anyone is free to copy, modify, publish, use, compile, sell, or
+distribute this software, either in source code form or as a compiled
+binary, for any purpose, commercial or non-commercial, and by any
+means.
+
+In jurisdictions that recognize copyright laws, the author or authors
+of this software dedicate any and all copyright interest in the
+software to the public domain. We make this dedication for the benefit
+of the public at large and to the detriment of our heirs and
+successors. We intend this dedication to be an overt act of
+relinquishment in perpetuity of all present and future rights to this
+software under copyright law.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
+OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+OTHER DEALINGS IN THE SOFTWARE.
+
+For more information, please refer to <http://unlicense.org>
+

data/README.md

@@ -0,0 +1,82 @@
+# RDF::Normalize
+RDF Graph normalizer for [RDF.rb][RDF.rb].
+
+[![Gem Version](https://badge.fury.io/rb/rdf-normalize.png)](http://badge.fury.io/rb/rdf-normalize)
+[![Build Status](https://secure.travis-ci.org/ruby-rdf/rdf-normalize.png?branch=master)](http://travis-ci.org/ruby-rdf/rdf-normalize)
+
+## Description
+This is a [Ruby][] implementation of a [RDF Normalize][] for [RDF.rb][].
+
+## Features
+RDF::Normalize generates normalized [N-Quads][] output for an RDF Dataset using the algorithm
+defined in [RDF Normalize][]. It also implements an RDF Writer interface, which can be used
+to serialize normalized statements.
+
+Algorithms implemented:
+
+* [URGNA2012](http://json-ld.github.io/normalization/spec/index.html#dfn-urgna2012)
+* [URDNA2014](http://json-ld.github.io/normalization/spec/index.html#dfn-urdna2015)
+
+Install with `gem install rdf-normalize`
+
+* 100% free and unencumbered [public domain](http://unlicense.org/) software.
+* Compatible with  Ruby >= 1.9.3.
+
+## Usage
+
+## Documentation
+Full documentation available on [Rubydoc.info][Normalize doc]
+
+### Principle Classes
+* {RDF::Normalize}
+  * {RDF::Normalize::Base}
+  * {RDF::Normalize::Format}
+  * {RDF::Normalize::Writer}
+  * {RDF::Normalize::URGNA2012}
+  * {RDF::Normalize::URDNA2015}
+
+
+## Dependencies
+
+* [Ruby](http://ruby-lang.org/) (>= 1.9.2)
+* [RDF.rb](http://rubygems.org/gems/rdf) (~> 1.1)
+
+## Installation
+
+The recommended installation method is via [RubyGems](http://rubygems.org/).
+To install the latest official release of the `RDF::Normalize` gem, do:
+
+    % [sudo] gem install rdf-normalize
+
+## Mailing List
+* <http://lists.w3.org/Archives/Public/public-rdf-ruby/>
+
+## Author
+* [Gregg Kellogg](http://github.com/gkellogg) - <http://greggkellogg.net/>
+
+## Contributing
+* Do your best to adhere to the existing coding conventions and idioms.
+* Don't use hard tabs, and don't leave trailing whitespace on any line.
+* Do document every method you add using [YARD][] annotations. Read the
+  [tutorial][YARD-GS] or just look at the existing code for examples.
+* Don't touch the `.gemspec`, `VERSION` or `AUTHORS` files. If you need to
+  change them, do so on your private branch only.
+* Do feel free to add yourself to the `CREDITS` file and the corresponding
+  list in the the `README`. Alphabetical order applies.
+* Do note that in order for us to merge any non-trivial changes (as a rule
+  of thumb, additions larger than about 15 lines of code), we need an
+  explicit [public domain dedication][PDD] on record from you.
+
+## License
+This is free and unencumbered public domain software. For more information,
+see <http://unlicense.org/> or the accompanying {file:LICENSE} file.
+
+[Ruby]:         http://ruby-lang.org/
+[RDF]:          http://www.w3.org/RDF/
+[YARD]:         http://yardoc.org/
+[YARD-GS]:      http://rubydoc.info/docs/yard/file/docs/GettingStarted.md
+[PDD]:          http://lists.w3.org/Archives/Public/public-rdf-ruby/2010May/0013.html
+[RDF.rb]:       http://rubydoc.info/github/ruby-rdf/rdf-normalize
+[N-Triples]:    http://www.w3.org/TR/rdf-testcases/#ntriples
+[RDF Normalize]:http://json-ld.github.io/normalization/spec/
+[Normalize doc]:http://rubydoc.info/github/ruby-rdf/rdf-normalize/master/file/README.markdown

data/VERSION

@@ -0,0 +1 @@
+0.1.0

data/lib/rdf/normalize.rb

@@ -0,0 +1,66 @@
+require 'rdf'
+
+module RDF
+  ##
+  # **`RDF::Normalize`** is an RDF Graph normalization plugin for RDF.rb.
+  #
+  # @example Requiring the `RDF::Normalize` module
+  #   require 'rdf/normalize'
+  #
+  # @example Returning an iterator for normalized statements
+  #
+  #   g = RDF::Graph.load("etc/doap.ttl")
+  #   RDF::Normalize.new(g).each_statement do |statement
+  #     puts statement.inspect
+  #   end
+  #
+  # @example Returning normalized N-Quads
+  #
+  #   g = RDF::Graph.load("etc/doap.ttl")
+  #   g.dump(:normalize)
+  #
+  # @example Writing a repository as normalized N-Quads
+  #
+  #   RDF::Normalize::Writer.open("etc/doap.nq") do |writer|
+  #     writer << RDF::Repository.load("etc/doap.ttl")
+  #   end
+  #
+  # @author [Gregg Kellogg](http://greggkellogg.net/)
+  module Normalize
+    require  'rdf/normalize/format'
+    require  'rdf/normalize/utils'
+    autoload :Base,       'rdf/normalize/base'
+    autoload :Carroll2001,'rdf/normalize/carroll2001'
+    autoload :URGNA2012,  'rdf/normalize/urgna2012'
+    autoload :URDNA2015,  'rdf/normalize/urdna2015'
+    autoload :VERSION,    'rdf/normalize/version'
+    autoload :Writer,     'rdf/normalize/writer'
+
+    # Enumerable to normalize
+    # @return [RDF::Enumerable]
+    attr_accessor :dataset
+
+    ALGORITHMS = {
+      carroll2001: :Carroll2001,
+      urgna2012:   :URGNA2012,
+      urdna2015:   :URDNA2015
+    }.freeze
+
+    ##
+    # Creates a new normalizer instance using either the specified or default normalizer algorithm
+    # @param [RDF::Enumerable] enumerable
+    # @param [Hash{Symbol => Object}] options
+    # @option options [Base] :algorithm (:urdna2015)
+    #   One of `:carroll2001`, `:urgna2012`, or `:urdna2015`
+    # @return [RDF::Normalize::Base]
+    # @raise [ArgumentError] selected algorithm not defined
+    def new(enumerable, options = {})
+      algorithm = options.fetch(:algorithm, :urdna2015)
+      raise ArgumentError, "No algoritm defined for #{algorithm.to_sym}" unless ALGORITHMS.has_key?(algorithm)
+      algorithm_class = const_get(ALGORITHMS[algorithm])
+      algorithm_class.new(enumerable, options)
+    end
+    module_function :new
+
+  end
+end

data/lib/rdf/normalize/base.rb

@@ -0,0 +1,15 @@
+module RDF::Normalize
+  ##
+  # Abstract class for pluggable normalization algorithms. Delegates to a default or selected algorithm if instantiated
+  module Base
+    attr_reader :dataset
+
+    # Enumerates normalized statements
+    #
+    # @yield statement
+    # @yieldparam [RDF::Statement] statement
+    def each(&block)
+      raise "Not Implemented"
+    end
+  end
+end

data/lib/rdf/normalize/carroll2001.rb

@@ -0,0 +1,166 @@
+module RDF::Normalize
+  class Carroll2001
+    include RDF::Enumerable
+    include Base
+    include Utils
+
+    ##
+    # Create an enumerable with grounded nodes
+    #
+    # @param [RDF::Enumerable] enumerable
+    # @return [RDF::Enumerable]
+    def initialize(enumerable, options)
+      @dataset = enumerable
+    end
+
+    def each(&block)
+      ground_statements, anon_statements = [], []
+      dataset.each_statement do |statement|
+        (statement.has_blank_nodes? ? anon_statements : ground_statements) << statement
+      end
+
+      nodes = anon_statements.map(&:to_quad).flatten.compact.select(&:node?).uniq
+
+      # Create a hash signature of every node, based on the signature of
+      # statements it exists in.  
+      # We also save hashes of nodes that cannot be reliably known; we will use
+      # that information to eliminate possible recursion combinations.
+      # 
+      # Any mappings given in the method parameters are considered grounded.
+      hashes, ungrounded_hashes = hash_nodes(anon_statements, nodes, {})
+
+      # FIXME: likely need to iterate until hashes and ungrounded_hashes are the same size
+      while hashes.size != ungrounded_hashes.size
+        raise "Not done"
+      end
+
+      # Enumerate all statements, replacing nodes with new ground nodes using the hash as an identifier
+      ground_statements.each(&block)
+      anon_statements.each do |statement|
+        quad = statement.to_quad.compact.map do |term|
+          term.node? ? RDF::Node.intern(hashes[term]) : term
+        end
+        block.call RDF::Statement.from(quad)
+      end
+    end
+
+  private
+
+    # Given a set of statements, create a mapping of node => SHA1 for a given
+    # set of blank nodes. 
+    #
+    # Returns a tuple of hashes:  one of grounded hashes, and one of all
+    # hashes.  grounded hashes are based on non-blank nodes and grounded blank
+    # nodes, and can be used to determine if a node's signature matches
+    # another.
+    #
+    # @param [Array] statements 
+    # @param [Array] nodes
+    # @param [Hash] grounded_hashes
+    #   mapping of node => SHA1 pairs as input, used to create more specific signatures of other nodes.
+    # @private
+    # @return [Hash, Hash]
+    def hash_nodes(statements, nodes, grounded_hashes)
+      hashes = grounded_hashes.dup
+      ungrounded_hashes = {}
+      hash_needed = true
+
+      # We may have to go over the list multiple times.  If a node is marked as
+      # grounded, other nodes can then use it to decide their own state of
+      # grounded.
+      while hash_needed
+        starting_grounded_nodes = hashes.size
+        nodes.each do | node |
+          unless hashes.member? node
+            grounded, hash = node_hash_for(node, statements, hashes)
+            if grounded
+              hashes[node] = hash
+            end
+            ungrounded_hashes[node] = hash
+          end
+        end
+
+        # after going over the list, any nodes with a unique hash can be marked
+        # as grounded, even if we have not tied them back to a root yet.
+        uniques = {}
+        ungrounded_hashes.each do |node, hash|
+          uniques[hash] = uniques.has_key?(hash) ? false : node
+        end
+        uniques.each do |hash, node|
+          hashes[node] = hash if node
+        end
+        hash_needed = starting_grounded_nodes != hashes.size
+      end
+      [hashes, ungrounded_hashes]
+    end
+
+    # Generate a hash for a node based on the signature of the statements it
+    # appears in.  Signatures consist of grounded elements in statements
+    # associated with a node, that is, anything but an ungrounded anonymous
+    # node.  Creating the hash is simply hashing a sorted list of each
+    # statement's signature, which is itself a concatenation of the string form
+    # of all grounded elements.
+    #
+    # Nodes other than the given node are considered grounded if they are a
+    # member in the given hash.
+    #
+    # @param [RDF::Node] node
+    # @param [Array<RDF::Statement>] statements
+    # @param [Hash] hashes
+    # @return [Boolean, String]
+    #   a tuple consisting of grounded being true or false and the String for the hash
+    def node_hash_for(node, statements, hashes)
+      statement_signatures = []
+      grounded = true
+      statements.each do | statement |
+        if statement.to_quad.include?(node)
+          statement_signatures << hash_string_for(statement, hashes, node)
+          statement.to_quad.compact.each do | resource |
+            grounded = false unless grounded?(resource, hashes) || resource == node
+          end
+        end
+      end
+      # Note that we sort the signatures--without a canonical ordering, 
+      # we might get different hashes for equivalent nodes.
+      [grounded,Digest::SHA1.hexdigest(statement_signatures.sort.to_s)]
+    end
+
+    # Provide a string signature for the given statement, collecting
+    # string signatures for grounded node elements.
+    # @return [String]
+    def hash_string_for(statement, hashes, node)
+      statement.to_quad.map {|r| string_for_node(r, hashes, node)}.join("")
+    end
+
+    # Returns true if a given node is grounded
+    # A node is groundd if it is not a blank node or it is included
+    # in the given mapping of grounded nodes.
+    # @return [Boolean]
+    def grounded?(node, hashes)
+      (!(node.node?)) || (hashes.member? node)
+    end
+
+    # Provides a string for the given node for use in a string signature
+    # Non-anonymous nodes will return their string form.  Grounded anonymous
+    # nodes will return their hashed form.
+    # @return [String]
+    def string_for_node(node, hashes, target)
+      case
+      when node.nil?
+        ""
+      when node == target
+        "itself"
+      when node.node? && hashes.member?(node)
+        hashes[node]
+      when node.node?
+        "a blank node"
+      # RDF.rb auto-boxing magic makes some literals the same when they
+      # should not be; the ntriples serializer will take care of us
+      when node.literal?
+        node.class.name + RDF::NTriples.serialize(node)
+      else
+        node.to_s
+      end
+    end
+  end
+end

data/lib/rdf/normalize/format.rb

@@ -0,0 +1,11 @@
+require 'rdf/nquads'
+
+module RDF::Normalize
+  class Format < RDF::Format
+    content_encoding 'utf-8'
+
+    # It reads like normal N-Quads
+    reader { RDF::NQuads::Reader}
+    writer { RDF::Normalize::Writer }
+  end
+end

data/lib/rdf/normalize/urdna2015.rb

@@ -0,0 +1,264 @@
+module RDF::Normalize
+  class URDNA2015
+    include RDF::Enumerable
+    include Base
+    include Utils
+
+    ##
+    # Create an enumerable with grounded nodes
+    #
+    # @param [RDF::Enumerable] enumerable
+    # @return [RDF::Enumerable]
+    def initialize(enumerable, options)
+      @dataset, @options = enumerable, options
+    end
+
+    def each(&block)
+      ns = NormalizationState.new(@options)
+      normalize_statements(ns, &block)
+    end
+
+    protected
+    def normalize_statements(ns, &block)
+      # Map BNodes to the statements they are used by
+      dataset.each_statement do |statement|
+        statement.to_quad.compact.select(&:node?).each do |node|
+          ns.add_statement(node, statement)
+        end
+      end
+
+      non_normalized_identifiers, simple = ns.bnode_to_statements.keys, true
+
+      while simple
+        simple = false
+        ns.hash_to_bnodes = {}
+
+        # Calculate hashes for first degree nodes
+        non_normalized_identifiers.each do |node|
+          hash = depth {ns.hash_first_degree_quads(node)}
+          debug("1deg") {"hash: #{hash}"}
+          ns.add_bnode_hash(node, hash)
+        end
+
+        # Create canonical replacements for hashes mapping to a single node
+        ns.hash_to_bnodes.keys.sort.each do |hash|
+          identifier_list = ns.hash_to_bnodes[hash]
+          next if identifier_list.length > 1
+          node = identifier_list.first
+          id = ns.canonical_issuer.issue_identifier(node)
+          debug("single node") {"node: #{node.to_ntriples}, hash: #{hash}, id: #{id}"}
+          non_normalized_identifiers -= identifier_list
+          ns.hash_to_bnodes.delete(hash)
+          simple = true
+        end
+      end
+
+      # Iterate over hashs having more than one node
+      ns.hash_to_bnodes.keys.sort.each do |hash|
+        identifier_list = ns.hash_to_bnodes[hash]
+
+        debug("multiple nodes") {"node: #{identifier_list.map(&:to_ntriples).join(",")}, hash: #{hash}"}
+        hash_path_list = []
+
+        # Create a hash_path_list for all bnodes using a temporary identifier used to create canonical replacements
+        identifier_list.each do |identifier|
+          next if ns.canonical_issuer.issued.include?(identifier)
+          temporary_issuer = IdentifierIssuer.new("_:b")
+          temporary_issuer.issue_identifier(identifier)
+          hash_path_list << depth {ns.hash_n_degree_quads(identifier, temporary_issuer)}
+        end
+        debug("->") {"hash_path_list: #{hash_path_list.map(&:first).inspect}"}
+
+        # Create canonical replacements for nodes
+        hash_path_list.sort_by(&:first).map(&:last).each do |issuer|
+          issuer.issued.each do |node|
+            id = ns.canonical_issuer.issue_identifier(node)
+            debug("-->") {"node: #{node.to_ntriples}, id: #{id}"}
+          end
+        end
+      end
+
+      # Yield statements using BNodes from canonical replacements
+      dataset.each_statement do |statement|
+        if statement.has_blank_nodes?
+          quad = statement.to_quad.compact.map do |term|
+            term.node? ? RDF::Node.intern(ns.canonical_issuer.identifier(term)[2..-1]) : term
+          end
+          block.call RDF::Statement.from(quad)
+        else
+          block.call statement
+        end
+      end
+    end
+
+  private
+
+    class NormalizationState
+      include Utils
+
+      attr_accessor :bnode_to_statements
+      attr_accessor :hash_to_bnodes
+      attr_accessor :canonical_issuer
+
+      def initialize(options)
+        @options = options
+        @bnode_to_statements, @hash_to_bnodes, @canonical_issuer = {}, {}, IdentifierIssuer.new("_:c14n")
+      end
+
+      def add_statement(node, statement)
+        bnode_to_statements[node] ||= []
+        bnode_to_statements[node] << statement unless bnode_to_statements[node].include?(statement)
+      end
+
+      def add_bnode_hash(node, hash)
+        hash_to_bnodes[hash] ||= []
+        hash_to_bnodes[hash] << node unless hash_to_bnodes[hash].include?(node)
+      end
+
+      # @param [RDF::Node] node
+      # @return [String] the SHA1 hexdigest hash of statements using this node, with replacements
+      def hash_first_degree_quads(node)
+        quads = bnode_to_statements[node].
+          map do |statement|
+            quad = statement.to_quad.map do |t|
+              case t
+              when node then RDF::Node("a")
+              when RDF::Node then RDF::Node("z")
+              else t
+              end
+            end
+            RDF::NQuads::Writer.serialize(RDF::Statement.from(quad))
+          end
+
+        debug("1deg") {"node: #{node}, quads: #{quads}"}
+        hexdigest(quads.sort.join)
+      end
+
+      # @param [RDF::Node] related
+      # @param [RDF::Statement] statement
+      # @param [IdentifierIssuer] issuer
+      # @param [String] position one of :s, :o, or :g
+      # @return [String] the SHA1 hexdigest hash
+      def hash_related_node(related, statement, issuer, position)
+        identifier = canonical_issuer.identifier(related) ||
+                     issuer.identifier(related) ||
+                     hash_first_degree_quads(related)
+        input = position.to_s
+        input << statement.predicate.to_ntriples unless position == :g
+        input << identifier
+        debug("hrel") {"input: #{input.inspect}, hash: #{hexdigest(input)}"}
+        hexdigest(input)
+      end
+
+      # @param [RDF::Node] identifier
+      # @param [IdentifierIssuer] issuer
+      # @return [Array<String,IdentifierIssuer>] the Hash and issuer
+      def hash_n_degree_quads(identifier, issuer)
+        debug("ndeg") {"identifier: #{identifier.to_ntriples}"}
+
+        # hash to related blank nodes map
+        map = {}
+
+        bnode_to_statements[identifier].each do |statement|
+          hash_related_statement(identifier, statement, issuer, map)
+        end
+
+        data_to_hash = ""
+
+        debug("ndeg") {"map: #{map.map {|h,l| "#{h}: #{l.map(&:to_ntriples)}"}.join('; ')}"}
+        depth do
+          map.keys.sort.each do |hash|
+            list = map[hash]
+            # Iterate over related nodes
+            chosen_path, chosen_issuer = "", nil
+            data_to_hash += hash
+
+            list.permutation do |permutation|
+              debug("ndeg") {"perm: #{permutation.map(&:to_ntriples).join(",")}"}
+              issuer_copy, path, recursion_list = issuer.dup, "", []
+
+              permutation.each do |related|
+                if canonical_issuer.identifier(related)
+                  path << canonical_issuer.issue_identifier(related)
+                else
+                  recursion_list << related if !issuer_copy.identifier(related)
+                  path << issuer_copy.issue_identifier(related)
+                end
+
+                # Skip to the next permutation if chosen path isn't empty and the path is greater than the chosen path
+                break if !chosen_path.empty? && path.length >= chosen_path.length
+              end
+              debug("ndeg") {"hash: #{hash}, path: #{path}, recursion: #{recursion_list.map(&:to_ntriples)}"}
+
+              recursion_list.each do |related|
+                result = depth {hash_n_degree_quads(related, issuer_copy)}
+                path << issuer_copy.issue_identifier(related)
+                path << "<#{result.first}>"
+                issuer_copy = result.last
+                break if !chosen_path.empty? && path.length >= chosen_path.length && path > chosen_path
+              end
+
+              if chosen_path.empty? || path < chosen_path
+                chosen_path, chosen_issuer = path, issuer_copy
+              end
+            end
+
+            data_to_hash += chosen_path
+            issuer = chosen_issuer
+          end
+        end
+
+        debug("ndeg") {"datatohash: #{data_to_hash.inspect}, hash: #{hexdigest(data_to_hash)}"}
+        return [hexdigest(data_to_hash), issuer]
+      end
+
+      protected
+
+      # FIXME: should be SHA-256.
+      def hexdigest(val)
+        Digest::SHA1.hexdigest(val)
+      end
+
+      # Group adjacent bnodes by hash
+      def hash_related_statement(identifier, statement, issuer, map)
+        statement.to_hash(:s, :p, :o, :g).each do |pos, term|
+          next if !term.is_a?(RDF::Node) || term == identifier
+
+          hash = depth {hash_related_node(term, statement, issuer, pos)}
+          map[hash] ||= []
+          map[hash] << term unless map[hash].include?(term)
+        end
+      end
+    end
+
+    class IdentifierIssuer 
+      def initialize(prefix = "_:c14n")
+        @prefix, @counter, @issued = prefix, 0, {}
+      end
+
+      # Return an identifier for this BNode
+      def issue_identifier(node)
+        @issued[node] ||= begin
+          res, @counter = @prefix + @counter.to_s, @counter + 1
+          res
+        end
+      end
+
+      def issued
+        @issued.keys
+      end
+
+      def identifier(node)
+        @issued[node]
+      end
+
+      # Duplicate this issuer, ensuring that the issued identifiers remain distinct
+      # @return [IdentifierIssuer]
+      def dup
+        other = super
+        other.instance_variable_set(:@issued, @issued.dup)
+        other
+      end
+    end
+  end
+end

data/lib/rdf/normalize/urgna2012.rb

@@ -0,0 +1,47 @@
+module RDF::Normalize
+  class URGNA2012 < URDNA2015
+
+    def each(&block)
+      ns = NormalizationState.new(@options)
+      normalize_statements(ns, &block)
+    end
+
+    class NormalizationState < URDNA2015::NormalizationState
+      protected
+
+      # 2012 version uses SHA-1
+      def hexdigest(val)
+        Digest::SHA1.hexdigest(val)
+      end
+
+      # @param [RDF::Node] related
+      # @param [RDF::Statement] statement
+      # @param [IdentifierIssuer] issuer
+      # @param [String] position one of :s, :o, or :g
+      # @return [String] the SHA1 hexdigest hash
+      def hash_related_node(related, statement, issuer, position)
+        identifier = canonical_issuer.identifier(related) ||
+                     issuer.identifier(related) ||
+                     hash_first_degree_quads(related)
+        input = position.to_s
+        input << statement.predicate.to_s
+        input << identifier
+        debug("hrel") {"input: #{input.inspect}, hash: #{hexdigest(input)}"}
+        hexdigest(input)
+      end
+
+      # In URGNA2012, the position parameter passed to the Hash Related Blank Node algorithm was instead modeled as a direction parameter, where it could have the value p, for property, when the related blank node was a `subject` and the value r, for reverse or reference, when the related blank node was an `object`. Since URGNA2012 only normalized graphs, not datasets, there was no use of the `graph` position.
+      def hash_related_statement(identifier, statement, issuer, map)
+        if statement.subject.node? && statement.subject != identifier
+          hash = depth {hash_related_node(statement.subject, statement, issuer, :p)}
+          map[hash] ||= []
+          map[hash] << statement.subject unless map[hash].include?(statement.subject)
+        elsif statement.object.node? && statement.object != identifier
+          hash = depth {hash_related_node(statement.object, statement, issuer, :r)}
+          map[hash] ||= []
+          map[hash] << statement.object unless map[hash].include?(statement.object)
+        end
+      end
+    end
+  end
+end

data/lib/rdf/normalize/utils.rb

@@ -0,0 +1,33 @@
+module RDF::Normalize
+  module Utils
+    # Add debug event to debug array, if specified
+    #
+    #   param [String] message
+    #   yieldreturn [String] appended to message, to allow for lazy-evaulation of message
+    def debug(*args)
+      options = args.last.is_a?(Hash) ? args.pop : {}
+      return unless options[:debug] || @options[:debug]
+      depth = options[:depth] || @options[:depth]
+      d_str = depth > 100 ? ' ' * 100 + '+' : ' ' * depth
+      list = args
+      list << yield if block_given?
+      message = d_str + (list.empty? ? "" : list.join(": "))
+      options[:debug] << message if options[:debug].is_a?(Array)
+      @options[:debug] << message if @options[:debug].is_a?(Array)
+      $stderr.puts(message) if @options[:debug] == TrueClass
+    end
+    module_function :debug
+
+    # Increase depth around a method invocation
+    # @yield
+    #   Yields with no arguments
+    # @yieldreturn [Object] returns the result of yielding
+    # @return [Object]
+    def depth
+      @options[:depth] += 1
+      ret = yield
+      @options[:depth] -= 1
+      ret
+    end
+  end
+end

data/lib/rdf/normalize/writer.rb

@@ -0,0 +1,79 @@
+module RDF::Normalize
+  ##
+  # A RDF Graph normalization serialiser.
+  #
+  # Normalizes the enumerated statements into normal form in the form of N-Quads.
+  #
+  # @author [Gregg Kellogg](http://kellogg-assoc.com/)
+  class Writer < RDF::NQuads::Writer
+    format RDF::Normalize::Format
+
+    # @attr_accessor [RDF::Repository] Repository of statements to serialized
+    attr_accessor :repo
+
+    ##
+    # Initializes the writer instance.
+    #
+    # @param  [IO, File] output
+    #   the output stream
+    # @param  [Hash{Symbol => Object}] options
+    #   any additional options
+    # @yield  [writer] `self`
+    # @yieldparam  [RDF::Writer] writer
+    # @yieldreturn [void]
+    # @yield  [writer]
+    # @yieldparam [RDF::Writer] writer
+    def initialize(output = $stdout, options = {}, &block)
+      super do
+        @options[:depth] ||= 0
+        @repo = RDF::Repository.new
+        if block_given?
+          case block.arity
+            when 0 then instance_eval(&block)
+            else block.call(self)
+          end
+        end
+      end
+    end
+
+    ##
+    # Defer writing to epilogue
+    def write_statement(statement)
+      self
+    end
+
+    ##
+    # Outputs the Graph representation of all stored triples.
+    #
+    # @return [void]
+    def write_epilogue
+      statements = RDF::Normalize.new(@repo, @options).
+        statements.
+        reject(&:variable?).
+        map {|s| format_statement(s)}.
+        sort.
+        each do |line|
+          puts line
+        end
+    end
+
+    protected
+
+    ##
+    # Adds a statement to be serialized
+    # @param  [RDF::Statement] statement
+    # @return [void]
+    def insert_statement(statement)
+      @repo.insert(statement)
+    end
+
+    ##
+    # Insert an Enumerable
+    #
+    # @param  [RDF::Enumerable] graph
+    # @return [void]
+    def insert_statements(enumerable)
+      @repo = enumerable
+    end
+  end
+end

metadata

@@ -0,0 +1,160 @@
+--- !ruby/object:Gem::Specification
+name: rdf-normalize
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Gregg Kellogg
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2015-05-20 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: rdf
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.1'
+- !ruby/object:Gem::Dependency
+  name: rdf-spec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.1'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.1'
+- !ruby/object:Gem::Dependency
+  name: open-uri-cached
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.0'
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 0.0.5
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.0'
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 0.0.5
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.2'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.2'
+- !ruby/object:Gem::Dependency
+  name: webmock
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.17'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.17'
+- !ruby/object:Gem::Dependency
+  name: json-ld
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.1'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.1'
+- !ruby/object:Gem::Dependency
+  name: yard
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.8'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.8'
+description: RDF::Normalize is a Graph normalizer for the RDF.rb library suite.
+email: public-rdf-ruby@w3.org
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- AUTHORS
+- LICENSE
+- README.md
+- VERSION
+- lib/rdf/normalize.rb
+- lib/rdf/normalize/base.rb
+- lib/rdf/normalize/carroll2001.rb
+- lib/rdf/normalize/format.rb
+- lib/rdf/normalize/urdna2015.rb
+- lib/rdf/normalize/urgna2012.rb
+- lib/rdf/normalize/utils.rb
+- lib/rdf/normalize/writer.rb
+homepage: http://github.com/gkellogg/rdf-normalize
+licenses:
+- Public Domain
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: 1.9.2
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: rdf-normalize
+rubygems_version: 2.4.7
+signing_key: 
+specification_version: 4
+summary: RDF Graph normalizer for Ruby.
+test_files: []
+has_rdoc: false