rdf 0.3.3 → 0.3.4

data/README

@@ -27,44 +27,98 @@ Features
   use of any one part of the library without needing to load up the rest.
 * Compatible with Ruby 1.8.7+, Ruby 1.9.x, and JRuby 1.4/1.5.
 * Compatible with older Ruby versions with the help of the [Backports][] gem.
+* Performs auto-detection of input to select appropriate Reader class if one
+  cannot be determined from file characteristics.
 
 Tutorials
 ---------
 
-* [Getting data from the Semantic Web using Ruby and RDF.rb]
-  (http://semanticweb.org/wiki/Getting_data_from_the_Semantic_Web_%28Ruby%29)
-* [Using RDF.rb and Spira to process RDF data from the British Ordnance Survey]
-  (http://ultramagnus.org/?p=85)
-* [Getting started with RDF and SPARQL using 4store and RDF.rb]
-  (http://www.jenitennison.com/blog/node/152)
+* [Getting data from the Semantic Web using Ruby and RDF.rb](http://semanticweb.org/wiki/Getting_data_from_the_Semantic_Web_%28Ruby%29)
+* [Using RDF.rb and Spira to process RDF data from the British Ordnance Survey](http://stephenpope.co.uk/?p=85)
+* [Getting started with RDF and SPARQL using 4store and RDF.rb](http://www.jenitennison.com/blog/node/152)
+
+Command Line
+------------
+When installed, RDF.rb includes a `rdf` shell script which acts as a wrapper to perform a number of different
+operations on RDF files using available readers and writers.
+
+* `serialize`: Parse an RDF input and re-serializing to N-Triples or another available format using `--output-format` option.
+* `count`: Parse and RDF input and count the number of statements.
+* `subjects`: Returns unique subjects from parsed input.
+* `objects`: Returns unique objects from parsed input.
+* `predicates`: Returns unique objects from parsed input.
 
 Examples
 --------
 
     require 'rdf'
-    
     include RDF
 
 ### Writing RDF data using the N-Triples format
 
     require 'rdf/ntriples'
+    graph = RDF::Graph.new << [:hello, RDF::DC.title, "Hello, world!"]
+    graph.dump(:ntriples)
     
-    RDF::Writer.open("hello.nt") do |writer|
-      writer << RDF::Graph.new do |graph|
-        graph << [:hello, RDF::DC.title, "Hello, world!"]
-      end
-    end
+or
+
+  RDF::Writer.open("hello.nt") { |writer| writer << graph }
 
 ### Reading RDF data in the N-Triples format
 
     require 'rdf/ntriples'
+    graph = RDF::Graph.load("http://rdf.rubyforge.org/doap.nt")
     
+or
+
     RDF::Reader.open("http://rdf.rubyforge.org/doap.nt") do |reader|
       reader.each_statement do |statement|
         puts statement.inspect
       end
     end
 
+### Reading RDF data in other formats
+{RDF::Reader.open} and {RDF::Repository.load} use a number of mechanisms to determine the appropriate reader
+to use when loading a file. The specific format to use can be forced using, e.g. `:format => :ntriples`
+option where the specific format symbol is determined by the available readers. Both also use
+MimeType or file extension, where available.
+
+    require 'linkeddata'
+    
+    graph = RDF::Graph.load("etc/doap.nq", :format => :nquads)
+
+A specific sub-type of Reader can also be invoked directly:
+
+    require 'rdf/nquads'
+    
+    RDF::NQuads::Reader.open("http://rdf.rubyforge.org/doap.nq") do |reader|
+      reader.each_statement do |statement|
+        puts statement.inspect
+      end
+    end
+
+Reader/Writer implementations may override {RDF::Format.detect}, which takes a small sample if input
+and return a boolean indicating if it matches that specific format. In the case that a format cannot
+be detected from filename or other options, or that more than one format is identified,
+{RDF::Format.for} will query each loaded format by invoking it's `detect` method, and the first successful
+match will be used to read the input.
+
+### Writing RDF data using other formats
+{RDF::Writer.open}, {RDF::Enumerable#dump}, {RDF::Writer.dump} take similar options to {RDF::Reader.open} to determine the
+appropriate writer to use.
+
+    require 'linkeddata'
+
+    RDF::Writer.open("hello.nq", :format => :nquads) do |writer|
+      writer << RDF::Repository.new do |repo|
+        repo << RDF::Statement.new(:hello, RDF::DC.title, "Hello, world!", :context => RDF::URI("context"))
+      end
+    end
+
+A specific sub-type of Writer can also be invoked directly:
+
+    graph.dump(:nq)
+
 ### Querying RDF data using basic graph patterns (BGPs)
 
     require 'rdf/ntriples'
@@ -82,6 +136,8 @@ Examples
       puts "name=#{solution.name} email=#{solution.email}"
     end
 
+A separate [SPARQL][SPARQL doc] gem builds on basic BGP support to provide full support for [SPARQL 1.0](http://www.w3.org/TR/rdf-sparql-query/) queries.
+
 ### Using pre-defined RDF vocabularies
 
     DC.title      #=> RDF::URI("http://purl.org/dc/terms/title")
@@ -111,7 +167,16 @@ Documentation
 * {RDF::Value}
   * {RDF::Term}
     * {RDF::Literal}
+      * {RDF::Literal::Boolean}
+      * {RDF::Literal::Date}
+      * {RDF::Literal::DateTime}
+      * {RDF::Literal::Decimal}
+      * {RDF::Literal::Double}
+      * {RDF::Literal::Integer}
+      * {RDF::Literal::Time}
+      * [RDF::XSD](http://rubydoc.info/github/gkellogg/rdf-xsd/master/frames) (plugin)
     * {RDF::Resource}
+      * {RDF::List}
       * {RDF::Node}
       * {RDF::URI}
       * {RDF::Graph}
@@ -127,14 +192,23 @@ Documentation
 
 ### RDF Serialization Formats
 
+The following is a partial list of RDF formats implemented either natively, or through the inclusion of
+other gems:
+
 * {RDF::NTriples}
+* {RDF::NQuads}
+* [JSON::LD][JSONLD doc] (plugin)
 * [RDF::JSON](http://rdf.rubyforge.org/json/) (plugin)
-* [RDF::N3](http://rdf.rubyforge.org/n3/) (plugin)
+* [RDF::Microdata][Microdata doc] (plugin)
+* [RDF::N3][N3 doc] (plugin)
 * [RDF::Raptor::RDFXML](http://rdf.rubyforge.org/raptor/) (plugin)
 * [RDF::Raptor::Turtle](http://rdf.rubyforge.org/raptor/) (plugin)
-* [RDF::RDFa](http://rdf.rubyforge.org/rdfa/) (plugin)
-* [RDF::RDFXML](http://rdf.rubyforge.org/rdfxml/) (plugin)
+* [RDF::RDFa][RDFa doc] (plugin)
+* [RDF::RDFXML][RDFXML doc] (plugin)
 * [RDF::Trix](http://rdf.rubyforge.org/trix/) (plugin)
+* [RDF::Turtle][Turtle doc] (plugin)
+
+The meta-gem [LinkedData][LinkedData doc] includes many of these gems.
 
 ### RDF Storage
 
@@ -149,6 +223,8 @@ Documentation
   * {RDF::Mutable}
   * {RDF::Durable}
 * {RDF::Transaction}
+* [RDF::AllegroGraph](http://rubydoc.info/github/emk/rdf-agraph/master/frames) (plugin)
+* [RDF::Mongo](http://rubydoc.info/github/pius/rdf-mongo/master/frames) (plugin)
 * [RDF::DataObjects](http://rdf.rubyforge.org/do/) (plugin)
 * [RDF::Sesame](http://rdf.rubyforge.org/sesame/) (plugin)
 
@@ -159,6 +235,8 @@ Documentation
   * {RDF::Query::Solution}
   * {RDF::Query::Solutions}
   * {RDF::Query::Variable}
+* [SPARQL](http://rubydoc.info/github/gkellogg/sparql/frames) (plugin)
+
 
 ### RDF Vocabularies
 
@@ -267,8 +345,17 @@ License
 This is free and unencumbered public domain software. For more information,
 see <http://unlicense.org/> or the accompanying {file:UNLICENSE} file.
 
-[RDF]:       http://www.w3.org/RDF/
-[YARD]:      http://yardoc.org/
-[YARD-GS]:   http://rubydoc.info/docs/yard/file/docs/GettingStarted.md
-[PDD]:       http://lists.w3.org/Archives/Public/public-rdf-ruby/2010May/0013.html
-[Backports]: http://rubygems.org/gems/backports
+[RDF]:              http://www.w3.org/RDF/
+[YARD]:             http://yardoc.org/
+[YARD-GS]:          http://rubydoc.info/docs/yard/file/docs/GettingStarted.md
+[PDD]:              http://lists.w3.org/Archives/Public/public-rdf-ruby/2010May/0013.html
+[Backports]:        http://rubygems.org/gems/backports
+[JSONLD doc]:       http://rubydoc.info/github/gkellogg/json-ld/frames
+[LinkedData doc]:   http://rubydoc.info/github/datagraph/linkeddata/master/frames
+[Microdata doc]:    http://rubydoc.info/github/gkellogg/rdf-microdata/frames
+[N3 doc]:           http://rubydoc.info/github/gkellogg/rdf-n3/master/frames
+[RDFa doc]:         http://rubydoc.info/github/gkellogg/rdf-rdfa/master/frames
+[RDFXML doc]:       http://rubydoc.info/github/gkellogg/rdf-rdfxml/master/frames
+[Turtle doc]:       http://rubydoc.info/github/gkellogg/rdf-turtle/master/frames
+[SPARQL doc]:       http://rubydoc.info/github/gkellogg/sparql/frames
+[SPARQL 1.0]:       http://www.w3.org/TR/rdf-sparql-query/

data/VERSION

@@ -1 +1 @@
-0.3.3
+0.3.4

data/bin/rdf

@@ -1,22 +1,18 @@
 #!/usr/bin/env ruby
 $:.unshift(File.expand_path(File.join(File.dirname(__FILE__), '..', 'lib')))
+require 'rubygems'
 require 'rdf/cli'
 
 options = RDF::CLI.options do
-  self.banner = "Usage: #{RDF::CLI.basename} [options] command [args...]"
-  self.on('-d', '--debug',   'Enable debug output for troubleshooting.') do
-    $DEBUG = true
-  end
   self.on('-v', '--verbose', 'Enable verbose output. May be given more than once.') do
     $VERBOSE = true
   end
+
   self.on('-V', '--version', 'Display the RDF.rb version and exit.') do
     puts RDF::VERSION; exit
   end
 end
 
-abort options.banner if ARGV.empty?
+abort options.banner if ARGV.empty? && !options.has_key?(:evaluate)
 
-unless RDF::CLI.exec_command(command = ARGV.shift, *ARGV)
-  abort "#{File.basename($0)}: unknown command `#{command}'"
-end
+RDF::CLI.exec_command(command = ARGV.shift, ARGV, options.options)

data/etc/doap.nt

@@ -12,11 +12,11 @@
 <http://bhuga.net/#ben> <http://xmlns.com/foaf/0.1/mbox> <mailto:blavender@gmail.com> .
 <http://bhuga.net/#ben> <http://xmlns.com/foaf/0.1/mbox_sha1sum> "dbf45f4ffbd27b67aa84f02a6a31c144727d10af" .
 <http://bhuga.net/#ben> <http://xmlns.com/foaf/0.1/name> "Ben Lavender" .
-<http://kellogg-assoc.com/#me> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://xmlns.com/foaf/0.1/Person> .
-<http://kellogg-assoc.com/#me> <http://xmlns.com/foaf/0.1/homepage> <http://kellogg-assoc.com/> .
-<http://kellogg-assoc.com/#me> <http://xmlns.com/foaf/0.1/mbox> <mailto:gregg@kellogg-assoc.com> .
-<http://kellogg-assoc.com/#me> <http://xmlns.com/foaf/0.1/mbox_sha1sum> "35bc44e6d0070e5ad50ccbe0d24403c96af2b9bd" .
-<http://kellogg-assoc.com/#me> <http://xmlns.com/foaf/0.1/name> "Gregg Kellogg" .
+<http://greggkellogg.net/foaf.rdf#me> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://xmlns.com/foaf/0.1/Person> .
+<http://greggkellogg.net/foaf.rdf#me> <http://xmlns.com/foaf/0.1/name> "Gregg Kellogg" .
+<http://greggkellogg.net/foaf.rdf#me> <http://xmlns.com/foaf/0.1/mbox> <mailto:gregg@kellogg-assoc.com> .
+<http://greggkellogg.net/foaf.rdf#me> <http://xmlns.com/foaf/0.1/mbox_sha1sum> "35bc44e6d0070e5ad50ccbe0d24403c96af2b9bd" .
+<http://greggkellogg.net/foaf.rdf#me> <http://xmlns.com/foaf/0.1/homepage> <http://greggkellogg.net/> .
 <http://rubygems.org/gems/rdf> <http://purl.org/dc/terms/creator> <http://ar.to/#self> .
 <http://rubygems.org/gems/rdf> <http://usefulinc.com/ns/doap#blog> <http://ar.to/> .
 <http://rubygems.org/gems/rdf> <http://usefulinc.com/ns/doap#blog> <http://blog.datagraph.org/> .
@@ -27,10 +27,10 @@
 <http://rubygems.org/gems/rdf> <http://usefulinc.com/ns/doap#description> "RDF.rb is a pure-Ruby library for working with Resource Description Framework (RDF) data."@en .
 <http://rubygems.org/gems/rdf> <http://usefulinc.com/ns/doap#developer> <http://ar.to/#self> .
 <http://rubygems.org/gems/rdf> <http://usefulinc.com/ns/doap#developer> <http://bhuga.net/#ben> .
-<http://rubygems.org/gems/rdf> <http://usefulinc.com/ns/doap#developer> <http://kellogg-assoc.com/#me> .
+<http://rubygems.org/gems/rdf> <http://usefulinc.com/ns/doap#developer> <http://greggkellogg.net/foaf.rdf#me> .
 <http://rubygems.org/gems/rdf> <http://usefulinc.com/ns/doap#documenter> <http://ar.to/#self> .
 <http://rubygems.org/gems/rdf> <http://usefulinc.com/ns/doap#documenter> <http://bhuga.net/#ben> .
-<http://rubygems.org/gems/rdf> <http://usefulinc.com/ns/doap#documenter> <http://kellogg-assoc.com/#me> .
+<http://rubygems.org/gems/rdf> <http://usefulinc.com/ns/doap#documenter> <http://greggkellogg.net/foaf.rdf#me> .
 <http://rubygems.org/gems/rdf> <http://usefulinc.com/ns/doap#download-page> <http://rubyforge.org/projects/rdf/> .
 <http://rubygems.org/gems/rdf> <http://usefulinc.com/ns/doap#helper> _:genid1 .
 <http://rubygems.org/gems/rdf> <http://usefulinc.com/ns/doap#helper> _:genid2 .
@@ -45,7 +45,7 @@
 <http://rubygems.org/gems/rdf> <http://usefulinc.com/ns/doap#license> <http://creativecommons.org/licenses/publicdomain/> .
 <http://rubygems.org/gems/rdf> <http://usefulinc.com/ns/doap#maintainer> <http://ar.to/#self> .
 <http://rubygems.org/gems/rdf> <http://usefulinc.com/ns/doap#maintainer> <http://bhuga.net/#ben> .
-<http://rubygems.org/gems/rdf> <http://usefulinc.com/ns/doap#maintainer> <http://kellogg-assoc.com/#me> .
+<http://rubygems.org/gems/rdf> <http://usefulinc.com/ns/doap#maintainer> <http://greggkellogg.net/foaf.rdf#me> .
 <http://rubygems.org/gems/rdf> <http://usefulinc.com/ns/doap#name> "RDF.rb" .
 <http://rubygems.org/gems/rdf> <http://usefulinc.com/ns/doap#platform> "Ruby" .
 <http://rubygems.org/gems/rdf> <http://usefulinc.com/ns/doap#shortdesc> "A Ruby library for working with Resource Description Framework (RDF) data."@en .

data/lib/rdf.rb

@@ -34,6 +34,7 @@ module RDF
   autoload :Mutable,     'rdf/mixin/mutable'
   autoload :Queryable,   'rdf/mixin/queryable'
   autoload :Readable,    'rdf/mixin/readable'
+  autoload :TypeCheck,   'rdf/mixin/type_check'
   autoload :Writable,    'rdf/mixin/writable'
 
   # RDF objects

data/lib/rdf/cli.rb

@@ -1,8 +1,75 @@
 require 'rdf'
+require 'rdf/ntriples'
+require 'rdf/nquads'
 require 'optparse'
+begin
+  gem 'linkeddata'
+  require 'linkeddata'
+rescue LoadError
+  # Silently load without linkeddata
+end
+
+class OptionParser
+  def options; @options || {}; end
+  def options=(value); @options = value; end
+end
 
 module RDF
   class CLI
+    
+    COMMANDS = {
+      "count"       => lambda do |argv, opts|
+        start = Time.new
+        count = 0
+        self.parse(argv, opts) do |reader|
+          reader.each_statement do |statement|
+            count += 1
+          end
+        end
+        secs = Time.new - start
+        $stdout.puts "Parsed #{count} statements with #{@readers.join(', ')} in #{secs} seconds @ #{count/secs} statements/second."
+      end,
+      "lenghts"     => lambda do |argv, opts|
+        self.parse(argv, opts) do |reader|
+          reader.each_statement do |statement|
+            $stdout.puts statement.to_s.size
+          end
+        end
+      end,
+      "objects"     => lambda do |argv, opts|
+        self.parse(argv, opts) do |reader|
+          reader.each_statement do |statement|
+            $stdout.puts statement.object.to_ntriples
+          end
+        end
+      end,
+      "predicates"   => lambda do |argv, opts|
+        self.parse(argv, opts) do |reader|
+          reader.each_statement do |statement|
+            $stdout.puts statement.predicate.to_ntriples
+          end
+        end
+      end,
+      "serialize" => lambda do |argv, opts|
+        writer_class = RDF::Writer.for(opts[:output_format]) || RDF::NTriples::Writer
+        out = opts[:output] || $stdout
+        opts = opts.merge(:prefixes => {})
+        writer_opts = opts.merge(:standard_prefixes => true)
+        self.parse(argv, opts) do |reader|
+          writer_class.new(out, writer_opts) do |writer|
+            writer << reader
+          end
+        end
+      end,
+      "subjects"   => lambda do |argv, opts|
+        self.parse(argv, opts) do |reader|
+          reader.each_statement do |statement|
+            $stdout.puts statement.subject.to_ntriples
+          end
+        end
+      end
+    }
+    
     ##
     # @return [String]
     def self.basename() File.basename($0) end
@@ -13,20 +80,70 @@ module RDF
     # @return [OptionParser]
     def self.options(&block)
       options = OptionParser.new
+      opts = options.options = {
+        :base_uri       => nil,
+        :canonicalize   => false,
+        :debug          => false,
+        :evaluate       => nil,
+        :format         => nil,
+        :output         => $stdout,
+        :output_format  => :ntriples,
+        :validate       => false,
+      }
 
+      # Command-specific options
       if block_given?
         case block.arity
           when 1 then block.call(options)
           else options.instance_eval(&block)
         end
       end
+      options.banner ||= "Usage: #{self.basename} [options] command [args...]"
+
+      options.on('--canonicalize', 'Canonicalize input.') do
+        opts[:canonicalize] = true
+      end
+
+      options.on('-d', '--debug',   'Enable debug output for troubleshooting.') do
+        opts[:debug] = $DEBUG = true
+      end
+
+      options.on("-e", "--evaluate STRING", "Evaluate argument as RDF input, if no files are specified") do |arg|
+        opts[:evaluate] = arg
+      end
+
+      options.on("--input-format FORMAT", "Format of input file, uses heuristic if not specified") do |arg|
+        opts[:format] = arg.downcase.to_sym
+      end
+
+      options.on("-o", "--output FILE", "File to write output, defaults to STDOUT") do |arg|
+        opts[:output] = File.open(arg, "w")
+      end
+
+      options.on("--output-format FORMAT", "Format of output file, defaults to NTriples") do |arg|
+        opts[:output_format] = arg.downcase.to_sym
+      end
+
+      options.on('--uri URI', 'Base URI of input file, defaults to the filename.') do |arg|
+        opts[:base_uri] = arg
+      end
+
+      options.on('--validate', 'Validate input file.') do
+        opts[:validate] = true
+      end
 
+      options.on_tail("-h", "--help", "Show this message") do
+        $stdout.puts options
+        $stdout.puts "Available commands:\n\t#{self.commands.join("\n\t")}"
+        exit
+      end
+      
       begin
         options.parse!
       rescue OptionParser::InvalidOption => e
         abort e
       end
-
+      
       options
     end
 
@@ -34,35 +151,45 @@ module RDF
     # @param  [String] command
     # @param  [Array<String>] args
     # @return [Boolean]
-    def self.exec_command(command, *args)
-      if binary = RDF::CLI.find_binary(command)
-        exec(binary, *args)
-      else
-        false
+    def self.exec_command(command, args, options = {})
+      unless COMMANDS.has_key?(command)
+        abort "#{File.basename($0)}: unknown command `#{command}'"
       end
+      
+      COMMANDS[command].call(args, options)
     end
-
+    
     ##
-    # @param  [String] command
-    # @return [String, nil]
-    def self.find_binary(command)
-      binary = File.join(File.dirname(__FILE__), '..', '..', 'bin', 'rdf-' + command)
-      File.exists?(binary) ? binary : nil
+    # @return [Array<String>] list of executable commands
+    def self.commands
+      COMMANDS.keys
     end
 
     ##
+    # Parse each file, STDIN or specified string in options[:evaluate] yielding
+    # a reader
+    #
     # @param  [Array<String>] files
-    # @yield  [statement]
-    # @yieldparam [Statement]
+    # @yield  [reader]
+    # @yieldparam [RDF::Reader]
     # @return [nil]
-    def self.each_statement(*files, &block)
-      files.each do |file|
-        RDF::NTriples::Reader.open(file) do |reader|
-          reader.each_statement(&block)
+    def self.parse(files, options = {}, &block)      
+      if files.empty?
+        # If files are empty, either use options[:execute]
+        input = options[:evaluate] ? StringIO.new(options[:evaluate]) : STDIN
+        RDF::Reader.for(options[:format] || :ntriples).new(input, options) do |reader|
+          yield(reader)
+        end
+      else
+        files.each do |file|
+          RDF::Reader.open(file, options) do |reader|
+            (@readers ||= []) << reader.class.to_s
+            yield(reader)
+          end
         end
       end
     end
-
+    
     ##
     # @param  [String] msg
     # @return [void]