biointerchange 0.2.2 → 1.0.0

data/Gemfile

@@ -6,6 +6,7 @@ gem "rdf", ">= 0.3.4.1"
 gem "json", ">= 1.6.4"
 gem "getopt", ">= 1.4.1"
 gem "addressable", ">= 2.3.2"
+gem "bio", ">= 1.4.2"
 
 # Add dependencies to develop your gem here.
 # Include everything needed to run rake, tests, features, etc.

data/README.md

@@ -9,6 +9,7 @@ Supported input file formats (see examples directory):
 
 * [GFF3](http://www.sequenceontology.org/resources/gff3.html)
 * [GVF](http://www.sequenceontology.org/resources/gvf.html)
+* [Newick](http://evolution.genetics.washington.edu/phylip/newicktree.html)
 * [Pubannos JSON](http://pubannotation.dbcls.jp/)
 * [PDFx XML](http://pdfx.cs.man.ac.uk/)
 
@@ -18,6 +19,8 @@ Supported RDF output formats:
 
 Ontologies used in the RDF output:
 
+* [Comparative Data Analysis Ontology](http://sourceforge.net/apps/mediawiki/cdao/index.php?title=Main_Page) (CDAO)
+* [Friend of a Friend](http://xmlns.com/foaf/spec) (FOAF)
 * [Generic Feature Format Version 3 Ontology](http://www.biointerchange.org/ontologies.html) (GFF3O)
 * [Genome Variation Format Version 1 Ontology](http://www.biointerchange.org/ontologies.html) (GVF1O)
 * [Semanticscience Integrated Ontology](http://code.google.com/p/semanticscience/wiki/SIO) (SIO)
@@ -48,14 +51,17 @@ BioInterchange's command-line tool `biointerchange` can be installed as a comman
 
 Examples:
 
-    biointerchange --input dbcls.catanns.json --rdf rdf.bh12.sio --file examples/pubannotation.10096561.json --name 'Peter Smith' --name_id 'peter.smith@example.com'
-    biointerchange --input uk.ac.man.pdfx --rdf rdf.bh12.sio --file examples/gb-2007-8-3-R40.xml --name 'Peter Smith' --name_id 'peter.smith@example.com'
+    biointerchange --input biointerchange.gvf --rdf rdf.biointerchange.gvf --batchsize 100 --file examples/estd176_Banerjee_et_al_2011.2012-11-29.NCBI36.gvf
+    biointerchange --input dbcls.catanns.json --rdf rdf.bh12.sio --file examples/pubannotation.10096561.json --annotate_name 'Peter Smith' --annotate_name_id 'peter.smith@example.com'
+    biointerchange --input uk.ac.man.pdfx --rdf rdf.bh12.sio --file examples/gb-2007-8-3-R40.xml --annotate_name 'Peter Smith' --annotate_name_id 'peter.smith@example.com'
+    biointerchange --input phylotastic.newick --rdf rdf.phylotastic.newick --file examples/tree2.new --annotate_date '1 June 2006'
 
 Input formats:
 
 *  `biointerchange.gff3`
 *  `biointerchange.gvf`
 *  `dbcls.catanns.json`
+*  `phylotastic.newick`
 *  `uk.ac.man.pdfx`
 
 Output formats:
@@ -63,6 +69,7 @@ Output formats:
 *  `rdf.biointerchange.gff3`
 *  `rdf.biointerchange.gvf`
 *  `rdf.bh12.sio`
+*  `rdf.phylotastic.newick`
 
 #### Using a Triple Store
 
@@ -79,7 +86,7 @@ RDF data produced by BioInterchange can be directly loaded into a triple store.
     testrepo> load <path-to-your-rdf-data> .
     testrepo> sparql select * where { ?s ?p ?o } .
 
-To list all `seqid` entries from a GFF3/GVF-file conversion in the store, the following SPARQL query can be used:
+To list all `seqid` entries from a GVF-file conversion in the store, the following SPARQL query can be used:
 
     testrepo> sparql select * where { ?s <http://www.biointerchange.org/gvf1o#GVF1_0004> ?o } .
 
@@ -96,6 +103,7 @@ Another approach is to load the data and its related GFF3O/GVF1O ontology into [
 
 The following list provides information on the origin of the example-data files in the `examples` directory:
 
+*  `bininda_emonds_mammals.new`: Newick formatted Bininda-Emonds mammals tree (see [The delayed rise of present-day mammals](http://www.ncbi.nlm.nih.gov/pubmed/17392779)). Downloaded from [https://github.com/bendmorris/rdf-treestore/blob/master/trees/bininda_emonds_mammals.new](https://github.com/bendmorris/rdf-treestore/blob/master/trees/bininda_emonds_mammals.new)
 *  `BovineGenomeChrX.gff3.gz`: Gzipped GFF3 file describing a Bos taurus chromosome X. Downloaded from [http://bovinegenome.org/?q=download_chromosome_gff3](http://bovinegenome.org/?q=download_chromosome_gff3)
 *  `chromosome_BF.gff`: GFF3 file of floating contigs from the Baylor Sequencing Centre. Downloaded from [http://dictybase.org/Downloads](http://dictybase.org/Downloads)
 *  `estd176_Banerjee_et_al_2011.2012-11-29.NCBI36.gvf`: GVF file of EBI's [DGVa](http://www.ebi.ac.uk/dgva/database-genomic-variants-archive). Downloaded from [ftp://ftp.ebi.ac.uk/pub/databases/dgva/estd176_Banerjee_et_al_2011/gvf/estd176_Banerjee_et_al_2011.2012-11-29.NCBI36.gvf](ftp://ftp.ebi.ac.uk/pub/databases/dgva/estd176_Banerjee_et_al_2011/gvf/estd176_Banerjee_et_al_2011.2012-11-29.NCBI36.gvf)
@@ -106,13 +114,220 @@ The following list provides information on the origin of the example-data files
 
 #### Ruby
 
-The Ruby gem is under active development, so the following may or may not work out of the box.
+BioInterchange is available as Ruby gem that can be installed as follows:
 
-    gem install biointerchange
+    sudo gem install biointerchange
+
+The API provides vocabulary wrappers to ontologies that are used within the BioInterchange framework as well as access to RDFization implementations. 
+
+##### Using Vocabulary Wrappers
 
-To use BioInterchange in your Ruby projects, include the following line in your code:
+Ruby classes are provided for the ontologies that is used for serializing RDF. Each ontology is represented by its own Ruby class. The classes provide access to the ontology terms and additional methods for resolving OWL classes, datatype properties and object properties.
 
+Usage example (see also [vocabulary.rb](https://github.com/BioInterchange/BioInterchange/blob/master/examples/vocabulary.rb)):
+
+    require 'rubygems'
     require 'biointerchange'
+    
+    include BioInterchange
+    
+    def print_resource(resource)
+      puts "    #{resource}"
+      puts "        Ontology class:             #{GFF3O.is_class?(resource)}"
+      puts "        Ontology object property:   #{GFF3O.is_object_property?(resource)}"
+      puts "        Ontology datatype property: #{GFF3O.is_datatype_property?(resource)}"
+    end
+    
+    # Get the URI of an ontology term by label:
+    puts "'seqid' property:"
+    print_resource(GFF3O.seqid())
+    
+    # Ambiguous labels will return an array of URIs:
+    # "start" can refer to a sub-property of "feature_properties" or "target_properties"
+    puts "'start' properties:"
+    GFF3O.start().each { |start_synonym|
+      print_resource(start_synonym)
+    }
+    # "feature_properties" can be either a datatype or object property
+    puts "'feature_properties' properties:"
+    GFF3O.feature_properties().each { |feature_properties_synonym|
+      print_resource(feature_properties_synonym)
+    }
+    
+    # Use build-in method "is_datatype_property" to resolve ambiguity:
+    # (Note: there is exactly one item in the result set, so the selection of the first item is acceptable.)
+    feature_properties = GFF3O.feature_properties().select { |uri| GFF3O.is_datatype_property?(uri) }
+    puts "'feature_properties' properties, which are a datatype property:"
+    feature_properties.each { |feature_property|
+      print_resource(feature_property)
+    }
+    
+    # Use build-in method "with_parent" to pick properties based on their context:
+    puts "'start' property with parent datatype property 'feature_properties':"
+    GFF3O.with_parent(GFF3O.start(), feature_properties[0]).each { |feature_property|
+      print_resource(feature_property)
+    }
+
+With the BioInterchange gem installed, the example can be executed on the command line via:
+
+    git clone git://github.com/BioInterchange/BioInterchange.git
+    cd BioInterchange
+    git checkout v1.0.0
+    ruby examples/vocabulary.rb
+
+##### RDFization Framework
+
+Usage example (see also [rdfization.rb](https://github.com/BioInterchange/BioInterchange/blob/master/examples/rdfization.rb)):
+
+    require 'rubygems'
+    require 'biointerchange'
+    
+    include BioInterchange::Phylogenetics
+    
+    # Create a reader that reads phylogenetic trees in Newick format:
+    reader = NewickReader.new()
+    
+    # Create a model from a single example tree:
+    # (Note: the `deserialize` method also takes streams as parameter -- not just strings.)
+    model = reader.deserialize('((B:0.2,(C:0.3,D:0.4)E:0.5)F:0.1)A;')
+    
+    # Serialize the model as RDF N-Triples to STDOUT:
+    CDAORDFWriter.new(STDOUT).serialize(model)
+
+##### Implementing New Readers, Models and Writers
+
+New readers, models and writers are best adopted from or build upon the existing implementations. The phylogenetic trinity of Newick file format reader, [BioRuby](http://bioruby.org) based tree model, and [CDAO](http://sourceforge.net/apps/mediawiki/cdao/index.php?title=Main_Page) RDF writer is used here as a guidline due to its simplicity.
+
+###### Reader: Creating an Object Model
+
+The quintessential Newick tree reader is depicted below. Its class is placed in a Ruby module that encapsulates all phylogenetic related source code. The `NewickReader` class inherits from the BioInterchange framework class `Reader` that provides method stubs which need to be overwritten. Using the central registry `BioInterchange::Registry`, the reader informs the framework of its: unique identifier (`phylotastic.newick`), Ruby class (`NewickReader`), command line parameters that it accepts (`date`, which becomes `--annotate_date`), whether the reader can operate without reading the complete input all at once (`true`), a descriptive name of the reader (`Newick Tree [...]`), and an array with descriptions for each parameter stated above.
+
+Deserialization of Newick trees is done using the `deserialize` method, which must take both strings and input streams as valid arguments. If this contraint is not satisfied, then an `ImplementationReaderError` is thrown that is caught by the framework and handled appropriately.
+
+Finally, the `postponed?` method keeps track of deferred input processing. If the batch size was reached and the model was passed on for serialization to a writer, then this method will have to return `true`.
+
+    require 'bio'
+    require 'date'
+    
+    module BioInterchange::Phylogenetics
+    
+    class NewickReader < BioInterchange::Reader
+    
+      # Register reader:
+      BioInterchange::Registry.register_reader(
+        'phylotastic.newick',
+        NewickReader,
+        [ 'date' ],
+        true,
+        'Newick Tree File Format reader',
+        [
+          [ 'date <date>', 'date when the Newick file was created (optional)' ]
+        ]
+      )
+    
+      # Creates a new instance of a Newick file format reader.
+      #
+      # The reader supports batch processing.
+      #
+      # +date+:: Optional date of when the Newick file was produced, annotated, etc.
+      # +batch_size+:: Optional integer that determines that number of features that
+      # should be processed in one go.
+      def initialize(date = nil, batch_size = nil)
+        @date = date
+        @batch_size = batch_size
+      end
+    
+      # Reads a Newick file from the input stream and returns an associated model.
+      #
+      # If this method is called when +postponed?+ returns true, then the reading will
+      # continue from where it has been interrupted beforehand.
+      #
+      # +inputstream+:: an instance of class IO or String that holds the contents of a Newick file
+      def deserialize(inputstream)
+        if inputstream.kind_of?(IO)
+          create_model(inputstream)
+        elsif inputstream.kind_of?(String) then
+          create_model(StringIO.new(inputstream))
+        else
+          raise BioInterchange::Exceptions::ImplementationReaderError, 'The provided input stream needs to be either of type IO or String.'
+        end
+      end
+    
+      # Returns true if the reading of the input was postponed due to a full batch.
+      def postponed?
+        @postponed
+      end
+    
+    protected
+    
+    # ...concrete implementation omitted.
+
+###### Tree Model
+
+A model is created by a reader and it is subsequently consumed by a writer. The phylogenetic tree model inherits `BioInterchange::Model` which defines the `prune` method. If batch operation is in place, i.e. the input is not completely read into memory, then the `prune` method will be called to instruct the model to drop all information that has not to be kept in memory anymore. In a sense, this can be seen as a form of garbage collection, where data that has been serialized is purged from memory.
+
+    module BioInterchange::Phylogenetics
+    
+    # A phylogenetic tree set that can contain multiple phylogenetic trees.
+    class TreeSet < BioInterchange::Model
+    
+      # Create a new instance of a tree set. A tree set can contain multiple phylogenetic trees.
+      def initialize
+        # Trees are stored as the keys of a hash map to increase performance:
+        @set = {}
+      end
+    
+      # ...omitted internal data structure handling.
+    
+      # Removes all features from the set, but keeps additional data (e.g., the date).
+      def prune
+        @set.clear
+      end
+    
+    end
+    
+    end
+
+###### Writer: From Object Model to RDF
+
+The writer takes an object model and serializes it via the `BioInterchange::Writer` derived `serialize` method. A writer uses `BioInterchange::Registry` to make itself known to the BioInterchange framework, where it signs up using the following arguments: a unique identifier (`rdf.phylotastic.newick`), its implementing class (`CDAORDFWriter`), a list of readers that it is compatible with (`phylotastic.newick`), whether the writer supports batch processing where only parts of the input need to be kept in memory (`true`), and a descriptive name for the writer.
+
+    require 'rdf'
+    require 'rdf/ntriples'
+    
+    module BioInterchange::Phylogenetics
+    
+    # Serialized phylogenetic tree models based on BioRuby's phylogenetic tree implementation.
+    class CDAORDFWriter < BioInterchange::Writer
+    
+      # Register writers:
+      BioInterchange::Registry.register_writer(
+        'rdf.phylotastic.newick',
+        CDAORDFWriter,
+        [ 'phylotastic.newick' ],
+        true,
+        'Comparative Data Analysis Ontology (CDAO) based RDFization'
+      )
+    
+      # Creates a new instance of a CDAORDFWriter that will use the provided output stream to serialize RDF.
+      #
+      # +ostream+:: instance of an IO class or derivative that is used for RDF serialization
+      def initialize(ostream)
+        @ostream = ostream
+      end
+    
+      # Serialize a model as RDF.
+      #
+      # +model+:: a generic representation of input data that is an instance of BioInterchange::Phylogenetics::TreeSet
+      def serialize(model)
+        model.contents.each { |tree|
+          serialize_model(model, tree)
+        }
+      end
+    
+    protected
+    
+    # ...omitted actual serialization implementation.
 
 #### Python
 
@@ -122,28 +337,54 @@ BioInterchange available.
 To install the BioInterchange egg, run:
 
     sudo easy_install rdflib
-    sudo easy_install http://www.biointerchange.org/eggs/biointerchange-0.2.2-py2.7.egg
+    sudo easy_install http://www.biointerchange.org/eggs/biointerchange-1.0.0-py2.7.egg
 
 Usage examples:
 
     import biointerchange
     from biointerchange import *
+    from rdflib.namespace import Namespace
+    
+    def print_resource(resource):
+        print "    " + resource
+        print "        Ontology class:             " + str(GFF3O.is_class(resource))
+        print "        Ontology object property:   " + str(GFF3O.is_object_property(resource))
+        print "        Ontology datatype property: " + str(GFF3O.is_datatype_property(resource))
     
     # Get the URI of an ontology term by label:
-    GFF3O.seqid()
+    print "'seqid' property:"
+    print_resource(GFF3O.seqid())
     
     # Ambiguous labels will return an array of URIs:
     # "start" can refer to a sub-property of "feature_properties" or "target_properties"
-    GFF3O.start()
+    print "'start' properties:"
+    for start_synonym in GFF3O.start():
+        print_resource(start_synonym)
+    
     # "feature_properties" can be either a datatype or object property
-    GFF3O.feature_properties()
+    print "'feature_properties' properties:"
+    for feature_properties_synonym in GFF3O.feature_properties():
+        print_resource(feature_properties_synonym)
     
     # Use build-in method "is_datatype_property" to resolve ambiguity:
     # (Note: there is exactly one item in the result set, so the selection of the first item is acceptable.)
-    feature_properties = filter(lambda uri: GFF3O.is_datatype_property(uri), GFF3O.feature_properties())[0]
+    feature_properties = filter(lambda uri: GFF3O.is_datatype_property(uri), GFF3O.feature_properties())
+    print "'feature_properties' properties, which are a datatype property:"
+    for feature_property in feature_properties:
+        print_resource(feature_property)
     
     # Use build-in method "with_parent" to pick properties based on their context:
-    GFF3O.with_parent(GFF3O.start(), feature_properties)
+    print "'start' property with parent datatype property 'feature_properties':"
+    for feature_property in GFF3O.with_parent(GFF3O.start(), feature_properties[0]):
+        print_resource(feature_property)
+
+The example can be executed on the command line via:
+
+    git clone git://github.com/BioInterchange/BioInterchange.git
+    cd BioInterchange
+    git checkout v1.0.0
+    cd supplemental/python
+    python example.py
 
 #### Java
 
@@ -163,15 +404,10 @@ To use the BioInterchange artifact, set-up add the following to your Maven POM f
       <dependency>
         <groupId>org.biointerchange</groupId>
         <artifactId>vocabularies</artifactId>
-        <version>0.2.2</version>
+        <version>1.0.0</version>
       </dependency>
     </dependencies>
 
-Current vocabularies:
-
-*  Generic Feature Format Version 3 Ontology (GFF3O)
-*  Genome Variation Format Version 1 Ontology (GVF1O)
-
 Usage examples of accessing GFF3O's vocabulary:
 
     package org.biointerchange;
@@ -233,6 +469,14 @@ Usage examples of accessing GFF3O's vocabulary:
         }
     }
 
+Another example that uses SIO instead of GFF3O is provided as [AppSIO.java](https://github.com/BioInterchange/BioInterchange/blob/master/supplemental/java/biointerchange/src/main/java/org/biointerchange/AppSIO.java).
+
+The examples can be executed through Maven:
+
+    cd supplemental/java/biointerchange
+    mvn exec:java -Dexec.mainClass="org.biointerchange.App"
+    mvn exec:java -Dexec.mainClass="org.biointerchange.AppSIO"
+
 ### RESTful Web-Service
 
 A RESTful web-service is available via the URI: [http://www.biointerchange.org/service/rdfizer.biocgi](http://www.biointerchange.org/service/rdfizer.biocgi)
@@ -251,11 +495,13 @@ RDFization parameters and data are send as a single HTTP POST requests containin
    *  `biointerchange.gff3`: [Generic Feature Format Version 3](http://www.sequenceontology.org/resources/gff3.html)
    *  `biointerchange.gvf`: [Genome Variation Format](http://www.sequenceontology.org/resources/gvf.html)
    *  `dbcls.catanns.json`: [PubAnnotation categorical annotations](http://pubannotation.dbcls.jp) JSON
+   *  `phylotastic.newick`: [Newick](http://evolution.genetics.washington.edu/phylip/newicktree.html)
    *  `uk.ac.man.pdfx`: [PDFx](http://pdfx.cs.man.ac.uk) XML
 *  `OUTPUT_METHOD`: determines the RDFization method that should be used, output will always be RDF N-Triples; available output formats are
    *  `rdf.biointerchange.gff3`: RDFization of `biointerchange.gff3`
    *  `rdf.biointerchange.gvf`: RDFization of `biointerchange.gvf`
    *  `rdf.bh12.sio`: RDFization of `dbcls.catanns.json` or `uk.ac.man.pdfx`
+   *  `rdf.phylotastic.newick`: RDFization of `phylotastic.newick`
 *  `URL_ENCODED_DATA`: data for RDFization as [URL encoded](http://en.wikipedia.org/wiki/Percent-encoding) string
 
 #### Example
@@ -310,10 +556,13 @@ The last step, `bundle`, will install gem dependencies of BioInterchange automat
 
 ### Building Vocabulary Classes
 
-Building a new version of the Ruby vocabulary classes for FALDO, GFF3O, GVF1O, SIO, SOFA (requires that the OBO files are saves as RDF/XML using [Protege](http://protege.stanford.edu); Apache [Jena](http://jena.apache.org)'s `rdfcat` tool is required to reformat RDF Turtle as RDF/XML):
+Building a new version of the Ruby vocabulary classes for CDAO, FALDO, GFF3O, GVF1O, SIO, SOFA (requires that the OBO files are saves as RDF/XML using [Protege](http://protege.stanford.edu); Apache [Jena](http://jena.apache.org)'s `rdfcat` tool is required to reformat RDF Turtle as RDF/XML):
 
     sudo gem install rdf
     sudo gem install rdf-rdfxml
+    echo -e "require 'rdf'\nmodule BioInterchange\n" > lib/biointerchange/cdao.rb
+    ruby generators/rdfxml.rb <path-to-rdf/xml-version-of-cdao> CDAO >> lib/biointerchange/cdao.rb
+    echo -e "\nend" >> lib/biointerchange/cdao.rb
     echo -e "require 'rdf'\nmodule BioInterchange\n" > lib/biointerchange/faldo.rb
     rdfcat -ttl <path-to-turtle-version-of-faldo> > faldo.xml.tmp
     ruby generators/rdfxml.rb faldo.xml.tmp FALDO >> lib/biointerchange/faldo.rb
@@ -438,6 +687,7 @@ In alphabetical order of the last name:
 * [Kevin B. Cohen](http://compbio.ucdenver.edu/Hunter_lab/Cohen/index.shtml)
 * [Geraint Duck](http://www.cs.man.ac.uk/~duckg)
 * [Michel Dumontier](http://dumontierlab.com)
+* [Begum Durgahee](http://utah.academia.edu/BDurgahee)
 * [Jin-Dong Kim](http://www.bioontology.org/Jin-Dong_Kim)
 
 Cite

data/VERSION

@@ -1 +1 @@
-0.2.2
+1.0.0