qa 1.0.0 → 1.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: aaf2ea321ec50c2790d269dee43856176fcc22e1
-  data.tar.gz: 18d617298ba9675024a9003c2f4183040cc535c8
+  metadata.gz: 571cac6a8c4ec9ddf8a61a7d54ea857481af0a1c
+  data.tar.gz: e33cead405236f73a1dc443ee11c48a132f64b30
 SHA512:
-  metadata.gz: cec86fa96e3c2c767eeb8b7a7866a3269d975659bc872f5acacde03448385584351420b3c4c6f8f4a6275b8b9b7a8cf9259b027e8c1212a286877ceabcdf156d
-  data.tar.gz: 3a1d18ec438ac477ab6b22a4f08db1b14dc83d8b348a36d754530f1ba5b799747b0a5ffea9b3f8864fd131289e917d76de45b2aa7685f96e350dbee6aab863ee
+  metadata.gz: ccd4534d37546bbfc12514de7ba0058668fc1f380d6eab4a9889c45d34ea035df73dc6e2977a14ae273d215c24ff02b6228e82593aa8c51f3a5bb333e6112f28
+  data.tar.gz: af6074497e2bfeafdf3f9c9fd71d13e45c7917889084d241174059d5e3062697589286edb4f6260603414ec0d22d771683695d91e26a8b390d03bfbbd21dbce5

data/README.md

@@ -1,10 +1,38 @@
 # Questioning Authority
 
-[![Build Status](https://travis-ci.org/projecthydra-labs/questioning_authority.png?branch=master)](https://travis-ci.org/projecthydra-labs/questioning_authority) [![Gem Version](https://badge.fury.io/rb/qa.png)](http://badge.fury.io/rb/qa)
-[![Coverage Status](https://coveralls.io/repos/github/projecthydra-labs/questioning_authority/badge.svg?branch=master)](https://coveralls.io/github/projecthydra-labs/questioning_authority?branch=master)
+[![Build Status](https://travis-ci.org/projecthydra/questioning_authority.png?branch=master)](https://travis-ci.org/projecthydra/questioning_authority) [![Gem Version](https://badge.fury.io/rb/qa.png)](http://badge.fury.io/rb/qa)
+[![Coverage Status](https://coveralls.io/repos/github/projecthydra/questioning_authority/badge.svg?branch=master)](https://coveralls.io/github/projecthydra/questioning_authority?branch=master)
 
 You should question your authorities.
 
+----
+## Table of Contents
+
+  * [What does this do?](#what-does-this-do)
+  * [How does it work?](#how-does-it-work)
+    * [Sub-Authorities](#sub-authorities)
+  * [How do I use this?](#how-do-i-use-this)
+    * [Examples](#examples)
+    * [JSON Results](#json-results)
+  * [Authority Sources information](#authority-sources-information)
+    * [FAST](#fast)
+    * [Geonames](#geonames)
+    * [Adding your own authorities](#adding-your-own-authorities)
+    * [Local Sub-Authorities](#local-sub-authorities)
+      * [In YAML files](#in-yaml-files)
+      * [In database tables](#in-database-tables)
+    * [Medical Subject Headings (Mesh)](#medical-subject-headings-mesh)
+    * [Linked Open Data (LOD) Authorities](#linked-open-data-lod-authorities)
+      * [Configuring a LOD Authority](#configuring-a-lod-authority)
+      * [Query](#query)
+      * [Find term](#find-term)
+      * [Add javascript to support autocomplete](#add-javascript-to-support-autocomplete)
+  * [Developer Notes](#developer-notes)
+    * [Compatibility](#compatibility)
+  * [Help](#help)
+
+----
+
 ## What does this do?
 
 Provides a set of uniform RESTful routes to query any controlled vocabulary or set of authority terms.
@@ -333,6 +361,260 @@ This may take a few minutes to finish.
 
 **Note:** Updating the tables with new terms is currently not supported.
 
+### Linked Open Data (LOD) Authorities
+
+You will need to add gems that process the type of linked data returned for the authorities you use.
+
+To cover all possible formats, include the [ruby-rdf/linkeddata](https://github.com/ruby-rdf/linkeddata) gem.
+
+```
+gem 'linkeddata'
+```
+
+This gem is included in QA for development and testing of QA, but is not automatically included in the released gem.
+Additionally, it is unlikely that you will need all the formats included by that gem.  You may want to select only those
+gems that are for the formats you need supported.
+
+See all gems in [linkeddata.gemspec](https://github.com/ruby-rdf/linkeddata/blob/develop/linkeddata.gemspec).
+
+For example, if you know the authorites you are working with support rdf-xml, you can include the following gem instead of linkeddata.
+
+```
+gem 'rdf-rdfxml'
+```
+
+#### Configuring a LOD Authority
+
+Access to LOD authorities can be configured.  Currently, a configuration exists in QA for OCLC Fast Linked Data, Library of
+Congress (terms only), and Agrovoc.  Look for configuration files in
+[/config/authorities/linked_data](https://github.com/projecthydra/questioning_authority/tree/master/config/authorities/linked_data).
+
+Example configuration...
+
+```json
+{
+  "term": {
+    "url": {
+      "@context": "http://www.w3.org/ns/hydra/context.jsonld",
+      "@type":    "IriTemplate",
+      "template": "http://id.worldcat.org/fast/{?term_id}/rdf.xml",
+      "variableRepresentation": "BasicRepresentation",
+      "mapping": [
+        {
+          "@type":    "IriTemplateMapping",
+          "variable": "term_id",
+          "property": "hydra:freetextQuery",
+          "required": true
+        }
+      ]
+    },
+    "qa_replacement_patterns": {
+      "term_id": "term_id"
+    },
+    "language": ["en","fr"]
+    "term_id": "ID",
+    "results": {
+      "id_predicate":       "http://purl.org/dc/terms/identifier",
+      "label_predicate":    "http://www.w3.org/2004/02/skos/core#prefLabel",
+      "altlabel_predicate": "http://www.w3.org/2004/02/skos/core#altLabel",
+      "sameas_predicate":   "http://schema.org/sameAs"
+    }
+  },
+  "search": {
+    "url": {
+      "@context": "http://www.w3.org/ns/hydra/context.jsonld",
+      "@type": "IriTemplate",
+      "template": "http://experimental.worldcat.org/fast/search?query={?subauth}+all+%22{?query}%22&sortKeys=usage&maximumRecords={?maximumRecords}",
+      "variableRepresentation": "BasicRepresentation",
+      "mapping": [
+        {
+          "@type": "IriTemplateMapping",
+          "variable": "query",
+          "property": "hydra:freetextQuery",
+          "required": true
+        },
+        {
+          "@type": "IriTemplateMapping",
+          "variable": "subauth",
+          "property": "hydra:freetextQuery",
+          "required": false,
+          "default": "cql.any"
+        },
+        {
+          "@type": "IriTemplateMapping",
+          "variable": "maximumRecords",
+          "property": "hydra:freetextQuery",
+          "required": false,
+          "default": "20"
+        }
+      ]
+    },
+    "qa_replacement_patterns": {
+      "query":   "query",
+      "subauth": "subauth"
+    },
+    "language": ["en"]
+    "results": {
+      "id_predicate":       "http://purl.org/dc/terms/identifier",
+      "label_predicate":    "http://www.w3.org/2004/02/skos/core#prefLabel",
+      "sort_predicate":     "http://www.w3.org/2004/02/skos/core#prefLabel"
+    },
+    "subauthorities": {
+      "topic":          "oclc.topic",
+      "geographic":     "oclc.geographic",
+      "event_name":     "oclc.eventName",
+      "personal_name":  "oclc.personalName",
+      "corporate_name": "oclc.corporateName",
+      "uniform_title":  "oclc.uniformTitle",
+      "period":         "oclc.period",
+      "form":           "oclc.form",
+      "alt_lc":         "oclc.altlc"
+    }
+  }
+}
+```
+
+NOTES:
+* term: (optional) is used to define how to request term information from the authority and how to interpret results.
+  * url: (required) templated link representation of the authority API URL and mapping of parameters for requesting term information from the authority
+    * template: is the authority API URL with placeholders for substitution parameters in the form {?var_name}
+      * NOTE: {?term_id} (required) and {?subauth} (optional) are expected to match to QA params (see qa_replacement_patterns to match QA params with mapping variables)
+      * Additional substitutions can be made in the authority API if supported by the authority by adding additional mappings.  Search has an example with maximumRecords.
+        * variable: should match a replacement pattern in the template  (e.g. variable: maximumRecords  ==>  {?maximumRecords}
+        * required: true | false  (NOTE: Not enforced at this time.)
+        * default: provide a default value that will be used if not specified
+      * See (documentation of templated-links)[http://www.hydra-cg.com/spec/latest/core/#templated-links] for more information.
+  * qa_replacement_patterns: identifies which mapping variables are being used for term_id and subauth.
+    * NOTE: The URL to make a term request via QA always uses term_id and subauth as the param names.  qa_replacement_patters allows the url template to use a different variable name for pattern replacement.
+  * language:  (optional)  values:  array of en | fr | etc.  -- identify language to use to include in results, filtering out triples of other languages
+    * NOTE: Some authoritys' API URL allows language to be specified as a parameter.  In that case, use pattern replacement to add the language to the API URL to prevent alternate languages from being returned in the results.
+    * NOTE: At this writing, only label and altlabel are filtered.
+  * term_id:  (optional)  values:  ID (default) | URI  - This tells apps whether `__TERM_ID__` replacement is expecting an ID or URI.
+  * results: (required)  lists predicates to select out for normalization in the hash results
+    * id_predicate:  (optional)
+    * label_predicate:  (required)
+    * altlabel_predicate:  (optional)
+    * sameas_predicate:  (optional)
+    * narrower_predicate:  (optional)
+    * broader_predicate:  (optional)
+  * subauthorities:  (optional)
+    * subauthority name (e.g. topic:, personal_name:, corporate_name, etc.)  Value for {?subauth} are limited to the values in the list of subauthorities.
+
+* search: (optional) is used to define how to send a query to the authority and how to interpret results.
+  * url: (required) templated link representation of the authority API URL and mapping of parameters for sending a query to the authority
+    * template: is the authority API URL with placeholders for substitution parameters in the form {?var_name}
+      * NOTE: {?query} (required) and {?subauth} (optional) are expected to match to QA params (see qa_replacement_patterns to match QA params with mapping variables)
+      * Additional substitutions can be made in the authority API if supported by the authority by adding additional mappings.  Search has an example with maximumRecords.
+        * variable: should match a replacement pattern in the template  (e.g. variable: maximumRecords  ==>  {?maximumRecords}
+        * required: true | false  (NOTE: Not enforced at this time.)
+        * default: provide a default value that will be used if not specified
+      * See (documentation of templated-links)[http://www.hydra-cg.com/spec/latest/core/#templated-links] for more information.
+  * qa_replacement_patterns: identifies which mapping variables are being used for term_id and subauth.
+    * NOTE: The URL to make a term request via QA always uses term_id and subauth as the param names.  qa_replacement_patters allows the url template to use a different variable name for pattern replacement.
+  * language:  (optional)  values:  array of en | fr | etc.  -- identify language to use to include in results, filtering out triples of other languages
+    * NOTE: Some authoritys' API URL allows language to be specified as a parameter.  In that case, use pattern replacement to add the language to the API URL to prevent alternate languages from being returned in the results.
+    * NOTE: At this writing, only label and altlabel are filtered.
+  * results: (required)  lists predicates to normalize and include in json results
+    * id_predicate:  (optional)
+    * label_predicate:  (required)
+    * altlabel_predicate:  (optional)
+  * subauthorities:  (optional)
+    * subauthority name (e.g. topic:, personal_name:, corporate_name, etc.)  Value for {?subauth} are limited to the values in the list of subauthorities.
+
+
+##### Add new configuration
+You can add linked data authorities by adding configuration files to your rails app in `Rails.root/config/authorities/linked_data/YOUR_AUTH.json`
+
+##### Modify existing configuration
+To modify one of the QA supplied configurations, copy it to your app in `Rails.root/config/authorities/linked_data/YOUR_AUTH.json`.  Make your modifications to the json configuration file in your app.
+
+#### Query
+To query OCLC Fast Linked Data service by code...
+
+```ruby
+# Search OCLC Fast all sub-authorities with default value for number of results to return
+lda = Qa::Authorities::LinkedData::GenericAuthority.new(:OCLC_FAST)
+ld_results = lda.search "Cornell University"
+
+# Search OCLC Fast all sub-authorities passing in value for number of results to return
+lda = Qa::Authorities::LinkedData::GenericAuthority.new(:OCLC_FAST)
+ld_results = lda.search "Cornell University",{"maximumRecords" => "5"}
+
+# Search OCLC Fast Corporate Name sub-authority passing in value for number of results to return
+lda = Qa::Authorities::LinkedData::GenericAuthority.new(:OCLC_FAST,'corporate_name')
+ld_results = lda.search "Cornell University",{"maximumRecords" => "3"}
+```
+
+or by URL when QA is an installed gem in an app...
+
+```
+http://localhost:3000/qa/search/linked_data/oclc_fast?q=Cornell&maximumRecords=3
+```
+
+Returns results in the format...
+
+```json
+[{"uri":"http://id.worldcat.org/fast/530369","id":"530369","label":"Cornell University"},
+ {"uri":"http://id.worldcat.org/fast/5140","id":"5140","label":"Cornell, Joseph"},
+ {"uri":"http://id.worldcat.org/fast/557490","id":"557490","label":"New York State School of Industrial and Labor Relations"}]
+```
+
+NOTE: For some authorities, the uri and id will both be the uri.
+
+and with subauthority...
+
+```
+http://localhost:3000/qa/search/linked_data/oclc_fast/personal_name?q=Cornell&maximumRecords=3
+```
+
+returning results...
+
+```json
+[{"uri":"http://id.worldcat.org/fast/5140","id":"5140","label":"Cornell, Joseph"},
+ {"uri":"http://id.worldcat.org/fast/72456","id":"72456","label":"Cornell, Sarah Maria, 1802-1832"},
+ {"uri":"http://id.worldcat.org/fast/409667","id":"409667","label":"Cornell, Ezra, 1807-1874"}]
+```
+
+#### Find term
+To find a single term in OCLC Fast Linked Data service by code...
+
+```ruby
+# Search OCLC Fast all sub-authorities with default value for number of results to return
+lda = Qa::Authorities::LinkedData::GenericAuthority.new(:OCLC_FAST_ALL)
+ld_results = lda.find 530369
+```
+
+or by URL when QA is an installed gem in an app...
+
+```
+http://localhost:3000/qa/show/linked_data/oclc_fast/530369
+```
+
+Returns results in the format...
+
+```json
+{"uri":"http://id.worldcat.org/fast/530369",
+ "id":"530369","label":"Cornell University",
+ "altlabel":["Ithaca (N.Y.). Cornell University","Kornelʹskii universitet","Kʻang-nai-erh ta hsüeh"],
+ "sameas":["http://id.loc.gov/authorities/names/n79021621","https://viaf.org/viaf/126293486"],
+ "predicates":{
+   "http://purl.org/dc/terms/identifier":"530369",
+   "http://www.w3.org/2004/02/skos/core#inScheme":["http://id.worldcat.org/fast/ontology/1.0/#fast","http://id.worldcat.org/fast/ontology/1.0/#facet-Corporate"],
+   "http://www.w3.org/1999/02/22-rdf-syntax-ns#type":"http://schema.org/Organization",
+   "http://www.w3.org/2004/02/skos/core#prefLabel":"Cornell University",
+   "http://schema.org/name":["Cornell University","Ithaca (N.Y.). Cornell University","Kornelʹskii universitet","Kʻang-nai-erh ta hsüeh"],
+   "http://www.w3.org/2004/02/skos/core#altLabel":["Ithaca (N.Y.). Cornell University","Kornelʹskii universitet","Kʻang-nai-erh ta hsüeh"],
+   "http://schema.org/sameAs":["http://id.loc.gov/authorities/names/n79021621","https://viaf.org/viaf/126293486"]}}
+```
+
+NOTE: All predicates with the URI as the subject will be included under "predicates" key.  The selected keys are determined by the configuration file and can be one or more of id_predicate, label_predicate (required), altlabel_predicate, sameas_predicate, narrower_predicate, or broader_predicate.
+
+#### Add javascript to support autocomplete
+
+See [Using with autocomplete in Sufia](https://github.com/projecthydra/questioning_authority/wiki/Using-with-autocomplete-in-Sufia) in the wiki documentation for QA.
+
+
+
 # Developer Notes
 
 [How to Contribute](./CONTRIBUTING.md)

data/app/controllers/qa/linked_data_terms_controller.rb

@@ -0,0 +1,127 @@
+# This controller is used for all requests to linked data authorities. It will verify params and figure
+# out which linked data authority to query based on the 'vocab' param.
+
+class Qa::LinkedDataTermsController < ApplicationController
+  before_action :check_authority, :init_authority
+  before_action :check_search_subauthority, :check_query_param, only: :search
+  before_action :check_show_subauthority, :check_id_param, only: :show
+
+  # Provide a warning if there is a request for all terms.
+  def index
+    logger.warn 'Linked data authorities do not support retrieving all terms.'
+    head :not_found
+  end
+
+  # Return a list of terms based on a query
+  # @see Qa::Authorities::LinkedData::SearchQuery#search
+  def search
+    begin
+      terms = @authority.search(query, subauth: subauthority, language: language, replacements: replacement_params)
+    rescue Qa::ServiceUnavailable
+      logger.warn "Service Unavailable - Search query #{query} unsuccessful for#{subauth_warn_msg} authority #{vocab_param}"
+      head :not_found
+      return
+    rescue RDF::FormatError
+      logger.warn "RDF Format Error - Results from search query #{query} for#{subauth_warn_msg} authority #{vocab_param} was not identified as a valid RDF format.  You may need to include the linkeddata gem."
+      head :not_found
+      return
+    end
+    render json: terms
+  end
+
+  # Return all the information for a given term
+  # @see Qa::Authorities::LinkedData::FindTerm#find
+  def show
+    begin
+      term = @authority.find(id, subauth: subauthority, language: language, replacements: replacement_params)
+    rescue Qa::TermNotFound
+      logger.warn "Term Not Found - Fetch term #{id} unsuccessful for#{subauth_warn_msg} authority #{vocab_param}"
+      head :not_found
+      return
+    rescue Qa::ServiceUnavailable
+      logger.warn "Service Unavailable - Fetch term #{id} unsuccessful for#{subauth_warn_msg} authority #{vocab_param}"
+      head :not_found
+      return
+    rescue RDF::FormatError
+      logger.warn "RDF Format Error - Results from fetch term #{id} for#{subauth_warn_msg} authority #{vocab_param} was not identified as a valid RDF format.  You may need to include the linkeddata gem."
+      head :not_found
+      return
+    end
+    render json: term
+  end
+
+  private
+
+    def check_authority
+      if params[:vocab].nil? || !params[:vocab].size.positive?
+        logger.warn "Required param 'vocab' is missing or empty"
+        head :not_found
+      end
+    end
+
+    def check_search_subauthority
+      return if subauthority.nil?
+      unless @authority.search_subauthority?(subauthority)
+        logger.warn "Unable to initialize linked data search sub-authority '#{subauthority}' for authority '#{vocab_param}'"
+        head :not_found
+      end
+    end
+
+    def check_show_subauthority
+      return if subauthority.nil?
+      unless @authority.term_subauthority?(subauthority)
+        logger.warn "Unable to initialize linked data term sub-authority '#{subauthority}' for authority '#{vocab_param}'"
+        head :not_found
+      end
+    end
+
+    def init_authority
+      @authority = Qa::Authorities::LinkedData::GenericAuthority.new(vocab_param)
+    rescue Qa::InvalidLinkedDataAuthority => e
+      logger.warn e.message
+      head :not_found
+    end
+
+    def vocab_param
+      params[:vocab].upcase.to_sym
+    end
+
+    def check_query_param
+      if params[:q].nil? || !params[:q].size.positive?
+        logger.warn "Required search param 'q' is missing or empty"
+        head :not_found
+      end
+    end
+
+    # converts wildcards into URL-encoded characters
+    def query
+      params[:q].gsub("*", "%2A")
+    end
+
+    def check_id_param
+      if params[:id].nil? || !params[:id].size.positive?
+        logger.warn "Required show param 'id' is missing or empty"
+        head :not_found
+      end
+    end
+
+    def id
+      params[:id]
+    end
+
+    def language
+      params[:lang]
+    end
+
+    def subauthority
+      params[:subauthority]
+    end
+
+    def replacement_params
+      params.reject { |k, _v| ['q', 'vocab', 'controller', 'action', 'subauthority', 'language', 'id'].include?(k) }
+    end
+
+    def subauth_warn_msg
+      subauthority.nil? ? "" : " sub-authority #{subauthority} in"
+    end
+end

data/config/authorities/linked_data/agrovoc.json

@@ -0,0 +1,61 @@
+{
+  "term": {
+    "url": {
+      "@context": "http://www.w3.org/ns/hydra/context.jsonld",
+      "@type":    "IriTemplate",
+      "template": "http://aims.fao.org/skosmos/rest/v1/data?uri=http://aims.fao.org/aos/agrovoc/{?term_id}",
+      "variableRepresentation": "BasicRepresentation",
+      "mapping": [
+        {
+          "@type":    "IriTemplateMapping",
+          "variable": "term_id",
+          "property": "hydra:freetextQuery",
+          "required": true
+        }
+      ]
+    },
+    "qa_replacement_patterns": {
+      "term_id": "term_id"
+    },
+    "term_id": "ID",
+    "language": ["en"],
+    "results": {
+      "label_predicate":    "http://www.w3.org/2004/02/skos/core#prefLabel",
+      "altlabel_predicate": "http://www.w3.org/2004/02/skos/core#altLabel",
+      "broader_predicate":  "http://www.w3.org/2004/02/skos/core#broader",
+      "narrower_predicate": "http://www.w3.org/2004/02/skos/core#narrower",
+      "sameas_predicate":   "http://www.w3.org/2004/02/skos/core#exactMatch"
+    }
+  },
+  "search": {
+    "url": {
+      "@context": "http://www.w3.org/ns/hydra/context.jsonld",
+      "@type":    "IriTemplate",
+      "template": "http://aims.fao.org/skosmos/rest/v1/search/?query=*{?query}*&lang={?lang}",
+      "variableRepresentation": "BasicRepresentation",
+      "mapping": [
+        {
+          "@type":    "IriTemplateMapping",
+          "variable": "query",
+          "property": "hydra:freetextQuery",
+          "required": true
+        },
+        {
+          "@type":    "IriTemplateMapping",
+          "variable": "lang",
+          "property": "hydra:freetextQuery",
+          "required": false,
+          "default":  "en"
+        }
+      ]
+    },
+    "qa_replacement_patterns": {
+      "query": "query"
+    },
+    "results": {
+      "id_predicate":       "http://purl.org/dc/terms/identifier",
+      "label_predicate":    "http://www.w3.org/2004/02/skos/core#prefLabel",
+      "sort_predicate":     "http://www.w3.org/2004/02/skos/core#prefLabel"
+    }
+  }
+}