qa 5.1.0 → 5.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 8158fccc9df38a91d74c1dbda277ce9308cf08c3
-  data.tar.gz: 28be737c24fc1cdd77fed1fc5da6216c68b675fd
+SHA256:
+  metadata.gz: 73811a2b2371ead4042bab0a11e4b4b0f496e94eb6e92a2f6ddb550ac90f9d41
+  data.tar.gz: a8888e0db1aa46b562eaab6d45be5fd0de8df52ac9753189416a3ca0fc0d44a3
 SHA512:
-  metadata.gz: d8e1fdedaa020bf6ffcac6bf1248729439d758be0966bb397aeaf05c54c681fd0f4f5e5c061e9b5ece5f8b8c9dc0494c6d1c9091c2f691e185256cb1cc42ec18
-  data.tar.gz: 49e4402ab1422fee3e78cc7fdf485175144642b26fc6697a64873877fbbdc6095ad011881e8c42a35c9fb21c5704a056443f0ce6bc7e26c14494c958007d6434
+  metadata.gz: e4aaecb73516d38ddca2ed4dc61571aa1c70fc4fdedee75d4aa1a7763193147303b97124d4e858471508f09ba6adb6158fb63111c5f2ca817d4cf4ec7164690f
+  data.tar.gz: 9446f03eeaad6aa21890f97d4def246a6915f932f83b764a1bea2b6c9a818c8eb43c45f5d3b8b8235cd57d1efc59c4c08caab93fe1ed1ab5bea65d5a690f23cf

data/README.md

@@ -1,77 +1,83 @@
 # Questioning Authority
 
-Code:
-[![Gem Version](https://badge.fury.io/rb/qa.png)](http://badge.fury.io/rb/qa)
-[![Build Status](https://circleci.com/gh/samvera/questioning_authority.svg?style=svg)](https://circleci.com/gh/samvera/questioning_authority)
-[![Coverage Status](https://coveralls.io/repos/github/samvera/questioning_authority/badge.svg?branch=master)](https://coveralls.io/github/samvera/questioning_authority?branch=master)
+Code: [![Gem Version](https://badge.fury.io/rb/qa.png)](http://badge.fury.io/rb/qa) [![Build Status](https://circleci.com/gh/samvera/questioning_authority.svg?style=svg)](https://circleci.com/gh/samvera/questioning_authority) [![Coverage Status](https://coveralls.io/repos/github/samvera/questioning_authority/badge.svg?branch=master)](https://coveralls.io/github/samvera/questioning_authority?branch=master)
 
-Docs:
-[![Contribution Guidelines](http://img.shields.io/badge/CONTRIBUTING-Guidelines-blue.svg)](./CONTRIBUTING.md)
-[![Apache 2.0 License](http://img.shields.io/badge/APACHE2-license-blue.svg)](./LICENSE)
+Docs: [![Contribution Guidelines](http://img.shields.io/badge/CONTRIBUTING-Guidelines-blue.svg)](./CONTRIBUTING.md) [![Apache 2.0 License](http://img.shields.io/badge/APACHE2-license-blue.svg)](./LICENSE)
 
 Jump In: [![Slack Status](http://slack.samvera.org/badge.svg)](http://slack.samvera.org/)
 
 You should question your authorities.
 
-----
+--------------------------------------------------------------------------------
+
 ## Table of Contents
 
-  * [What does this do?](#what-does-this-do)
-  * [How does it work?](#how-does-it-work)
-    * [Sub-Authorities](#sub-authorities)
-  * [How do I use this?](#how-do-i-use-this)
-    * [Basic QA Requests](#basic-qa-requests)
-    * [Typical JSON Results](#typical-json-results)
-  * [Authority Sources information](#authority-sources-information)
-  * [Developer Notes](#developer-notes)
-    * [Compatibility](#compatibility)
-    * [Product Owner & Maintenance](#product-owner--maintenance)
-      * [Product Owner](#product-owner)
-  * [Help](#help)
-  * [Acknowledgments](#acknowledgments)
-
-## Not seeing documentation you used to find in the README?  
-
-Much of the documentation has moved to the [Questioning Authority wiki](https://github.com/samvera/questioning_authority/wiki) to allow for better organization.  We hope that you will find this easier to use.
-
-----
+- [What does this do?](#what-does-this-do)
+- [How does it work?](#how-does-it-work)
+
+  - [Sub-Authorities](#sub-authorities)
+
+- [How do I use this?](#how-do-i-use-this)
+
+  - [Basic QA Requests](#basic-qa-requests)
+  - [Typical JSON Results](#typical-json-results)
+
+- [Authority Sources information](#authority-sources-information)
+
+- [Developer Notes](#developer-notes)
+
+  - [Compatibility](#compatibility)
+  - [Product Owner & Maintenance](#product-owner--maintenance)
+
+    - [Product Owner](#product-owner)
+
+- [Help](#help)
+
+- [Acknowledgments](#acknowledgments)
+
+## Not seeing documentation you used to find in the README?
+
+Much of the documentation has moved to the [Questioning Authority wiki](https://github.com/samvera/questioning_authority/wiki) to allow for better organization. We hope that you will find this easier to use.
+
+--------------------------------------------------------------------------------
+
 ## What does this do?
 
-Provides a set of uniform RESTful routes to query any controlled vocabulary or set of authority terms.
-Results are returned in JSON and can be used within the context of a Rails application or any other
-Ruby environment. Primary examples would include providing auto-complete functionality via Javascript
-or populating a dropdown menu with a set of terms.
+Provides a set of uniform RESTful routes to query any controlled vocabulary or set of authority terms. Results are returned in JSON and can be used within the context of a Rails application or any other Ruby environment. Primary examples would include providing auto-complete functionality via Javascript or populating a dropdown menu with a set of terms.
 
 ## How does it work?
 
-Authorities are defined as classes, each implementing a set of methods allowing a controller to return
-results from a given vocabulary in the JSON format.  The controller does three things:
+Authorities are defined as classes, each implementing a set of methods allowing a controller to return results from a given vocabulary in the JSON format. The controller does three things:
 
-* provide a list of all terms (if allowed by the class)
-* return a set of terms matching a given query
-* return the complete information for a specific term given its identifier
+- provide a list of all terms (if allowed by the class)
+- return a set of terms matching a given query
+- return the complete information for a specific term given its identifier
 
-Depending on the kind of authority or its API, the controller may not do all of these things such
-as return a complete list of terms.
+Depending on the kind of authority or its API, the controller may not do all of these things such as return a complete list of terms.
 
 ### Sub-Authorities
 
-Some authorities, such as Library of Congress, allow sub-authorities which is an additional parameter that
-further defines the kind of authority to use with the context of a larger one.
+Some authorities, such as Library of Congress, allow sub-authorities which is an additional parameter that further defines the kind of authority to use with the context of a larger one.
 
 ## How do I use this?
 
 Add the gem to your Gemfile
 
-    gem 'qa'
+```
+gem 'qa'
+```
 
 Run bundler
 
-    bundle install
+```
+bundle install
+```
 
 Install the gem to your application
 
-    rails generate qa:install
+```
+rails generate qa:install
+```
 
 This will copy over some additional config files and add the engine's routes to your `config/route.rb`.
 
@@ -79,84 +85,104 @@ Start questioning your authorities!
 
 ### Basic QA Requests
 
-These show the basic routing patterns for connecting to authorities.  See the [Questioning Authority wiki](https://github.com/samvera/questioning_authority/wiki) documentation for detailed documentation and examples for each authority and local authorities.
+These show the basic routing patterns for connecting to authorities. See the [Questioning Authority wiki](https://github.com/samvera/questioning_authority/wiki) documentation for detailed documentation and examples for each authority and local authorities.
 
 Return a complete list of terms:
 
-    /qa/terms/:vocab
-    /qa/terms/:vocab/:subauthority
+```
+/qa/terms/:vocab
+/qa/terms/:vocab/:subauthority
+```
 
 Return a set of terms matching a given query
 
-    /qa/search/:vocab?q=search_term
-    /qa/search/:vocab/:subauthority?q=search_term
+```
+/qa/search/:vocab?q=search_term
+/qa/search/:vocab/:subauthority?q=search_term
+```
 
 Return the complete information for a specific term given its identifier
 
-    /qa/show/:vocab/:id
-    /qa/show/:vocab/:subauthority/:id
-
-
+```
+/qa/show/:vocab/:id
+/qa/show/:vocab/:subauthority/:id
+```
 
 ### Typical JSON Results
 
 Results are returned in JSON in this format:
 
-    [
-        {"id" : "subject_id_1", "label" : "First labels"},
-        {"id" : "subject_id_2", "label" : "Printing labels"},
-        {"id" : "", "label" : "This term has no id number"},
-        {"id" : "", "label" : "Neither does this"}
-    ]
-
+```
+[
+    {"id" : "subject_id_1", "label" : "First labels"},
+    {"id" : "subject_id_2", "label" : "Printing labels"},
+    {"id" : "", "label" : "This term has no id number"},
+    {"id" : "", "label" : "Neither does this"}
+]
+```
 
 # Authority Sources information
 
 See the [Questioning Authority wiki](https://github.com/samvera/questioning_authority/wiki) for documentation on how to connect to the supported authorities, documentation on how to create new authorities, and other useful tips.
 
-
 # Developer Notes
 
 [How to Contribute](./CONTRIBUTING.md)
 
 To develop this gem, clone the repository, then run:
 
-    bundle install
-    rake ci
+```
+bundle install
+rake ci
+```
 
-This will install the gems, create a dummy application under spec/internal and run the tests.  After you've made changes,
-make sure you've included tests and run the test suite with a new sample application:
+This will install the gems, create a dummy application under spec/internal and run the tests. After you've made changes, make sure you've included tests and run the test suite with a new sample application:
 
-    rake engine_cart:clean
-    rake ci
+```
+rake engine_cart:clean
+rake ci
+```
 
 Commit your features into a new branch and submit a pull request.
 
 ## Compatibility
 
-* Ruby 2.5 or the latest 2.4 version is recommended.  Later versions may also work.
-* Rails 5 is required. We recommend the latest Rails 5.2 release.
+- Ruby 2.5 or the latest 2.4 version is recommended. Later versions may also work.
+- Rails 5 is required. We recommend the latest Rails 5.2 release.
 
 ## Product Owner & Maintenance
 
-Questioning Authority is a Core Component of the Samvera community. The documentation for
-what this means can be found [here](http://samvera.github.io/core_components.html#requirements-for-a-core-component).
+Questioning Authority is a Core Component of the Samvera community. The documentation for what this means can be found [here](http://samvera.github.io/core_components.html#requirements-for-a-core-component).
 
 ### Product Owner
 
 [elrayle](https://github.com/elrayle)
 
+## Releasing
+
+1. `bundle install`
+2. Increase the version number in `lib/qa/version.rb`
+3. Increase the same version number in `.github_changelog_generator`
+4. Update `CHANGELOG.md` by running this command:
+
+  ```
+  github_changelog_generator --user samvera --project questioning_authority --token YOUR_GITHUB_TOKEN_HERE
+  ```
+
+5. Commit these changes to the master branch
+
+6. Run `rake release`
+
 # Help
 
 The Samvera community is here to help. Please see our [support guide](./SUPPORT.md).
 
 # Acknowledgments
 
-This software has been developed by and is brought to you by the Samvera community.  Learn more at the
-[Samvera website](http://samvera.org/).
+This software has been developed by and is brought to you by the Samvera community. Learn more at the [Samvera website](http://samvera.org/).
 
 ![Samvera Logo](https://wiki.duraspace.org/download/thumbnails/87459292/samvera-fall-font2-200w.png?version=1&modificationDate=1498550535816&api=v2)
 
-### Special thanks to...
+## Special thanks to...
 
 [Jeremy Friesen](https://github.com/jeremyf) who gave us the name for our gem.

data/app/services/qa/linked_data/graph_service.rb

@@ -91,6 +91,7 @@ module Qa
           end
 
           def process_error(e, url)
+            Rails.logger.warn("******** RDF::Graph#load failure: exception=#{e.inspect}, url=#{url}")
             uri = URI(url)
             raise RDF::FormatError, "Unknown RDF format of results returned by #{uri}. (RDF::FormatError)  You may need to include gem 'linkeddata'." if e.is_a? RDF::FormatError
             response_code = ioerror_code(e)

data/app/services/qa/linked_data/request_header_service.rb

@@ -1,8 +1,9 @@
 # Service to construct a request header that includes optional attributes for search and fetch requests.
+require 'geocoder'
 module Qa
   module LinkedData
     class RequestHeaderService
-      attr_reader :request, :params
+      attr_reader :request, :params, :request_id
 
       # @param request [HttpRequest] request from controller
       # @param params [Hash] attribute-value pairs holding the request parameters
@@ -16,6 +17,8 @@ module Qa
       def initialize(request:, params:)
         @request = request
         @params = params
+        @request_id = request.request_id
+        log_request
       end
 
       # Construct request parameters to pass to search_query (linked data module).
@@ -23,6 +26,8 @@ module Qa
       # @see Qa::Authorities::LinkedData::SearchQuery
       def search_header
         header = {}
+        header[:request] = request
+        header[:request_id] = request_id
         header[:subauthority] = params.fetch(:subauthority, nil)
         header[:user_language] = user_language
         header[:performance_data] = performance_data?
@@ -37,6 +42,8 @@ module Qa
       # @see Qa::Authorities::LinkedData::FindTerm
       def fetch_header
         header = {}
+        header[:request] = request
+        header[:request_id] = request_id
         header[:subauthority] = params.fetch(:subauthority, nil)
         header[:user_language] = user_language
         header[:performance_data] = performance_data?
@@ -62,6 +69,18 @@ module Qa
 
       private
 
+        def log_request
+          msg = "******** #{request.path_parameters[:action].upcase}"
+          unless Qa.config.suppress_ip_data_from_log
+            gc = request.respond_to?(:location) ? request.location : nil
+            city = gc.nil? ? "UNKNOWN" : gc.city
+            state = gc.nil? ? "UNKNOWN" : gc.state
+            country = gc.nil? ? "UNKNOWN" : gc.country
+            msg += " from IP #{request.ip} in {city: #{city}, state: #{state}, country: #{country}}"
+          end
+          Rails.logger.info(msg)
+        end
+
         # filter literals in results to this language
         def user_language
           request_language = request.env['HTTP_ACCEPT_LANGUAGE']

data/lib/generators/qa/install/templates/config/initializers/qa.rb

@@ -23,4 +23,9 @@ Qa.config do |config|
   # When false, properties that do not override default optional behavior will be shown whether or not the property has a value in the graph.
   # When true, properties that do not override default optional behavior will not be shown whn the property does not have a value in the graph.
   # config.property_map_default_for_optional = false
+
+  # IP data including IP address, city, state, and country will be logged with each request.
+  # When false, IP data is logged
+  # When true, IP data will not be logged (default for backward compatibility)
+  # config.suppress_ip_data_from_log = true
 end

data/lib/qa/authorities/discogs/discogs_utils.rb

@@ -55,6 +55,28 @@ module Qa::Authorities
         "http://id.loc.gov/ontologies/bibframe/Role"
       end
 
+      def format(tc)
+        return 'json' unless tc.params.key?('format')
+        return 'json' if tc.params['format'].blank?
+        tc.params['format']
+      end
+
+      def jsonld?(tc)
+        format(tc).casecmp?('jsonld')
+      end
+
+      def n3?(tc)
+        format(tc).casecmp?('n3')
+      end
+
+      def ntriples?(tc)
+        format(tc).casecmp?('ntriples')
+      end
+
+      def graph_format?(tc)
+        jsonld?(tc) || n3?(tc) || ntriples?(tc)
+      end
+
       def discogs_genres
         DISCOGS_GENRE_MAPPING
       end
@@ -63,6 +85,19 @@ module Qa::Authorities
         DISCOGS_FORMATS_MAPPING
       end
 
+      # @param json results
+      # @param json results
+      # @return [String] status information
+      def check_for_msg_response(release_resp, master_resp)
+        if release_resp.key?("message") && master_resp.key?("message")
+          "no responses"
+        elsif !release_resp.key?("message") && !master_resp.key?("message")
+          "two responses"
+        else
+          "mixed"
+        end
+      end
+
       # both the work and the instance require a statement for the release year
       # @param [Hash] the http response from discogs
       # @return [Array] rdf statements

data/lib/qa/authorities/discogs/generic_authority.rb

@@ -4,6 +4,7 @@ module Qa::Authorities
   class Discogs::GenericAuthority < Base
     include WebServiceBase
     include Discogs::DiscogsTranslation
+    include Discogs::DiscogsUtils
 
     class_attribute :discogs_secret, :discogs_key
     attr_accessor :primary_artists, :selected_format, :work_uri, :instance_uri
@@ -28,7 +29,14 @@ module Qa::Authorities
         Rails.logger.error "Questioning Authority tried to call Discogs, but no secret and/or key were set."
         return []
       end
-      parse_authority_response(json(build_query_url(q, tc.params["subauthority"])))
+      response = json(build_query_url(q, tc))
+      if tc.params["response_header"] == "true"
+        response_hash = {}
+        response_hash["results"] = parse_authority_response(response)
+        response_hash["response_header"] = build_response_header(response)
+        return response_hash
+      end
+      parse_authority_response(response)
     end
 
     # If the subauthority = "all" (though it shouldn't), call the fetch_discogs_results method to determine
@@ -52,9 +60,18 @@ module Qa::Authorities
     # @param [String] the query
     # @param [String] the subauthority
     # @return [String] the url
-    def build_query_url(q, subauthority)
+    def build_query_url(q, tc)
+      page = nil
+      per_page = nil
+      if tc.params["startRecord"].present?
+        page = (tc.params["startRecord"].to_i - 1) / tc.params["maxRecords"].to_i + 1
+        per_page = tc.params["maxRecords"]
+      else
+        page = tc.params["page"]
+        per_page = tc.params["per_page"]
+      end
       escaped_q = ERB::Util.url_encode(q)
-      "https://api.discogs.com/database/search?q=#{escaped_q}&type=#{subauthority}&key=#{discogs_key}&secret=#{discogs_secret}"
+      "https://api.discogs.com/database/search?q=#{escaped_q}&type=#{tc.params['subauthority']}&page=#{page}&per_page=#{per_page}&key=#{discogs_key}&secret=#{discogs_secret}"
     end
 
     # @param [String] the id of the selected item
@@ -86,19 +103,6 @@ module Qa::Authorities
         release_resp
       end
 
-      # @param json results
-      # @param json results
-      # @return [String] status information
-      def check_for_msg_response(release_resp, master_resp)
-        if release_resp.key?("message") && master_resp.key?("message")
-          "no responses"
-        elsif !release_resp.key?("message") && !master_resp.key?("message")
-          "two responses"
-        else
-          "mixed"
-        end
-      end
-
       # @param [Hash] the http response from discogs
       # @example returns parsed discogs data with context
       # [{
@@ -131,6 +135,19 @@ module Qa::Authorities
         end
       end
 
+      # @param [Hash] the http response from discogs
+      # @param [Class] QA::TermsController
+      # @example returns parsed discogs pagination data
+      def build_response_header(response)
+        start_record = (response['pagination']['page'] - 1) * response['pagination']['per_page'] + 1
+        rh_hash = {}
+        rh_hash['start_record'] = start_record
+        rh_hash['requested_records'] = response['pagination']['per_page']
+        rh_hash['retrieved_records'] = response['results'].length
+        rh_hash['total_records'] = response['pagination']['items']
+        rh_hash
+      end
+
       # @param [Hash] the results hash from the JSON returned by Discogs
       def build_uri(result)
         result['uri'].present? ? "https://www.discogs.com" + result['uri'].to_s : result['resource_url'].to_s
@@ -156,27 +173,5 @@ module Qa::Authorities
       def get_context_for_array(item)
         item.present? ? item : [""]
       end
-
-      def format(tc)
-        return 'json' unless tc.params.key?('format')
-        return 'json' if tc.params['format'].blank?
-        tc.params['format']
-      end
-
-      def jsonld?(tc)
-        format(tc).casecmp?('jsonld')
-      end
-
-      def n3?(tc)
-        format(tc).casecmp?('n3')
-      end
-
-      def ntriples?(tc)
-        format(tc).casecmp?('ntriples')
-      end
-
-      def graph_format?(tc)
-        jsonld?(tc) || n3?(tc) || ntriples?(tc)
-      end
   end
 end