commonmeta-ruby 3.7.2 → 3.8.0

data/docs/index.qmd

@@ -0,0 +1,83 @@
+---
+title: Guide
+---
+
+[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.5785519.svg)](https://doi.org/10.5281/zenodo.5785519)
+[![Gem Version](https://badge.fury.io/rb/commonmeta-ruby.svg)](https://badge.fury.io/rb/commonmeta-ruby)
+![GitHub](https://img.shields.io/github/license/front-matter/commonmeta-ruby?logo=MIT)
+
+commonmeta-ruby is a Ruby gem and command-line utility for the conversion of scholarly metadata, including [schema.org](https://schema.org). Based on the [bolognese](https://github.com/datacite/bolognese) gem, but using [commonmeta](https://commonmeta.org) as the intermediate format, and supporting additional metadata formats. A Python version is available at [commonmeta-py](https://github.com/front-matter/commonmeta-py).
+
+## Installation
+
+Requires Ruby 3.0 or later (Ruby 2.x [has reached its end of life](https://endoflife.date/ruby) March 2023). Then add the following to your `Gemfile` to install the
+latest version:
+
+```
+gem 'commonmeta-ruby'
+```
+
+Then run `bundle install` to install into your environment.
+
+You can also install the gem system-wide in the usual way:
+
+```
+gem install commonmeta-ruby
+```
+
+## Commands
+
+Run the `commonmeta` command with either an identifier (DOI or URL) or filename:
+
+```
+commonmeta https://doi.org/10.7554/elife.01567
+```
+
+```
+commonmeta example.xml
+```
+
+commonmeta can read BibTeX files (file extension `.bib`), RIS files (file extension `.ris`), Crossref files (file extension `.xml`), DataCite files and CSL-JSON files.
+
+The input format (e.g. Crossref or BibteX) is automatically detected, but
+you can also provide the format with the `--from` or `-f` flag. The supported
+input formats are listed in the table above.
+
+The output format is determined by the `--to` or `-t` flag, and defaults to `schema_org`.
+
+Show all commands with `commonmeta help`:
+
+```
+Commands:
+  commonmeta                 # convert metadata
+  commonmeta --version, -v   # print the version
+  commonmeta help [COMMAND]  # Describe available commands or one specific command
+```
+
+## Errors
+
+Errors are returned to STDOUT.
+
+All input is validated against the commonmeta JSON schema.
+
+## Supported Metadata Formats
+
+commonmeta-ruby reads and/or writes these metadata formats:
+
+| Format | Name | Content Type | Read | Write |
+| ------ | ---- | ------------ | ---- | ----- |
+| [Commonmeta](https://commonmeta.org) | commonmeta | application/vnd.commonmeta+json | yes | yes |
+| [CrossRef Unixref XML](https://www.crossref.org/schema/documentation/unixref1.1/unixref1.1.html) | crossref | application/vnd.crossref.unixref+xml | yes | yes |
+| [Crossref](https://api.crossref.org) | crossref | application/vnd.crossref+json | yes | no |
+| [DataCite](https://api.datacite.org/) | datacite | application/vnd.datacite.datacite+json | yes | yes |
+| [Schema.org (in JSON-LD)](http://schema.org/) | schema_org | application/vnd.schemaorg.ld+json | yes | yes |
+| [RDF XML](http://www.w3.org/TR/rdf-syntax-grammar/) | rdf_xml | application/rdf+xml | no | yes |
+| [RDF Turtle](http://www.w3.org/TeamSubmission/turtle/) | turtle | text/turtle | no | yes |
+| [CSL-JSON](https://citationstyles.org/) | csl | application/vnd.citationstyles.csl+json | yes | yes |
+| [Formatted text citation](https://citationstyles.org/) | citation | text/x-bibliography | no | yes |
+| [Codemeta](https://codemeta.github.io/) | codemeta | application/vnd.codemeta.ld+json | yes | yes |
+| [Citation File Format (CFF)](https://citation-file-format.github.io/) | cff | application/vnd.cff+yaml | yes | yes |
+| [JATS](https://jats.nlm.nih.gov/) | jats | application/vnd.jats+xml | no | yes |
+| [CSV](https://en.wikipedia.org/wiki/Comma-separated_values) | csv | text/csv | no | yes |
+| [BibTex](http://en.wikipedia.org/wiki/BibTeX) | bibtex | application/x-bibtex | yes | yes |
+| [RIS](http://en.wikipedia.org/wiki/RIS_(file_format)) | ris | application/x-research-info-systems | yes | yes |

data/docs/readers/bibtex_reader.ipynb

@@ -0,0 +1,37 @@
+{
+ "cells": [
+  {
+   "cell_type": "raw",
+   "metadata": {},
+   "source": [
+    "---\n",
+    "title: BibTex Reader\n",
+    "description: |\n",
+    "  This notebook shows how to read metadata in BibTex format.\n",
+    "---"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Fetch the metadata"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Ruby 3.2.2",
+   "language": "ruby",
+   "name": "ruby"
+  },
+  "language_info": {
+   "file_extension": ".rb",
+   "mimetype": "application/x-ruby",
+   "name": "ruby",
+   "version": "3.2.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

data/docs/theme.scss

@@ -0,0 +1,7 @@
+/*-- scss:defaults --*/
+$toc-color: #38a169;
+$header-margin-bottom: 0;
+$link-color: #38a169;
+$link-hover-color: #276749;
+$link-decoration: none;
+$navbar-bg: #e5e7eb;

data/docs/utils/doi_utils.ipynb

@@ -0,0 +1,28 @@
+{
+ "cells": [
+  {
+   "cell_type": "raw",
+   "metadata": {},
+   "source": [
+    "---\n",
+    "title: doi_utils\n",
+    "---"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Ruby 3.2.2",
+   "language": "ruby",
+   "name": "ruby"
+  },
+  "language_info": {
+   "file_extension": ".rb",
+   "mimetype": "application/x-ruby",
+   "name": "ruby",
+   "version": "3.2.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

data/docs/writers/bibtex_writer.ipynb

@@ -0,0 +1,39 @@
+{
+ "cells": [
+  {
+   "cell_type": "raw",
+   "metadata": {},
+   "source": [
+    "---\n",
+    "title: BibTex Writer\n",
+    "description: |\n",
+    "  This notebook shows how to generate BibTex metadata from a scholarly resource in commonmeta format generated by commonmeta-ruby.\n",
+    "---"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Fetch the metadata\n",
+    "\n",
+    "As with all commonmeta-ruby notebooks, we start by fetching metadata, in this example a journal article via its Crossref DOI, and then convert it to the commonmeta format."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Ruby 3.2.2",
+   "language": "ruby",
+   "name": "ruby"
+  },
+  "language_info": {
+   "file_extension": ".rb",
+   "mimetype": "application/x-ruby",
+   "name": "ruby",
+   "version": "3.2.2"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

data/lib/commonmeta/cli.rb

@@ -71,8 +71,8 @@ module Commonmeta
 
     desc "", "encode_by_blog"
 
-    def encode_by_blog(blog_id)
-      prefix = get_doi_prefix_by_blog_id(blog_id)
+    def encode_by_blog(blog_slug)
+      prefix = get_doi_prefix_by_blog_slug(blog_slug)
       return nil unless prefix.present?
       puts encode_doi(prefix)
     end
@@ -115,10 +115,10 @@ module Commonmeta
       puts get_json_feed_by_blog(id)
     end
 
-    desc "", "json_feed_blog_id"
+    desc "", "json_feed_blog_slug"
 
-    def json_feed_blog_id(id)
-      puts get_json_feed_blog_id(id)
+    def json_feed_blog_slug(slug)
+      puts get_json_feed_blog_slug(slug)
     end
 
     desc "", "update_ghost_post"

data/lib/commonmeta/crossref_utils.rb

@@ -342,11 +342,17 @@ module Commonmeta
       return xml if doi_from_url(id).blank? || url.blank?
 
       xml.doi_data do
-        xml.doi(doi_from_url(id).downcase)
+        doi = doi_from_url(id).downcase
+        xml.doi(doi)
         xml.resource(url)
         xml.collection("property" => "text-mining") do
           xml.item do
             xml.resource(url, "mime_type" => "text/html")
+            if is_rogue_scholar_doi?(doi)
+              Array.wrap(files).each do |file|
+                xml.resource(file["url"], "mime_type" => file["mimeType"])
+              end
+            end
           end
         end
       end

data/lib/commonmeta/doi_utils.rb

@@ -77,5 +77,10 @@ module Commonmeta
 
       { 'id' => "https://api.crossref.org/members/#{member_id}", 'name' => name }
     end
+
+    def is_rogue_scholar_doi?(doi)
+      prefix = validate_prefix(doi)
+      %w[10.34732 10.53731 10.54900 10.57689 10.59348 10.59349 10.59350].include?(prefix)
+    end
   end
 end

data/lib/commonmeta/metadata.rb

@@ -1,6 +1,6 @@
 # frozen_string_literal: truefiles
 
-require_relative 'metadata_utils'
+require_relative "metadata_utils"
 
 module Commonmeta
   class Metadata
@@ -10,7 +10,11 @@ module Commonmeta
                   :email, :registrant
     attr_reader :doc, :page_start, :page_end
     attr_writer :id, :provider_id, :client_id, :doi, :alternate_identifiers, :contributors,
-                :titles, :publisher, :license, :date, :volume, :url, :version, :subjects, :descriptions, :language, :sizes, :formats, :schema_version, :meta, :container, :provider, :format, :funding_references, :state, :geo_locations, :type, :additional_type, :files, :references, :related_identifiers, :related_items, :style, :locale, :archive_locations
+                :titles, :publisher, :license, :date, :volume, :url, :version, :subjects, 
+                :descriptions, :language, :sizes, :formats, :schema_version, :meta, :container, 
+                :provider, :format, :funding_references, :state, :geo_locations, :type, 
+                :additional_type, :files, :references, :related_identifiers, :related_items, 
+                :style, :locale, :archive_locations
 
     def initialize(options = {})
       options.symbolize_keys!
@@ -23,29 +27,29 @@ module Commonmeta
 
         # mEDRA, KISTI, JaLC and OP DOIs are found in the Crossref index
         case @from
-        when 'medra'
-          ra = 'mEDRA'
-        when 'kisti'
-          ra = 'KISTI'
-        when 'jalc'
-          ra = 'JaLC'
-        when 'op'
-          ra = 'OP'
+        when "medra"
+          ra = "mEDRA"
+        when "kisti"
+          ra = "KISTI"
+        when "jalc"
+          ra = "JaLC"
+        when "op"
+          ra = "OP"
         end
 
         # generate name for method to call dynamically
         hsh = @from.present? ? send("get_#{@from}", id: id, **options) : {}
-        string = hsh.fetch('string', nil)
+        string = hsh.fetch("string", nil)
       elsif options[:input].present? && File.exist?(options[:input])
         filename = File.basename(options[:input])
         ext = File.extname(options[:input])
         if %w[.bib .ris .xml .json .cff].include?(ext)
           hsh = {
-            'url' => options[:url],
-            'state' => options[:state],
-            'provider_id' => options[:provider_id],
-            'client_id' => options[:client_id],
-            'files' => options[:files]
+            "url" => options[:url],
+            "state" => options[:state],
+            "provider_id" => options[:provider_id],
+            "client_id" => options[:client_id],
+            "files" => options[:files],
           }
           string = File.read(options[:input])
           @from = options[:from] || find_from_format(string: string, ext: ext)
@@ -55,14 +59,14 @@ module Commonmeta
         end
       else
         hsh = {
-          'url' => options[:url],
-          'state' => options[:state],
-          'provider_id' => options[:provider_id],
-          'client_id' => options[:client_id],
-          'files' => options[:files],
-          'contributors' => options[:contributors],
-          'titles' => options[:titles],
-          'publisher' => options[:publisher]
+          "url" => options[:url],
+          "state" => options[:state],
+          "provider_id" => options[:provider_id],
+          "client_id" => options[:client_id],
+          "files" => options[:files],
+          "contributors" => options[:contributors],
+          "titles" => options[:titles],
+          "publisher" => options[:publisher],
         }
         string = options[:input]
         @from = options[:from] || find_from_format(string: string)
@@ -70,7 +74,7 @@ module Commonmeta
 
       # make sure input is encoded as utf8
       if string.present? && string.is_a?(String)
-        dup_string = string.dup.force_encoding('UTF-8').encode!
+        dup_string = string.dup.force_encoding("UTF-8").encode!
       end
       @string = dup_string
 
@@ -81,11 +85,11 @@ module Commonmeta
       @sandbox = options[:sandbox]
 
       # options that come from the datacite database
-      @url = hsh.to_h['url'].presence || options[:url].presence
-      @state = hsh.to_h['state'].presence
-      @provider_id = hsh.to_h['provider_id'].presence
-      @client_id = hsh.to_h['client_id'].presence
-      @files = hsh.to_h['files'].presence
+      @url = hsh.to_h["url"].presence || options[:url].presence
+      @state = hsh.to_h["state"].presence
+      @provider_id = hsh.to_h["provider_id"].presence
+      @client_id = hsh.to_h["client_id"].presence
+      @files = hsh.to_h["files"].presence
 
       # options that come from the cli, needed
       # for crossref doi registration
@@ -125,23 +129,23 @@ module Commonmeta
     end
 
     def id
-      @id ||= meta.fetch('id', nil)
+      @id ||= meta.fetch("id", nil)
     end
 
     def doi
-      @doi ||= meta.fetch('doi', nil)
+      @doi ||= meta.fetch("doi", nil)
     end
 
     def provider_id
-      @provider_id ||= meta.fetch('provider_id', nil)
+      @provider_id ||= meta.fetch("provider_id", nil)
     end
 
     def client_id
-      @client_id ||= meta.fetch('client_id', nil)
+      @client_id ||= meta.fetch("client_id", nil)
     end
 
     def exists?
-      (@state || meta.fetch('state', nil)) != 'not_found'
+      (@state || meta.fetch("state", nil)) != "not_found"
     end
 
     def valid?
@@ -151,111 +155,111 @@ module Commonmeta
     # Catch errors in the reader
     # Then validate against JSON schema for Commonmeta
     def errors
-      meta.fetch('errors', nil) || json_schema_errors
+      meta.fetch("errors", nil) || json_schema_errors
     end
 
     def descriptions
-      @descriptions ||= meta.fetch('descriptions', nil)
+      @descriptions ||= meta.fetch("descriptions", nil)
     end
 
     def license
-      @license ||= meta.fetch('license', nil)
+      @license ||= meta.fetch("license", nil)
     end
 
     def subjects
-      @subjects ||= meta.fetch('subjects', nil)
+      @subjects ||= meta.fetch("subjects", nil)
     end
 
     def language
-      @language ||= meta.fetch('language', nil)
+      @language ||= meta.fetch("language", nil)
     end
 
     def sizes
-      @sizes ||= meta.fetch('sizes', nil)
+      @sizes ||= meta.fetch("sizes", nil)
     end
 
     def formats
-      @formats ||= meta.fetch('formats', nil)
+      @formats ||= meta.fetch("formats", nil)
     end
 
     def schema_version
-      @schema_version ||= meta.fetch('schema_version', nil)
+      @schema_version ||= meta.fetch("schema_version", nil)
     end
 
     def funding_references
-      @funding_references ||= meta.fetch('funding_references', nil)
+      @funding_references ||= meta.fetch("funding_references", nil)
     end
 
     def references
-      @references ||= meta.fetch('references', nil)
+      @references ||= meta.fetch("references", nil)
     end
 
     def related_identifiers
-      @related_identifiers ||= meta.fetch('related_identifiers', nil)
+      @related_identifiers ||= meta.fetch("related_identifiers", nil)
     end
 
     def related_items
-      @related_items ||= meta.fetch('related_items', nil)
+      @related_items ||= meta.fetch("related_items", nil)
     end
 
     def url
-      @url ||= meta.fetch('url', nil)
+      @url ||= meta.fetch("url", nil)
     end
 
     def version
-      @version ||= meta.fetch('version', nil)
+      @version ||= meta.fetch("version", nil)
     end
 
     def container
-      @container ||= meta.fetch('container', nil)
+      @container ||= meta.fetch("container", nil)
     end
 
     def geo_locations
-      @geo_locations ||= meta.fetch('geo_locations', nil)
+      @geo_locations ||= meta.fetch("geo_locations", nil)
     end
 
     def date
-      @date ||= meta.fetch('date', nil)
+      @date ||= meta.fetch("date", nil)
     end
 
     def publisher
-      @publisher ||= meta.fetch('publisher', nil)
+      @publisher ||= meta.fetch("publisher", nil)
     end
 
     def alternate_identifiers
-      @alternate_identifiers ||= meta.fetch('alternate_identifiers', nil)
+      @alternate_identifiers ||= meta.fetch("alternate_identifiers", nil)
     end
 
     def files
-      @files ||= meta.fetch('files', nil)
+      @files ||= meta.fetch("files", nil)
     end
 
     def provider
-      @provider ||= meta.fetch('provider', nil)
+      @provider ||= meta.fetch("provider", nil)
     end
 
     def state
-      @state ||= meta.fetch('state', nil)
+      @state ||= meta.fetch("state", nil)
     end
 
     def type
-      @type ||= meta.fetch('type', nil)
+      @type ||= meta.fetch("type", nil)
     end
 
     def additional_type
-      @additional_type ||= meta.fetch('additional_type', nil)
+      @additional_type ||= meta.fetch("additional_type", nil)
     end
 
     def titles
-      @titles ||= meta.fetch('titles', nil)
+      @titles ||= meta.fetch("titles", nil)
     end
 
     def contributors
-      @contributors ||= meta.fetch('contributors', nil)
+      @contributors ||= meta.fetch("contributors", nil)
     end
 
     def archive_locations
-      @archive_locations ||= meta.fetch('archive_locations', nil)
+      @archive_locations ||= meta.fetch("archive_locations", nil)
     end
   end
 end

data/lib/commonmeta/readers/json_feed_reader.rb

@@ -69,7 +69,7 @@ module Commonmeta
         funding_references = get_funding_references(meta)
         related_identifiers = get_related_identifiers(meta)
         alternate_identifiers = [{ "alternateIdentifier" => meta["id"], "alternateIdentifierType" => "UUID" }]
-        
+        files = get_files(id)
         archive_url = meta.fetch("archive_url", nil)
 
         if archive_url
@@ -99,10 +99,20 @@ module Commonmeta
           "funding_references" => funding_references.presence,
           "related_identifiers" => related_identifiers.presence,
           "alternate_identifiers" => alternate_identifiers,
+          "files" => files.presence,
           "archive_locations" => archive_locations,
           "state" => state }.compact.merge(read_options)
       end
 
+      def get_files(id)
+        doi = doi_from_url(id)
+        return nil unless is_rogue_scholar_doi?(doi)
+          
+        [{ "mimeType" => "text/markdown", "url" => "https://api.rogue-scholar.org/posts/#{doi}.md" },
+         { "mimeType" => "application/pdf", "url" => "https://api.rogue-scholar.org/posts/#{doi}.pdf" },
+         { "mimeType" => "application/epub+zip", "url" => "https://api.rogue-scholar.org/posts/#{doi}.epub" }]
+      end
+
       def get_references(meta)
         # check that references resolve
         Array.wrap(meta["reference"]).reduce([]) do |sum, reference|
@@ -212,8 +222,8 @@ module Commonmeta
         blog["items"].map { |item| item["id"] }.first
       end
 
-      def get_json_feed_blog_id(id)
-        # get JSON Feed item by id and return blog id
+      def get_json_feed_blog_slug(id)
+        # get JSON Feed item by id and return blog slug
 
         url = json_feed_item_by_id_url(id)
         response = HTTP.get(url)
@@ -223,10 +233,10 @@ module Commonmeta
         post.to_h.dig("blog", "slug")
       end
 
-      def get_doi_prefix_by_blog_id(blog_id)
+      def get_doi_prefix_by_blog_slug(blog_slug)
         # for generating a random DOI.
 
-        url = json_feed_by_blog_url(blog_id)
+        url = json_feed_by_blog_url(blog_slug)
         response = HTTP.get(url)
         return nil unless response.status.success?
 

data/lib/commonmeta/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Commonmeta
-  VERSION = '3.7.2'
+  VERSION = '3.8.0'
 end

data/spec/cli_spec.rb

@@ -379,9 +379,9 @@ describe Commonmeta::CLI do
       expect { subject.json_feed_by_blog input }.to output(/3749f8c4-1ba7-4e51-9dd6-9d9551ad451a/).to_stdout
     end
 
-    it "json_feed_blog_id" do
+    it "json_feed_blog_slug" do
       input = "01a92f9a-f8e0-442b-86e2-11530d9d5635"
-      expect { subject.json_feed_blog_id input }.to output(/eve/).to_stdout
+      expect { subject.json_feed_blog_slug input }.to output(/eve/).to_stdout
     end
   end
 end

data/spec/doi_utils_spec.rb

@@ -315,4 +315,18 @@ describe Commonmeta::Metadata, vcr: true do
       expect(response).to eq('10.5061')
     end
   end
+
+  context 'is_rogue_scholar_doi' do
+    it 'rogue scholar doi' do
+      doi = '10.53731/cjx855h-hn5jtq8'
+      response = subject.is_rogue_scholar_doi?(doi)
+      expect(response).to be true
+    end
+
+    it 'not a rogue scholar doi' do
+      doi = '10.1371/journal.pone.0000030'
+      response = subject.is_rogue_scholar_doi?(doi)
+      expect(response).to be false
+    end
+  end
 end