resync 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: d976bb155b49ce7339df955bfa20d30392d22b5e
+  data.tar.gz: 3c3fdaebce38202912ce7d56805a429578f77b1d
+SHA512:
+  metadata.gz: d567ccc2977f4c46ea357765bb46b72d40a8ee20673c2fdd9f76e66f3cc1e5ca18c16f7fe9cbeb51788121b1f33b70bec6e0ba7b71fcf09eae431ee4a3875afa
+  data.tar.gz: f97193935fc1dc19de8e715a9502e6afc2cd150528eb71a471acb41be24bd7a95ab8e139f9245026d1ee45607e79aeb05ce231a6902599261994659ca14ed3ae

data/.gitignore

@@ -0,0 +1,42 @@
+# Ruby defaults
+
+/.bundle/
+/.yardoc
+/Gemfile.lock
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/tmp/
+*.bundle
+*.so
+*.o
+*.a
+mkmf.log
+
+# Rails engine
+
+.bundle/
+log/*.log
+spec/dummy/db/*.sqlite3
+spec/dummy/db/*.sqlite3-journal
+spec/dummy/log/*.log
+spec/dummy/tmp/
+spec/dummy/.sass-cache
+
+# IntellJ
+
+*.iml
+*.ipr
+*.iws
+.rakeTasks
+.idea
+
+# Emacs
+
+*~
+
+# Mac OS
+
+.DS_Store

data/.rubocop.yml

@@ -0,0 +1,23 @@
+# Disable line-length check; it's too easy for the cure to be worse than the disease
+Metrics/LineLength:
+  Enabled: False
+
+# Disable problematic module documentation check (see https://github.com/bbatsov/rubocop/issues/947)
+Style/Documentation:
+  Enabled: false
+
+# Allow one line around class body (Style/EmptyLines will still disallow two or more)
+Style/EmptyLinesAroundClassBody:
+  Enabled: false
+
+# Allow one line around module body (Style/EmptyLines will still disallow two or more)
+Style/EmptyLinesAroundModuleBody:
+  Enabled: false
+
+# Allow one line around block body (Style/EmptyLines will still disallow two or more)
+Style/EmptyLinesAroundBlockBody:
+  Enabled: false
+
+# Allow %r notation for regexes with a single / character
+Style/RegexpLiteral:
+  MaxSlashes: 0

data/.ruby-version

@@ -0,0 +1 @@
+2.2.2

data/.travis.yml

@@ -0,0 +1,2 @@
+language: ruby
+

data/Gemfile

@@ -0,0 +1,3 @@
+source 'https://rubygems.org'
+
+gemspec

data/LICENSE.md

@@ -0,0 +1,22 @@
+The MIT License (MIT)
+
+Copyright (c) 2015 The Regents of the University of California
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,92 @@
+# resync
+
+A Ruby gem for working with the [ResourceSync](http://www.openarchives.org/rs/1.0/resourcesync) web synchronization framework.
+
+It consists of the following:
+
+  - Classes corresponding to the major document types defined in the ResourceSync specification, such as [Resource Lists](http://www.openarchives.org/rs/1.0/resourcesync#ResourceList), [Change Lists](http://www.openarchives.org/rs/1.0/resourcesync#ChangeList), [Source Descriptions](http://www.openarchives.org/rs/1.0/resourcesync#SourceDesc) and so on. Each of these classes has a `load_from_xml` method that can parse the corresponding XML document (as an `REXML::Element`), and a `save_to_xml` method that can serialize an instance of that class to XML (as an `REXML::Element`).
+  - Classes for the [major sub-structures](http://www.openarchives.org/rs/1.0/resourcesync#DocumentFormats) of those documents, such as the `<url>` and `<sitemap>` tags (subsumed under the [Resource](lib/resync/resource.rb) class) defined by the Sitemap specification, as well as the ResourceSync-specific `<rs:ln>` and `<rs:md>` tags (the [Link](lib/resync/link.rb) and [Metadata](lib/resync/metadata.rb) classes, respectively).
+  - An [XMLParser](lib/resync/xml_parser.rb) class that can take a ResourceSync-augmented Sitemap document (in the form of an `REXML::Element`, an `REXML::Document`, a string, an `IO`, or something sufficiently `IO`-like that `REXML::Document` can parse it) and produce an instance of the appropriate class based on the `capability` attribute in the root element's metadata.
+
+## Status
+
+This is a work in progress. We welcome bug reports and feature requests (particularly on the document creation side, which our use cases haven't really explored).
+
+## Usage
+
+### Parsing a ResourceSync document
+
+```ruby
+require 'resync'
+
+data = File.read('my-capability-list.xml')
+capability_list = Resync::XMLParser.parse(data)
+```
+
+### Writing a ResourceSync document
+
+```ruby
+require 'resync'
+
+change_list = Resync::ChangeList.new(
+  links: [ Resync::Link.new(rel='up', href='http://example.com/my-dataset/my-capability-list.xml') ],
+  metadata: Resync::Metadata.new(
+    capability = 'changelist',
+    from_time = Time.utc(2013, 1, 3)
+  )
+  resources: [
+    # ... generate list of changes here ...
+  ]
+)
+xml = change_list.save_to_xml
+formatter = REXML::Formatters::Pretty.new
+formatter.write(xml, $stdout)
+```
+
+## See also
+
+[resync-client](https://github.com/dmolesUC3/resync-client), a Ruby client library for ResourceSync.
+
+## Limitations
+
+### Structural inconvenience and unnecessary repetition
+
+There are certain well-specified relationships between elements: most document types should always have a link with an `up` relationship, many resources should have metadata with a defined `capability` attribute, and so on. In some cases there are convenience getters for these attributes on the 'parent' object (e.g. you can ask for the `capability` directly without violating the law of Demeter), but there generally aren't corresponding convenience setters, or convenience initializer parameters.
+
+Document types (`ChangeList`, `ResourceList`, etc.) will create a `Metadata` with the appropriate capability for themselves if none is specified, but if they're initialized with one that doesn't declare a capability, they'll raise an exception rather than fill it in (just as they'll raise an exception if the wrong capability is specified).
+
+### Logical relationships between elements
+
+A `ChangeList` should contain only resources with `Metadata` declaring a `change` type. The resources in a `ResourceDumpManifest` should each declare a `path` indicating their locations in the ZIP file. `resync` doesn't currently do anything to enforce, validate, or assist in compliance with these and similar restrictions.
+
+(An exception: document types will complain if initialized with `Metadata` having the wrong capability.)
+
+### Time attribute requirements
+
+The required/forbidden time attributes defined in Appendix A,
+"[Time Attribute Requirements](http://www.openarchives.org/rs/1.0/resourcesync#TimeAttributeReqs)",
+of the ResourceSync specification are not enforced; it's possible to
+create, e.g., a `ResourceList` with a `from_time` on its metadata, or a `ChangeList` with members whose metadata does not declare a `modified_time`, even though both scenarios are forbidden by the specification.
+
+### Value restrictions from XML schemata
+
+The [ResourceSync schema](http://www.openarchives.org/rs/0.9.1/resourcesync.xsd) defines restrictions on the values of several attributes:
+
+- Path values must start with a slash, must not end with a slash
+- Priorities must be positive and < 1,000,000
+- Link relation types must conform with [RFC 5988](http://tools.ietf.org/html/rfc5988)
+
+The [Sitemap](http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd) and [Sitemap index](http://www.sitemaps.org/schemas/sitemap/0.9/siteindex.xsd) schemas also define some restrictions:
+
+- URIs have a minimum length of 12 and a max of 2048 characters.
+- Priorities must be in the range 0.0-1.0 (inclusive)
+
+None of these restrictions are currently enforced by `resync`.
+
+### Element order
+
+When reading a ResourceSync document from XML and writing it back out, `<rs:ln>` elements will always appear before `<rs:md>` elements, regardless of their order in the original source.
+
+### Namespace weirdness
+
+The [XML::Mapping](https://github.com/multi-io/xml-mapping) library `resync` uses doesn't support namespaces, so namespace handling in `resync` is a bit hacky. In particular, you may see strange behavior when using `<rs:ln>`, `<rs:md>`, `<url>`, or `<sitemap>` tags outside the context of a `<urlset>`/`<sitemapindex>`.

data/Rakefile

@@ -0,0 +1,56 @@
+# ------------------------------------------------------------
+# RSpec
+
+require 'rspec/core'
+require 'rspec/core/rake_task'
+
+namespace :spec do
+
+  desc 'Run all unit tests'
+  RSpec::Core::RakeTask.new(:unit) do |task|
+    task.rspec_opts = %w(--color --format documentation --order default)
+    task.pattern = 'unit/**/*_spec.rb'
+  end
+
+  desc 'Run all acceptance tests'
+  RSpec::Core::RakeTask.new(:acceptance) do |task|
+    ENV['COVERAGE'] = nil
+    task.rspec_opts = %w(--color --format documentation --order default)
+    task.pattern = 'acceptance/**/*_spec.rb'
+  end
+
+  task all: [:unit, :acceptance]
+end
+
+desc 'Run all tests'
+task spec: 'spec:all'
+
+# ------------------------------------------------------------
+# Coverage
+
+desc 'Run all unit tests with coverage'
+task :coverage do
+  ENV['COVERAGE'] = 'true'
+  Rake::Task['spec:unit'].execute
+end
+
+# ------------------------------------------------------------
+# RuboCop
+
+require 'rubocop/rake_task'
+RuboCop::RakeTask.new
+
+# ------------------------------------------------------------
+# TODOs
+
+desc 'List TODOs (from spec/todo.rb)'
+RSpec::Core::RakeTask.new(:todo) do |task|
+  task.rspec_opts = %w(--color --format documentation --order default)
+  task.pattern = 'todo.rb'
+end
+
+# ------------------------------------------------------------
+# Defaults
+
+desc 'Run unit tests, check test coverage, run acceptance tests, check code style'
+task default: [:coverage, 'spec:acceptance', :rubocop]

data/example.rb

@@ -0,0 +1,100 @@
+#!/usr/bin/env ruby
+
+# Note: This assumes we're running from the root of the resync project
+$LOAD_PATH << File.dirname(__FILE__)
+require 'lib/resync'
+
+# ------------------------------------------------------------
+# Reading a capability list
+
+puts "\n------------------------------------------------------------"
+puts "A capability list:\n"
+
+capabilitylist_xml = '<?xml version="1.0" encoding="UTF-8"?>
+        <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
+                xmlns:rs="http://www.openarchives.org/rs/terms/">
+          <rs:ln rel="describedby"
+                 href="http://example.com/info_about_set1_of_resources.xml"/>
+          <rs:ln rel="up"
+                 href="http://example.com/resourcesync_description.xml"/>
+          <rs:md capability="capabilitylist"/>
+          <url>
+            <loc>http://example.com/dataset1/resourcelist.xml</loc>
+            <rs:md capability="resourcelist"/>
+          </url>
+          <url>
+            <loc>http://example.com/dataset1/resourcedump.xml</loc>
+            <rs:md capability="resourcedump"/>
+          </url>
+          <url>
+            <loc>http://example.com/dataset1/changelist.xml</loc>
+            <rs:md capability="changelist"/>
+          </url>
+          <url>
+            <loc>http://example.com/dataset1/changedump.xml</loc>
+            <rs:md capability="changedump"/>
+          </url>
+        </urlset>'
+
+capability_list = Resync::XMLParser.parse(capabilitylist_xml)
+puts '  Links:'
+capability_list.links.each do |l|
+  puts "    #{l.rel}: #{l.uri}"
+end
+puts '  Resources:'
+capability_list.resources.each do |r|
+  puts "    #{r.uri} (#{r.capability})"
+end
+
+# ------------------------------------------------------------
+# Creating a changelist
+
+puts "\n------------------------------------------------------------"
+puts "A change list:\n\n"
+
+change_list = Resync::ChangeList.new(
+  links: [
+    Resync::Link.new(
+      rel: 'up',
+      uri: 'http://example.com/dataset1/capabilitylist.xml'
+    )
+  ],
+  metadata: Resync::Metadata.new(
+    capability: 'changelist',
+    from_time: Time.utc(2013, 1, 3)
+  ),
+  resources: [
+    Resync::Resource.new(
+      uri: 'http://example.com/res4',
+      modified_time: Time.utc(2013, 1, 3, 17),
+      metadata: Resync::Metadata.new(
+        change: Resync::Types::Change::UPDATED,
+        hashes: { 'sha-256' => 'f4OxZX_x_DFGFDgghgdfb6rtSx-iosjf6735432nklj' },
+        length: 56_778,
+        mime_type: 'application/json'
+      ),
+      links: [
+        Resync::Link.new(
+          rel: 'http://www.openarchives.org/rs/terms/patch',
+          uri: 'http://example.com/res4-json-patch',
+          modified_time: Time.utc(2013, 1, 3, 17),
+          hashes: { 'sha-256' => 'f4OxZX_x_DFGFDgghgdfb6rtSx-iosjf6735432nklj' },
+          length: 73,
+          mime_type: 'application/json-patch'
+        )
+      ]
+    ),
+    Resync::Resource.new(
+      uri: 'http://example.com/res5-full.tiff',
+      modified_time: Time.utc(2013, 1, 3, 18),
+      metadata: Resync::Metadata.new(
+        change: Resync::Types::Change::DELETED
+      )
+    )
+  ]
+)
+
+xml = change_list.save_to_xml
+formatter = REXML::Formatters::Pretty.new
+formatter.write(xml, $stdout)
+puts

data/lib/resync/capability_list.rb

@@ -0,0 +1,85 @@
+require_relative 'shared/base_resource_list'
+require_relative 'xml'
+
+module Resync
+  # A capability list. See section 9,
+  # "{http://www.openarchives.org/rs/1.0/resourcesync#CapabilityList Advertising Capabilities}",
+  # in the ResourceSync specification.
+  class CapabilityList < BaseResourceList
+    include ::XML::Mapping
+
+    # The capability provided by this type.
+    CAPABILITY = 'capabilitylist'
+
+    # ------------------------------------------------------------
+    # Initializer
+
+    # Creates a new +BaseResourceList+.
+    #
+    # @param resources [Array<Resource>] The +<url>+ or +<sitemap>+ elements contained in this list.
+    #   All resources must have a capability, and there can be no more than one resource for each
+    #   specified capability.
+    # @param links [Array<Link>] Related links (+<rs:ln>+).
+    # @param metadata [Metadata] Metadata about this list. The +capability+ of the metadata must
+    #   be +'capabilitylist'+.
+    # @raise [ArgumentError] if a provided resource does not have a +capability+ attribute.
+    # @raise [ArgumentError] if more than one provided resource has the same +capability+ attribute.
+    # @raise [ArgumentError] if the specified metadata does not have the correct +capability+ attribute.
+    def initialize(resources: [], links: [], metadata: nil)
+      @source_descripton = source_description_from(links)
+      super(resources: resources, links: links, metadata: metadata)
+    end
+
+    # ------------------------------------------------------------
+    # Custom accessors
+
+    # Sets the +resources+ list. +nil+ is treated as an empty list.
+    # @raise [ArgumentError] if a provided resource does not have a +capability+ attribute.
+    # @raise [ArgumentError] if more than one provided resource has the same +capability+ attribute.
+    def resources=(value)
+      resources = value || []
+      @capabilities = to_capability_map(resources)
+      @resources = @capabilities.values
+    end
+
+    # Gets the resource for the specified capability.
+    #
+    # @param capability [String] The capability.
+    # @return [Resource] the resource providing the capability, or +nil+ if
+    #   there is no resource with that capability in this list.
+    def resource_for(capability:)
+      @capabilities[capability]
+    end
+
+    # Gets the URI of the description of the source whose capabilities are identified by this list.
+    #
+    # @return [URI] the URI of the description of the source whose capabilities are identified
+    #   by this list. See section 8,
+    #   "{http://www.openarchives.org/rs/1.0/resourcesync#SourceDesc Describing the Source}",
+    #   in the ResourceSync specification.
+    def source_description
+      @source_description ||= source_description_from(links)
+    end
+
+    # ------------------------------
+    # Conversions
+
+    def to_capability_map(resources)
+      capabilities = {}
+      (resources || []).each do |resource|
+        capability = resource.capability
+        fail ArgumentError, "No capability found for resource with URI #{resource.uri}" unless capability
+        fail ArgumentError, "Duplicate resource for capability #{capability}" if capabilities.key?(capability)
+        capabilities[capability] = resource
+      end
+      capabilities
+    end
+
+    def source_description_from(links)
+      return nil unless links
+      desc = links.map { |link| link.uri if link.rel == 'up' }.compact.first
+      fail ArgumentError, "No source descrption (<link rel='up'/>) provided" unless desc
+      desc
+    end
+  end
+end

data/lib/resync/change_dump.rb

@@ -0,0 +1,15 @@
+require_relative 'shared/sorted_resource_list'
+require_relative 'xml'
+
+module Resync
+  # A change dump. See section 13.1,
+  # "{http://www.openarchives.org/rs/1.0/resourcesync#ChangeDump Change Dump}",
+  # in the ResourceSync specification.
+  class ChangeDump < SortedResourceList
+    include ::XML::Mapping
+
+    # The capability provided by this type.
+    CAPABILITY = 'changedump'
+
+  end
+end

data/lib/resync/change_dump_manifest.rb

@@ -0,0 +1,15 @@
+require_relative 'shared/sorted_resource_list'
+require_relative 'xml'
+
+module Resync
+  # A change dump manifest. See section 13.2,
+  # "{http://www.openarchives.org/rs/1.0/resourcesync#ChangeDumpManifest Change Dump Manifest}",
+  # in the ResourceSync specification.
+  class ChangeDumpManifest < SortedResourceList
+    include ::XML::Mapping
+
+    # The capability provided by this type.
+    CAPABILITY = 'changedump-manifest'
+
+  end
+end

data/lib/resync/change_list.rb

@@ -0,0 +1,15 @@
+require_relative 'shared/sorted_resource_list'
+require_relative 'xml'
+
+module Resync
+  # A change list. See section 12.1,
+  # "{http://www.openarchives.org/rs/1.0/resourcesync#ChangeList Change List}",
+  # in the ResourceSync specification.
+  class ChangeList < SortedResourceList
+    include ::XML::Mapping
+
+    # The capability provided by this type.
+    CAPABILITY = 'changelist'
+
+  end
+end

data/lib/resync/change_list_index.rb

@@ -0,0 +1,26 @@
+require_relative 'shared/sorted_resource_list'
+require_relative 'shared/sitemap_index'
+
+module Resync
+  # A change list index. See section 12.2,
+  # "{http://www.openarchives.org/rs/1.0/resourcesync#ChangeListIndex Change List Index}",
+  # in the ResourceSync specification.
+  class ChangeListIndex < SortedResourceList
+    include ::XML::Mapping
+    include SitemapIndex
+
+    # The capability provided by this type.
+    CAPABILITY = 'changelist'
+
+    # use_mapping :sitemapindex
+    # root_element_name 'sitemapindex'
+    # array_node :resources, 'sitemap', class: Resource, default_value: [], sub_mapping: :_default
+    #
+    # # Ensures that an index is always written as a +<sitemapindex>+.
+    # # Overrides +::XML::Mapping.save_to_xml+.
+    # def save_to_xml(options = { mapping: :_default })
+    #   options = options.merge(mapping: :sitemapindex)
+    #   super(options)
+    # end
+  end
+end

data/lib/resync/link.rb

@@ -0,0 +1,87 @@
+require_relative 'shared/descriptor'
+require_relative 'xml'
+
+module Resync
+
+  # A link to a related resource. See "Linking to Related Resources"
+  # under section 5.1,
+  # {http://www.openarchives.org/rs/1.0/resourcesync#SourcePers Synchronization Processes: Source Perspective}
+  # in the ResourceSync specification.
+  #
+  # @!attribute [rw] rel
+  #   @return [String] the relationship of the linked resource to the original
+  #     resource. See {http://tools.ietf.org/html/rfc5988 RFC 5988}, "Web Linking".
+  #     for information on link relation types.
+  # @!attribute [rw] uri
+  #   @return [URI] the URI of the linked resource.
+  # @!attribute [rw] priority
+  #   @return [Integer] the priority of the linked resource among links with the
+  #     same relation type. Values should be in the range 1-999,999 (inclusive).
+  #     Lower values indicate higher priorities.
+  class Link < Descriptor
+    include ::XML::Mapping
+
+    # ------------------------------------------------------------
+    # Attributes
+
+    root_element_name 'ln'
+
+    text_node :rel, '@rel', default_value: nil
+    uri_node :uri, '@href', default_value: nil
+    numeric_node :priority, '@pri', default_value: nil
+
+    # ------------------------------------------------------------
+    # Initializer
+
+    # @param rel [String] the relationship of the linked resource to the
+    #   original resource. See {http://tools.ietf.org/html/rfc5988 RFC 5988},
+    #   "Web Linking". for information on link relation types.
+    # @param uri [URI] the URI of the linked resource.
+    # @param priority [Integer] the priority of the linked resource among links
+    #   with the same relation type. Values should be in the range
+    #   1-999,999 (inclusive). Lower values indicate higher priorities.
+    # @param modified_time [Time] The date and time when the referenced resource was last modified.
+    # @param length [Integer] The content length of the referenced resource.
+    # @param mime_type [MIME::Type] The media type of the referenced resource.
+    # @param encoding [String] Any content encoding (if any) applied to the data in the
+    #   referenced resource (e.g. for compression)
+    # @param hashes [Hash<String, String>] Fixity information for the referenced
+    #   resource, as a map from hash algorithm tokens (e.g. +md5+, +sha-256+)
+    #   to hex-encoded digest values.
+    # @param path [String] For +ResourceDumpManifests+ and +ChangeDumpManifests+,
+    #   the path to the referenced resource within the dump ZIP file.
+    # @raise [URI::InvalidURIError] if +uri+ cannot be converted to a URI.
+    def initialize( # rubocop:disable Metrics/MethodLength, Metrics/ParameterLists
+        rel:,
+        uri:,
+
+        priority: nil,
+
+        modified_time: nil,
+
+        length: nil,
+        mime_type: nil,
+        encoding: nil,
+        hashes: {},
+
+        path: nil
+    )
+      super(modified_time: modified_time, length: length, mime_type: mime_type, encoding: encoding, hashes: hashes, path: path)
+
+      self.rel = rel
+      self.uri = uri
+      self.priority = priority
+    end
+
+    # ------------------------------------------------------------
+    # Custom setters
+
+    # Sets the URI of the linked resource. Strings will be converted to +URI+ objects.
+    # @param value [URI, String] the URI.
+    # @raise [URI::InvalidURIError] if +value+ cannot be converted to a URI.
+    def uri=(value)
+      @uri = XML.to_uri(value)
+    end
+
+  end
+end