resync 0.1.0 → 0.1.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: d976bb155b49ce7339df955bfa20d30392d22b5e
-  data.tar.gz: 3c3fdaebce38202912ce7d56805a429578f77b1d
+  metadata.gz: 1ac849438f247e3776ef21a93c6466e83e3bb45f
+  data.tar.gz: 184a1d96dd47fd6bc0c1b413d7cbc318411e5e3c
 SHA512:
-  metadata.gz: d567ccc2977f4c46ea357765bb46b72d40a8ee20673c2fdd9f76e66f3cc1e5ca18c16f7fe9cbeb51788121b1f33b70bec6e0ba7b71fcf09eae431ee4a3875afa
-  data.tar.gz: f97193935fc1dc19de8e715a9502e6afc2cd150528eb71a471acb41be24bd7a95ab8e139f9245026d1ee45607e79aeb05ce231a6902599261994659ca14ed3ae
+  metadata.gz: 1cdd7b2571b0caaea6aaeff82212072edbfc5f3b020d4c924c0cec227fa8f2c0ab8c1c1f1ee88171f8d2ca5c1ce0379c01db56ae40013db9375354c3a6533a41
+  data.tar.gz: ac5c5df6d0c6f3c02f0dbde7c6a4c79b9b0aeb340a46a33cf3e01525f8383805eb849db0a0e8bda7a72fa11d7c6b46638a9dac00f16357ba6d7f92a6741e7f3d

data/CHANGES.md

@@ -0,0 +1,8 @@
+## 0.1.1
+
+- Add support for [Change Dump Indices](http://www.openarchives.org/rs/1.0/resourcesync#ChangeDumpIndex)
+- Add support for [Resource Dump Indices](http://www.openarchives.org/rs/1.0/resourcesync#ResourceDumpIndex)
+
+## 0.1.0
+
+- Initial release

data/README.md

@@ -8,10 +8,6 @@ It consists of the following:
   - Classes for the [major sub-structures](http://www.openarchives.org/rs/1.0/resourcesync#DocumentFormats) of those documents, such as the `<url>` and `<sitemap>` tags (subsumed under the [Resource](lib/resync/resource.rb) class) defined by the Sitemap specification, as well as the ResourceSync-specific `<rs:ln>` and `<rs:md>` tags (the [Link](lib/resync/link.rb) and [Metadata](lib/resync/metadata.rb) classes, respectively).
   - An [XMLParser](lib/resync/xml_parser.rb) class that can take a ResourceSync-augmented Sitemap document (in the form of an `REXML::Element`, an `REXML::Document`, a string, an `IO`, or something sufficiently `IO`-like that `REXML::Document` can parse it) and produce an instance of the appropriate class based on the `capability` attribute in the root element's metadata.
 
-## Status
-
-This is a work in progress. We welcome bug reports and feature requests (particularly on the document creation side, which our use cases haven't really explored).
-
 ## Usage
 
 ### Parsing a ResourceSync document
@@ -29,10 +25,10 @@ capability_list = Resync::XMLParser.parse(data)
 require 'resync'
 
 change_list = Resync::ChangeList.new(
-  links: [ Resync::Link.new(rel='up', href='http://example.com/my-dataset/my-capability-list.xml') ],
+  links: [ Resync::Link.new(rel: 'up', href: 'http://example.com/my-dataset/my-capability-list.xml') ],
   metadata: Resync::Metadata.new(
-    capability = 'changelist',
-    from_time = Time.utc(2013, 1, 3)
+    capability: 'changelist',
+    from_time: Time.utc(2013, 1, 3)
   )
   resources: [
     # ... generate list of changes here ...
@@ -47,28 +43,32 @@ formatter.write(xml, $stdout)
 
 [resync-client](https://github.com/dmolesUC3/resync-client), a Ruby client library for ResourceSync.
 
-## Limitations
+## Status
+
+This is a work in progress. Bug reports and feature requests are welcome (particularly on the document creation side, which our use cases haven't really explored).
+
+### Known limitations
 
-### Structural inconvenience and unnecessary repetition
+#### Structural inconvenience and unnecessary repetition
 
 There are certain well-specified relationships between elements: most document types should always have a link with an `up` relationship, many resources should have metadata with a defined `capability` attribute, and so on. In some cases there are convenience getters for these attributes on the 'parent' object (e.g. you can ask for the `capability` directly without violating the law of Demeter), but there generally aren't corresponding convenience setters, or convenience initializer parameters.
 
 Document types (`ChangeList`, `ResourceList`, etc.) will create a `Metadata` with the appropriate capability for themselves if none is specified, but if they're initialized with one that doesn't declare a capability, they'll raise an exception rather than fill it in (just as they'll raise an exception if the wrong capability is specified).
 
-### Logical relationships between elements
+#### Logical relationships between elements
 
 A `ChangeList` should contain only resources with `Metadata` declaring a `change` type. The resources in a `ResourceDumpManifest` should each declare a `path` indicating their locations in the ZIP file. `resync` doesn't currently do anything to enforce, validate, or assist in compliance with these and similar restrictions.
 
 (An exception: document types will complain if initialized with `Metadata` having the wrong capability.)
 
-### Time attribute requirements
+#### Time attribute requirements
 
 The required/forbidden time attributes defined in Appendix A,
 "[Time Attribute Requirements](http://www.openarchives.org/rs/1.0/resourcesync#TimeAttributeReqs)",
 of the ResourceSync specification are not enforced; it's possible to
 create, e.g., a `ResourceList` with a `from_time` on its metadata, or a `ChangeList` with members whose metadata does not declare a `modified_time`, even though both scenarios are forbidden by the specification.
 
-### Value restrictions from XML schemata
+#### Value restrictions from XML schemata
 
 The [ResourceSync schema](http://www.openarchives.org/rs/0.9.1/resourcesync.xsd) defines restrictions on the values of several attributes:
 
@@ -83,10 +83,10 @@ The [Sitemap](http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd) and [Site
 
 None of these restrictions are currently enforced by `resync`.
 
-### Element order
+#### Element order
 
 When reading a ResourceSync document from XML and writing it back out, `<rs:ln>` elements will always appear before `<rs:md>` elements, regardless of their order in the original source.
 
-### Namespace weirdness
+#### Namespace weirdness
 
 The [XML::Mapping](https://github.com/multi-io/xml-mapping) library `resync` uses doesn't support namespaces, so namespace handling in `resync` is a bit hacky. In particular, you may see strange behavior when using `<rs:ln>`, `<rs:md>`, `<url>`, or `<sitemap>` tags outside the context of a `<urlset>`/`<sitemapindex>`.

data/lib/resync/capability_list.rb

@@ -64,6 +64,8 @@ module Resync
     # ------------------------------
     # Conversions
 
+    private
+
     def to_capability_map(resources)
       capabilities = {}
       (resources || []).each do |resource|

data/lib/resync/change_dump_index.rb

@@ -0,0 +1,15 @@
+require_relative 'shared/sorted_resource_list'
+require_relative 'shared/sitemap_index'
+
+module Resync
+  # A change dump index. See under section 13.1,
+  # "{http://www.openarchives.org/rs/1.0/resourcesync#ChangeDumpIndex Change Dump}",
+  # in the ResourceSync specification.
+  class ChangeDumpIndex < SortedResourceList
+    include ::XML::Mapping
+    include SitemapIndex
+
+    # The capability provided by this type.
+    CAPABILITY = 'changedump'
+  end
+end

data/lib/resync/change_list_index.rb

@@ -11,16 +11,5 @@ module Resync
 
     # The capability provided by this type.
     CAPABILITY = 'changelist'
-
-    # use_mapping :sitemapindex
-    # root_element_name 'sitemapindex'
-    # array_node :resources, 'sitemap', class: Resource, default_value: [], sub_mapping: :_default
-    #
-    # # Ensures that an index is always written as a +<sitemapindex>+.
-    # # Overrides +::XML::Mapping.save_to_xml+.
-    # def save_to_xml(options = { mapping: :_default })
-    #   options = options.merge(mapping: :sitemapindex)
-    #   super(options)
-    # end
   end
 end

data/lib/resync/resource_dump_index.rb

@@ -0,0 +1,15 @@
+require_relative 'shared/base_resource_list'
+require_relative 'shared/sitemap_index'
+
+module Resync
+  # A resource dump index. See under section 11.1,
+  # "{http://www.openarchives.org/rs/1.0/resourcesync#ResourceDumpIndex Resource Dump}",
+  # in the ResourceSync specification.
+  class ResourceDumpIndex < BaseResourceList
+    include ::XML::Mapping
+    include SitemapIndex
+
+    # The capability provided by this type.
+    CAPABILITY = 'resourcedump'
+  end
+end

data/lib/resync/shared/base_resource_list.rb

@@ -54,6 +54,7 @@ module Resync
     # ------------------------------------------------------------
     # Custom getters
 
+    # @return [String] the capability provided by this document
     def capability
       @metadata.capability
     end

data/lib/resync/shared/sorted_resource_list.rb

@@ -19,13 +19,20 @@ module Resync
     # ------------------------------------------------------------
     # Custom accessors
 
+    # @return [Hash<URI, Array<Resource>>] resources grouped by URI.
+    #   Order is not guaranteed.
     attr_reader :resources_by_uri
 
+    # @param uri [URI] the URI of the resource
+    # @return [Resource] the resource with the most recent modified time
+    #   for the specified URI.
     def latest_for(uri:)
       uri = XML.to_uri(uri)
       @resources_by_uri[uri].last
     end
 
+    # @return [Array<URI>] the set of all URIs for which this list has
+    #   resources. Order is not guaranteed.
     def all_uris
       @resources_by_uri.keys
     end

data/lib/resync/types/change.rb

@@ -2,7 +2,8 @@ require 'ruby-enum'
 
 module Resync
   module Types
-    # The type of change indicated by a reference in a +ChangeList+, +ChangeDump+, etc.
+    # The type of change indicated by a reference in a {ChangeList}, {ChangeDump}, etc.
+    # Values: +CREATED+, +UPDATED+, +DELETED+
     class Change
       include Ruby::Enum
 

data/lib/resync/types/change_frequency.rb

@@ -3,6 +3,7 @@ require 'ruby-enum'
 module Resync
   module Types
     # The frequency of changes to a resource.
+    # Values: +ALWAYS+, +HOURLY+, +DAILY+, +WEEKLY+, +MONTHLY+, +YEARLY+, +NEVER+
     class ChangeFrequency
       include Ruby::Enum
 

data/lib/resync/version.rb

@@ -1,4 +1,4 @@
 module Resync
   # The version of this gem.
-  VERSION = '0.1.0'
+  VERSION = '0.1.1'
 end

data/lib/resync/xml_parser.rb

@@ -17,7 +17,9 @@ module Resync
       ],
       sitemapindex: [
         ChangeListIndex,
-        ResourceListIndex
+        ResourceListIndex,
+        ChangeDumpIndex,
+        ResourceDumpIndex
       ]
     }
     private_constant :ROOT_TYPES

data/spec/acceptance/xml_parser_spec.rb

@@ -1045,5 +1045,62 @@ module Resync
       expect(ln.length).to eq(8876)
       expect(ln.mime_type).to be_mime_type('text/html')
     end
+
+    it 'parses a change dump index' do
+      data = File.read('spec/data/changedump-index.xml')
+      sitemapindex = XMLParser.parse(data)
+      expect(sitemapindex).to be_a(ChangeDumpIndex)
+
+      links = sitemapindex.links
+      expect(links.size).to eq(1)
+      ln0 = links[0]
+      expect(ln0.rel).to eq('up')
+      expect(ln0.uri).to eq(URI('http://example.com/dataset1/capabilitylist.xml'))
+
+      md = sitemapindex.metadata
+      expect(md.capability).to eq('changedump')
+      expect(md.from_time).to be_time(Time.utc(2013, 1, 1))
+
+      sitemaps = sitemapindex.resources
+      expect(sitemaps.size).to eq(3)
+
+      expected_froms = [Time.utc(2013, 1, 1), Time.utc(2013, 1, 2), Time.utc(2013, 1, 3)]
+      expected_untils = [Time.utc(2013, 1, 2), Time.utc(2013, 1, 3), nil]
+      (0..2).each do |i|
+        sitemap = sitemaps[i]
+        expect(sitemap.uri).to eq(URI("http://example.com/2013010#{i + 1}-changedump.xml"))
+        md = sitemap.metadata
+        expect(md.from_time).to be_time(expected_froms[i])
+        expect(md.until_time).to be_time(expected_untils[i])
+      end
+    end
+
+    it 'parses a resource dump index' do
+      data = File.read('spec/data/resourcedump-index.xml')
+      sitemapindex = XMLParser.parse(data)
+      expect(sitemapindex).to be_a(ResourceDumpIndex)
+
+      links = sitemapindex.links
+      expect(links.size).to eq(1)
+      ln0 = links[0]
+      expect(ln0.rel).to eq('up')
+      expect(ln0.uri).to eq(URI('http://example.com/dataset1/capabilitylist.xml'))
+
+      md = sitemapindex.metadata
+      expect(md.capability).to eq('resourcedump')
+      expect(md.at_time).to be_time(Time.utc(2013, 1, 3, 9))
+      expect(md.completed_time).to be_time(Time.utc(2013, 1, 3, 9, 10))
+
+      sitemaps = sitemapindex.resources
+      expect(sitemaps.size).to eq(3)
+
+      expected_times = [Time.utc(2013, 1, 3, 9), Time.utc(2013, 1, 3, 9, 3), Time.utc(2013, 1, 3, 9, 7)]
+      (0..2).each do |i|
+        sitemap = sitemaps[i]
+        expect(sitemap.uri).to eq(URI("http://example.com/resourcedump#{i + 1}.xml"))
+        md = sitemap.metadata
+        expect(md.at_time).to be_time(expected_times[i])
+      end
+    end
   end
 end

data/spec/data/changedump-index.xml

@@ -0,0 +1,22 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
+              xmlns:rs="http://www.openarchives.org/rs/terms/">
+  <rs:ln rel="up"
+         href="http://example.com/dataset1/capabilitylist.xml"/>
+  <rs:md capability="changedump"
+         from="2013-01-01T00:00:00Z"/>
+  <sitemap>
+    <loc>http://example.com/20130101-changedump.xml</loc>
+    <rs:md from="2013-01-01T00:00:00Z"
+           until="2013-01-02T00:00:00Z"/>
+  </sitemap>
+  <sitemap>
+    <loc>http://example.com/20130102-changedump.xml</loc>
+    <rs:md from="2013-01-02T00:00:00Z"
+           until="2013-01-03T00:00:00Z"/>
+  </sitemap>
+  <sitemap>
+    <loc>http://example.com/20130103-changedump.xml</loc>
+    <rs:md from="2013-01-03T00:00:00Z"/>
+  </sitemap>
+</sitemapindex>

data/spec/data/resourcedump-index.xml

@@ -0,0 +1,21 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
+              xmlns:rs="http://www.openarchives.org/rs/terms/">
+  <rs:ln rel="up"
+         href="http://example.com/dataset1/capabilitylist.xml"/>
+  <rs:md capability="resourcedump"
+         at="2013-01-03T09:00:00Z"
+         completed="2013-01-03T09:10:00Z"/>
+  <sitemap>
+    <loc>http://example.com/resourcedump1.xml</loc>
+    <rs:md at="2013-01-03T09:00:00Z"/>
+  </sitemap>
+  <sitemap>
+    <loc>http://example.com/resourcedump2.xml</loc>
+    <rs:md at="2013-01-03T09:03:00Z"/>
+  </sitemap>
+  <sitemap>
+    <loc>http://example.com/resourcedump3.xml</loc>
+    <rs:md at="2013-01-03T09:07:00Z"/>
+  </sitemap>
+</sitemapindex>

data/spec/unit/resync/change_dump_index_spec.rb

@@ -0,0 +1,49 @@
+require_relative 'shared/sorted_list_examples'
+
+module Resync
+  describe ChangeDumpIndex do
+    it_behaves_like SortedResourceList
+
+    describe 'XML conversion' do
+      describe '#from_xml' do
+        it 'parses an XML string' do
+
+          xml = File.read('spec/data/changedump-index.xml')
+          sitemapindex = ChangeDumpIndex.load_from_xml(XML.element(xml))
+
+          links = sitemapindex.links
+          expect(links.size).to eq(1)
+          ln0 = links[0]
+          expect(ln0.rel).to eq('up')
+          expect(ln0.uri).to eq(URI('http://example.com/dataset1/capabilitylist.xml'))
+
+          md = sitemapindex.metadata
+          expect(md.capability).to eq('changedump')
+          expect(md.from_time).to be_time(Time.utc(2013, 1, 1))
+
+          sitemaps = sitemapindex.resources
+          expect(sitemaps.size).to eq(3)
+
+          expected_froms = [Time.utc(2013, 1, 1), Time.utc(2013, 1, 2), Time.utc(2013, 1, 3)]
+          expected_untils = [Time.utc(2013, 1, 2), Time.utc(2013, 1, 3), nil]
+          (0..2).each do |i|
+            sitemap = sitemaps[i]
+            expect(sitemap.uri).to eq(URI("http://example.com/2013010#{i + 1}-changedump.xml"))
+            md = sitemap.metadata
+            expect(md.from_time).to be_time(expected_froms[i])
+            expect(md.until_time).to be_time(expected_untils[i])
+          end
+        end
+      end
+
+      describe '#save_to_xml' do
+        it 'can round-trip to XML' do
+          data = File.read('spec/data/changedump-index.xml')
+          list = ChangeDumpIndex.load_from_xml(XML.element(data))
+          xml = list.save_to_xml
+          expect(xml).to be_xml(data)
+        end
+      end
+    end
+  end
+end

data/spec/unit/resync/resource_dump_index_spec.rb

@@ -0,0 +1,53 @@
+require_relative 'shared/base_resource_list_examples'
+
+module Resync
+  describe ResourceListIndex do
+    it_behaves_like BaseResourceList
+
+    describe 'links' do
+      it_behaves_like Augmented
+    end
+
+    describe 'XML conversion' do
+      describe '#from_xml' do
+        it 'parses an XML string' do
+          data = File.read('spec/data/resourcedump-index.xml')
+          sitemapindex = ResourceDumpIndex.load_from_xml(XML.element(data))
+
+          links = sitemapindex.links
+          expect(links.size).to eq(1)
+          ln0 = links[0]
+          expect(ln0.rel).to eq('up')
+          expect(ln0.uri).to eq(URI('http://example.com/dataset1/capabilitylist.xml'))
+
+          md = sitemapindex.metadata
+          expect(md.capability).to eq('resourcedump')
+          expect(md.at_time).to be_time(Time.utc(2013, 1, 3, 9))
+          expect(md.completed_time).to be_time(Time.utc(2013, 1, 3, 9, 10))
+
+          sitemaps = sitemapindex.resources
+          expect(sitemaps.size).to eq(3)
+
+          expected_times = [Time.utc(2013, 1, 3, 9), Time.utc(2013, 1, 3, 9, 3), Time.utc(2013, 1, 3, 9, 7)]
+          (0..2).each do |i|
+            sitemap = sitemaps[i]
+            expect(sitemap.uri).to eq(URI("http://example.com/resourcedump#{i + 1}.xml"))
+            md = sitemap.metadata
+            expect(md.at_time).to be_time(expected_times[i])
+          end
+        end
+      end
+
+      describe '#save_to_xml' do
+        it 'can round-trip to XML' do
+          data = File.read('spec/data/resourcedump-index.xml')
+          list = ResourceDumpIndex.load_from_xml(XML.element(data), mapping: :sitemapindex)
+          xml = list.save_to_xml
+          expect(xml).to be_xml(data)
+        end
+      end
+
+    end
+
+  end
+end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: resync
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.1.1
 platform: ruby
 authors:
 - David Moles
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2015-06-01 00:00:00.000000000 Z
+date: 2015-06-03 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: mime-types
@@ -161,6 +161,7 @@ files:
 - ".rubocop.yml"
 - ".ruby-version"
 - ".travis.yml"
+- CHANGES.md
 - Gemfile
 - LICENSE.md
 - README.md
@@ -169,6 +170,7 @@ files:
 - lib/resync.rb
 - lib/resync/capability_list.rb
 - lib/resync/change_dump.rb
+- lib/resync/change_dump_index.rb
 - lib/resync/change_dump_manifest.rb
 - lib/resync/change_list.rb
 - lib/resync/change_list_index.rb
@@ -176,6 +178,7 @@ files:
 - lib/resync/metadata.rb
 - lib/resync/resource.rb
 - lib/resync/resource_dump.rb
+- lib/resync/resource_dump_index.rb
 - lib/resync/resource_dump_manifest.rb
 - lib/resync/resource_list.rb
 - lib/resync/resource_list_index.rb
@@ -193,6 +196,7 @@ files:
 - lib/resync/xml_parser.rb
 - resync.gemspec
 - spec/acceptance/xml_parser_spec.rb
+- spec/data/changedump-index.xml
 - spec/data/examples/README.md
 - spec/data/examples/example-1.xml
 - spec/data/examples/example-12.xml
@@ -224,6 +228,7 @@ files:
 - spec/data/examples/example-6.xml
 - spec/data/examples/example-7.xml
 - spec/data/examples/example-8.xml
+- spec/data/resourcedump-index.xml
 - spec/data/resourcesync.xsd
 - spec/data/siteindex.xsd
 - spec/data/sitemap.xsd
@@ -231,12 +236,14 @@ files:
 - spec/spec_helper.rb
 - spec/todo.rb
 - spec/unit/resync/capability_list_spec.rb
+- spec/unit/resync/change_dump_index_spec.rb
 - spec/unit/resync/change_dump_manifest_spec.rb
 - spec/unit/resync/change_dump_spec.rb
 - spec/unit/resync/change_list_index_spec.rb
 - spec/unit/resync/change_list_spec.rb
 - spec/unit/resync/link_spec.rb
 - spec/unit/resync/metadata_spec.rb
+- spec/unit/resync/resource_dump_index_spec.rb
 - spec/unit/resync/resource_dump_manifest_spec.rb
 - spec/unit/resync/resource_dump_spec.rb
 - spec/unit/resync/resource_list_index_spec.rb
@@ -278,6 +285,7 @@ specification_version: 4
 summary: Utility library for ResourceSync
 test_files:
 - spec/acceptance/xml_parser_spec.rb
+- spec/data/changedump-index.xml
 - spec/data/examples/README.md
 - spec/data/examples/example-1.xml
 - spec/data/examples/example-12.xml
@@ -309,6 +317,7 @@ test_files:
 - spec/data/examples/example-6.xml
 - spec/data/examples/example-7.xml
 - spec/data/examples/example-8.xml
+- spec/data/resourcedump-index.xml
 - spec/data/resourcesync.xsd
 - spec/data/siteindex.xsd
 - spec/data/sitemap.xsd
@@ -316,12 +325,14 @@ test_files:
 - spec/spec_helper.rb
 - spec/todo.rb
 - spec/unit/resync/capability_list_spec.rb
+- spec/unit/resync/change_dump_index_spec.rb
 - spec/unit/resync/change_dump_manifest_spec.rb
 - spec/unit/resync/change_dump_spec.rb
 - spec/unit/resync/change_list_index_spec.rb
 - spec/unit/resync/change_list_spec.rb
 - spec/unit/resync/link_spec.rb
 - spec/unit/resync/metadata_spec.rb
+- spec/unit/resync/resource_dump_index_spec.rb
 - spec/unit/resync/resource_dump_manifest_spec.rb
 - spec/unit/resync/resource_dump_spec.rb
 - spec/unit/resync/resource_list_index_spec.rb