traject-marc4j_reader 1.0.2-java → 1.1.0-java

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 209b0ffa0955679de3c33a051bc872d60adb1981
-  data.tar.gz: 3ef2f390a73a6714c84286af700e14f4bfb0ba61
+  metadata.gz: 731f7a7338beafc60639283c73a4ac0f0ced73cc
+  data.tar.gz: b9c6733b4a14aead04cbbba9221dc7edd56ac294
 SHA512:
-  metadata.gz: ca446d5ab5180a42b9d553d2e2e4ef9d707b09b6422cb9e1ec7303b84dd5f865b42f4e23436c59e04e36a7bc4bc1fa9eea04dc2d14e98cbd00f035f222d2b5b9
-  data.tar.gz: d1bd3535b23938038a24a79a52292c36457054bbdd4fcc5d025b7f9c78d96b8c60926d212038be4fcd119714245ba703d2f018b84a760f0034d0d0810016eaf2
+  metadata.gz: f4fe9c0eb1f72492777b96778259a1a74e58a0e2633d36b8be0527da51c723209377a0542259629628081ad4695d8280d1e747b2eb9a151fa41a2c4932fac14a
+  data.tar.gz: e1ad15787f3e51ac71e58c83ea074b904979f35f3337989e643344361c14b318539f78923cf29b6b2cba0fa33153df6f23a519532f949739e75e322e9c3c52f7

data/CHANGES.md

@@ -0,0 +1,27 @@
+Adds a new setting to allow finer-grained control over which `marc4j` reader
+is used to process binary MARC. 
+
+`traject` uses a broadly permissive set of defaults to read binary MARC
+records, which may not always be what you want.  The `permissive` setting sets
+the flag of the same name on the
+(https://github.com/marc4j/marc4j/blob/master/src/org/marc4j/MarcPermissiveStreamReader.java#L164)['permissive'
+reader class] provided by marc4j (which, at the time of writing, controls how
+that reader guesses the encoding of input records.
+
+## Use the "strict" `org.marc4j.MarcStreamReader` class to read MARC21
+
+In situations where you want stricter record processing -- or in case your records can't be processed by the permissive stream reader (paradoxically, `MarcStreamReader` is more forgiving of certain non-standard MARC, e.g. uppercase subfields), you can specify that `traject` should use the `org.marc4j.MarcStreamReader` class:
+
+```ruby 
+settings do 
+    provide 'marc4j_reader.class', 'MarcStreamReader'
+end
+```
+
+## A note about `permissive`
+
+The `marc4j_reader.permissive` setting, which previously existed, is passed
+through to the constructor of the `MarcPermissiveStreamReader` class, and does
+not effect which class is used to read MARC21 input.  If you set both this parameter and the `marc4j_reader.class` parameter, the `permissive` setting will be ignored.
+
+

data/README.md

@@ -26,17 +26,28 @@ of the workload in a traject run, you'll almost certainly see performance gains.
 
 ## Installation
 
-Add this line to your application's Gemfile:
+Traject prior to 3.0 included this as a dependency on JRuby, and defaulted to using it.
 
-    gem 'traject-marc4j_reader'
+In Traject 3.0+, you need to manually add this gem and configure to use it.
 
-And then execute:
+If you are using bundler and a `Gemfile`, add `gem "traject-marc4j_reader", "~> 1.0"` to your `Gemfile`. Otherwise, just `gem install traject-marc4j_reader`.
 
-    $ bundle
+Then, in your traject config file:
 
-Or install it yourself as:
+    # Instead of require in config file, you could use the `-r` traject
+    # command-line option.
+    require 'traject/marc4j_reader'
 
-    $ gem install traject-marc4j_reader
+    settings do
+      provide "reader_class_name", "Traject::Marc4JReader"
+
+      # Recommend marc4j_reader.permissive true unless you have reason not to.
+      # true was default provided by core traject gem in Traject pre-3.0, but isn't
+      # anymore in traject 3.0 -- so set to true explicitly to maintain behavior
+      #
+      # Only relevant for binary MARC source data.
+      provide "marc4j_reader.permissive", true
+    end
 
 ## Traject::Marc4jReader settings
 
@@ -49,7 +60,7 @@ so output will always reflect that conversion.
 * `marc4j.jar_dir`:   Path to a directory containing Marc4J jar file to use. All .jar's in dir will
                       be loaded. If unset, uses marc4j.jar bundled with traject.
 
-* `marc4j_reader.permissive`: Used by Marc4JReader only when marc.source_type is 'binary', boolean, argument to the underlying MarcPermissiveStreamReader. Default true.
+* `marc4j_reader.permissive`: Used by Marc4JReader only when marc.source_type is 'binary', boolean, argument to the underlying MarcPermissiveStreamReader. Default false, but recommend true for most uses.
 
 * `marc4j_reader.source_encoding`: Used by Marc4JReader only when marc.source_type is 'binary', encoding strings accepted
   by marc4j MarcPermissiveStreamReader. Default "BESTGUESS", also "UTF-8", "MARC"
@@ -57,6 +68,8 @@ so output will always reflect that conversion.
 * `marc4j_reader.keep_marc4j`: After translating the marc4j record into a normal ruby-marc object,
 provides access to the former via `record#original_marc4j`.
 
+* 'marc4j_reader.class': Set to eg 'MarcStreamReader' to use that more strict Marc4J reader class, instead of the default Marc4J `MarcPermissiveStreamReader`.
+
 
 ## Sample use
 

data/lib/traject/marc4j_reader.rb

@@ -19,9 +19,9 @@ require 'marc/marc4j'
 #
 # * marc_source.type:     serialization type. default 'binary', also 'xml' (TODO: json/marc-in-json)
 #
-# * marc4j_reader.permissive:   default true, false to turn off permissive reading. Used as
+# * marc4j_reader.permissive: Used as
 #                             value to 'permissive' arg of MarcPermissiveStreamReader constructor.
-#                             Only used for 'binary'
+#                             Only used for 'binary'. Default false, but recommend true for most uses.
 #
 # * marc_source.encoding: Only used for 'binary', otherwise always UTF-8.
 #         String of the values MarcPermissiveStreamReader accepts:
@@ -81,6 +81,7 @@ class Traject::Marc4JReader
 
     # Convenience
     java_import org.marc4j.MarcPermissiveStreamReader
+    java_import org.marc4j.MarcStreamReader
     java_import org.marc4j.MarcXmlReader
 
   end
@@ -112,19 +113,26 @@ class Traject::Marc4JReader
   def create_marc_reader!
     case input_type
     when "binary"
-      permissive = settings["marc4j_reader.permissive"].to_s == "true"
-
-      # #to_inputstream turns our ruby IO into a Java InputStream
-      # third arg means 'convert to UTF-8, yes'
-      MarcPermissiveStreamReader.new(input_stream.to_inputstream, permissive, true, specified_source_encoding)
+      the_stream = input_stream.to_inputstream
+      if settings['marc4j_reader.class'] == 'MarcStreamReader'
+          MarcStreamReader.new(the_stream, specified_source_encoding)
+      else
+        permissive = settings["marc4j_reader.permissive"].to_s == "true"
+
+        # #to_inputstream turns our ruby IO into a Java InputStream
+        # third arg means 'convert to UTF-8, yes'
+        MarcPermissiveStreamReader.new(the_stream, permissive, true, specified_source_encoding)
+      end
     when "xml"
       MarcXmlReader.new(input_stream.to_inputstream)
     else
-      raise IllegalArgument.new("Unrecgonized marc_source.type: #{input_type}")
+      raise ArgumentError.new("Unrecgonized marc_source.type: #{input_type}")
     end
   end
 
   def each
+    return to_enum(:each) unless block_given?
+
     while (internal_reader.hasNext)
       begin
         marc4j = internal_reader.next

data/lib/traject/marc4j_reader/version.rb

@@ -1,5 +1,5 @@
 module Traject
   class Marc4JReader
-    VERSION = "1.0.2"
+    VERSION = "1.1.0"
   end
 end

metadata

@@ -1,83 +1,83 @@
 --- !ruby/object:Gem::Specification
 name: traject-marc4j_reader
 version: !ruby/object:Gem::Version
-  version: 1.0.2
+  version: 1.1.0
 platform: java
 authors:
 - Bill Dueber
-autorequire:
+autorequire: 
 bindir: bin
 cert_chain: []
-date: 2015-02-16 00:00:00.000000000 Z
+date: 2018-10-01 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
+  name: marc
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - ~>
+    - - "~>"
       - !ruby/object:Gem::Version
         version: '1.0'
-  name: marc
-  prerelease: false
   type: :runtime
+  prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - ~>
+    - - "~>"
       - !ruby/object:Gem::Version
         version: '1.0'
 - !ruby/object:Gem::Dependency
+  name: marc-marc4j
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - ~>
+    - - "~>"
       - !ruby/object:Gem::Version
         version: '1.0'
-  name: marc-marc4j
-  prerelease: false
   type: :runtime
+  prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - ~>
+    - - "~>"
       - !ruby/object:Gem::Version
         version: '1.0'
 - !ruby/object:Gem::Dependency
+  name: bundler
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - ~>
+    - - "~>"
       - !ruby/object:Gem::Version
         version: '1.6'
-  name: bundler
-  prerelease: false
   type: :development
+  prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - ~>
+    - - "~>"
       - !ruby/object:Gem::Version
         version: '1.6'
 - !ruby/object:Gem::Dependency
+  name: rake
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-  name: rake
-  prerelease: false
   type: :development
+  prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
+  name: minitest
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
-  name: minitest
-  prerelease: false
   type: :development
+  prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
 description: 'Allows jruby users to leverage marc-marc4j to use marc4j as a reader
@@ -88,8 +88,9 @@ executables: []
 extensions: []
 extra_rdoc_files: []
 files:
-- .gitignore
-- .travis.yml
+- ".gitignore"
+- ".travis.yml"
+- CHANGES.md
 - Gemfile
 - LICENSE.txt
 - README.md
@@ -110,24 +111,24 @@ homepage: ''
 licenses:
 - MIT
 metadata: {}
-post_install_message:
+post_install_message: 
 rdoc_options: []
 require_paths:
 - lib
 required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
-  - - '>='
+  - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
-  - - '>='
+  - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubyforge_project:
-rubygems_version: 2.1.9
-signing_key:
+rubyforge_project: 
+rubygems_version: 2.5.2.3
+signing_key: 
 specification_version: 4
 summary: Use marc4j (java) library under traject
 test_files:
@@ -140,4 +141,3 @@ test_files:
 - test/test_support/test_data.utf8.marc.xml
 - test/test_support/test_data.utf8.mrc
 - test/test_traject_marc4j_reader.rb
-has_rdoc: