traject 0.17.0 → 1.0.0.beta.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: ab462aadfb1252846b617cf1adb288eeb519b353
-  data.tar.gz: 7eac38dd8ac32e1dbfd417686ff04f95c108f011
+  metadata.gz: 19fae7c4d428bc40fd5c568f13daa7583fcebb2a
+  data.tar.gz: 9c71c57d86f6b9585451e958596a757cc75c432d
 SHA512:
-  metadata.gz: 331350a2a93083b10710943e71bdf31b30bb3c6aeed9dde97f05fd232eaa34681a7ac0bcdf0d7aae9e37fd6ea7b9d3e4da1c840f036ed9abc6d57a01aea02e12
-  data.tar.gz: 381e2c56dc2b92e0b91330bf20275e47462b86c1301ef723f31297cae1702b1f2fb77e6b8a016cf9213879f4c593ce001b68e854649613f12ba9a96238dc9da2
+  metadata.gz: 6730cd527d1ea285a5d60ebddeb95950dae2fdb5c548b905627eddf7a4a985e1891a039f409577903f181c60613861a3798fede6d5ecc8a942140cbb7a2d63d4
+  data.tar.gz: ee66b43f7bed57fcb21d82e7cd82c6e49c973bf7780b7a9f9c95fb5afc28744ffdb03e63d9d9b28bf567123571fa760ea9fed30fa62d20cf0ebfd0ef95d5880a

data/README.md

@@ -6,10 +6,10 @@ Might be used to index MARC data for a Solr-based discovery product like [Blackl
 Traject might also be generalized to a set of tools for getting structured data from a source, and sending it to a destination. 
 
 
-**Traject is nearing 1.0, it is robust, feature-rich and ready for trial use**
+**Traject is nearing 1.0, it is robust, feature-rich and being used in production by authors -- feedback invited**
 
 [![Gem Version](https://badge.fury.io/rb/traject.png)](http://badge.fury.io/rb/traject)
-[![Build Status](https://travis-ci.org/jrochkind/traject.png)](https://travis-ci.org/jrochkind/traject)
+[![Build Status](https://travis-ci.org/traject-project/traject.png)](https://travis-ci.org/traject-project/traject)
 
 
 ## Background/Goals
@@ -147,8 +147,10 @@ data out of a MARC record according to a tag/subfield specification.
 to matched specifications, but you can turn that off, or extract *only* corresponding
 880s. 
 
+~~~ruby
     to_field "title", extract_marc("245abc", :alternate_script => false)
     to_field "title_vernacular", extract_marc("245abc", :alternate_script => :only)
+~~~
 
 By default, specifications with multiple subfields (like "240abc") will produce one single string of output for each matching field. Specifications with single subfields (like "020a") will split subfields and produce an output string for each matching subfield.    
 
@@ -173,20 +175,25 @@ To see all options for `extract_marc`, see the [method documentation](http://rdo
 Other built-in methods that can be used with `to_field` include a hard-coded
 literal string:
 
+~~~ruby
     to_field "source", literal("LIB_CATALOG")
+~~~
 
 The current record serialized back out as MARC, in binary, XML, or json:
 
+~~~ruby
     # or :format => "json" for marc-in-json
     # or :format => "binary", by default Base64-encoded for Solr
     # 'binary' field, or, for more like what SolrMarc did, without
     # escaping:
     to_field "marc_record_raw", serialized_marc(:format => "binary", :binary_escape => false, :allow_oversized => true)
+~~~
 
 Text of all fields in a range:
 
+~~~ruby
     to_field "text", extract_all_marc_values(:from => 100, :to => 899)
-
+~~~
 
 All of these methods are defined at [Traject::Macros::Marc21](./lib/traject/macros/marc21.rb) ([rdoc](http://rdoc.info/gems/traject/Traject/Macros/Marc21))
 
@@ -198,6 +205,7 @@ them available to your indexing, you just need to use ruby `require` and `extend
 
 A number of methods are in [Traject::Macros::Marc21Semantics](./lib/traject/macros/marc21_semantics.rb) ([rdoc](http://rdoc.info/gems/traject/Traject/Macros/Marc21Semantics))
 
+~~~ruby
     require 'traject/macros/marc21_semantics'
     extend Traject::Macros::Marc21Semantics
 
@@ -205,15 +213,17 @@ A number of methods are in [Traject::Macros::Marc21Semantics](./lib/traject/macr
     to_field 'broad_subject',     marc_lcc_to_broad_category
     to_field "geographic_facet",  marc_geo_facet
     # And several more
+~~~
 
 And, there's a routine for classifying MARC to an internal
 format/genre/type vocabulary:
 
+~~~ruby
     require 'traject/macros/marc_format_classifier'
     extend Traject::Macros::MarcFormats
 
     to_field 'format_facet',    marc_formats
-
+~~~
 
 ## Custom logic
 
@@ -221,6 +231,7 @@ The built-in routines are there for your convenience, but if you need
 something local or custom, you can write ruby logic directly
 in a configuration file, using a ruby block, which looks like this:
 
+~~~ruby
     to_field "id" do |record, accumulator|
        # take the record's 001, prefix it with "bib_",
        # and then add it to the 'accumulator' argument,
@@ -229,6 +240,7 @@ in a configuration file, using a ruby block, which looks like this:
        value = "bib_#{value}"
        accumulator << value
     end
+~~~
 
 `do |record, accumulator|` is the definition of a ruby block taking
 two arguments.  The first one passed in will be a MARC record. The
@@ -239,21 +251,25 @@ Here's a more realistic example that shows how you'd get the
 record type byte 06 out of a MARC leader, then translate it
 to a human-readable string with a TranslationMap
 
+~~~ruby
     to_field "marc_type" do |record, accumulator|
       leader06 = record.leader.byteslice(6)
       # this translation map doesn't actually exist, but could
       accumulator << TranslationMap.new("marc_leader")[ leader06 ]
     end
+~~~
 
 You can also add a block onto the end of a built-in 'macro', to 
 further customize the output. The `accumulator` passed to your block
 will already have values in it from the first step, and you can
 use ruby methods like `map!` to modify it:
 
+~~~ruby
     to_field "big_title", extract_marc("245abcdefg") do |record, accumulator|
       # put it all in all uppercase, I don't know why. 
       accumulator.map! {|v| v.upcase}
     end
+~~~
 
 There are many more things you can do with custom logic blocks like this too,
 including additional features we haven't discussed yet. 
@@ -357,7 +373,6 @@ Also see `-I load_path` option and suggestions for Bundler use under Extending W
 See also [Hints for batch and cronjob use](./doc/batch_execution.md) of traject.
 
 
-
 ## Extending With Your Own Code
 
 Traject config files are full live ruby files, where you can do anything,
@@ -393,7 +408,11 @@ Own Code](./doc/extending.md)
 
 * [Other traject commands](./doc/other_commands.md) including `marcout`, and `commit`
 * [Hints for batch and cronjob use](./doc/batch_execution.md) of  traject.
-
+* Plugin extensions: Gems that add functionality to traject
+  * [traject_alephsequential_reader](https://github.com/traject-project/traject_alephsequential_reader/): read MARC files serialized in the AlephSequential format, as output by Ex Libris's Alpeh ILS.
+  * [traject_horizon](https://github.com/jrochkind/traject_horizon): Export MARC records directly from a Horizon ILS rdbms, as serialized MARC or to  index into Solr.
+  * [traject_umich_format](https://github.com/billdueber/traject_umich_format/): opinionated code and associated macros to extract format (book, audio file, etc.) and types (bibliography, conference report, etc.) from a MARC record. Code mirrors that used by the University of Michigan, and is an alternate approach to that taken by the `marc_formats` macro in `Traject::Macros::MarcFormatClassifier`.
+  
 
 # Development
 
@@ -415,6 +434,8 @@ and/or extra files in ./docs -- as appropriate for what needs to be docs.
 **Inline api docs** Note that our [`.yardopts` file](./.yardopts) used by rdoc.info to generate
 online api docs has a `--markup markdown` specified -- inline class/method docs are in markdown, not rdoc. 
 
+Bundler rake tasks included for gem releases: `rake release`
+
 ## TODO
 
 

data/doc/indexing_rules.md

@@ -58,7 +58,7 @@ you need to modify the array in-place.
 The third optional context argument 
 
 The third optional argument is a 
-[Traject::Indexer::Context](./lib/traject/indexer/context.rb)  ([rdoc](http://rdoc.info/github/jrochkind/traject/Traject/Indexer/Context))
+[Traject::Indexer::Context](./lib/traject/indexer/context.rb)  ([rdoc](http://rdoc.info/github/traject-project/traject/Traject/Indexer/Context))
 object. Most of the time you don't need it, but you can use it for 
 some sophisticated functionality, for example using these Context methods:
 

data/lib/traject/marc_extractor.rb

@@ -36,8 +36,7 @@ module Traject
   # and includes a tag and a a byte slice specification.
   #
   #      "008[35-37]:007[5]""
-  #      => bytes 35-37 inclusive of any field 008, and byte 5 of any field 007 (TODO: Should we support
-  #      "LDR" as a pseudo-tag to take byte slices of leader?)
+  #      => bytes 35-37 inclusive of any field 008, and byte 5 of any field 007 
   #
   # * subfields and indicators can only be provided for marc data/variable fields
   # * byte slice can only be provided for marc control fields (generally tags less than 010)

data/lib/traject/util.rb

@@ -56,8 +56,6 @@ module Traject
         Object.const_set("HttpSolrServer", org.apache.solr.client.solrj.impl.HttpSolrServer) unless defined? ::HttpSolrServer
         Object.const_set("SolrInputDocument", org.apache.solr.common.SolrInputDocument) unless defined? ::SolrInputDocument
       rescue NameError  => e
-        # /Users/jrochkind/code/solrj-gem/lib"
-
         included_jar_dir = File.expand_path("../../vendor/solrj/lib", File.dirname(__FILE__))
 
         jardir = settings["solrj.jar_dir"] || included_jar_dir

data/lib/traject/version.rb

@@ -1,3 +1,3 @@
 module Traject
-  VERSION = "0.17.0"
+  VERSION = "1.0.0.beta.1"
 end

data/test/indexer/macros_marc21_semantics_test.rb

@@ -157,6 +157,18 @@ describe "Traject::Macros::Marc21Semantics" do
       @record = MARC::Reader.new(support_file_path  "date_type_r_missing_date2.marc").to_a.first
       assert_equal 1957, Marc21Semantics.publication_date(@record)
     end
+
+    it "works correctly with date type 'q'" do
+      val = @record['008'].value
+      val[6] = 'q'
+      val[7..10] = '191u'
+      val[11..14] = '192u'
+      @record['008'].value = val
+
+      # Date should be date1 + date2 / 2 = (1910 + 1929) / 2 = 1919
+      estimate_tolerance = 30
+      assert_equal 1919, Marc21Semantics.publication_date(@record, estimate_tolerance)
+    end
   end
 
   describe "marc_lcc_to_broad_category" do

data/traject.gemspec

@@ -9,7 +9,7 @@ Gem::Specification.new do |spec|
   spec.authors       = ["Jonathan Rochkind", "Bill Dueber"]
   spec.email         = ["none@nowhere.org"]
   spec.summary       = %q{Index MARC to Solr; or generally process source records to hash-like structures}
-  spec.homepage      = "http://github.com/jrochkind/traject"
+  spec.homepage      = "http://github.com/traject-project/traject"
   spec.license       = "MIT"
 
   spec.files         = `git ls-files`.split($/)
@@ -21,7 +21,7 @@ Gem::Specification.new do |spec|
 
 
   spec.add_dependency "marc", ">= 0.7.1"
-  spec.add_dependency "marc-marc4j", ">=0.1.1"
+  spec.add_dependency "marc-marc4j", ">=0.1.1" # use and convert marc4j
   spec.add_dependency "hashie", ">= 2.0.5", "< 2.1" # used for Indexer#settings
   spec.add_dependency "slop", ">= 3.4.5", "< 4.0"   # command line parsing
   spec.add_dependency "yell" # logging

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: traject
 version: !ruby/object:Gem::Version
-  version: 0.17.0
+  version: 1.0.0.beta.1
 platform: ruby
 authors:
 - Jonathan Rochkind
@@ -9,7 +9,7 @@ authors:
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2013-10-10 00:00:00.000000000 Z
+date: 2013-10-14 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: marc
@@ -265,7 +265,7 @@ files:
 - vendor/solrj/lib/solr-solrj-4.3.1.jar
 - vendor/solrj/lib/wstx-asl-3.2.7.jar
 - vendor/solrj/lib/zookeeper-3.4.5.jar
-homepage: http://github.com/jrochkind/traject
+homepage: http://github.com/traject-project/traject
 licenses:
 - MIT
 metadata: {}
@@ -280,9 +280,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '0'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
-  - - '>='
+  - - '>'
     - !ruby/object:Gem::Version
-      version: '0'
+      version: 1.3.1
 requirements: []
 rubyforge_project:
 rubygems_version: 2.1.5