solrizer 3.1.1 → 3.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 2352b7c26cf55046a975d909108924a8ca9021f8
-  data.tar.gz: c83e79545b2caa984d75f2deb6bd14d3495a057e
+  metadata.gz: 8d77a71925151847cbb586039e03296e1b0aa2c9
+  data.tar.gz: a2bbb1286b7b037dc5a83d81f84ec098edc60283
 SHA512:
-  metadata.gz: 518b8942031a7de396ebfd51b0e3de9b933ec0f9a06e10d5f53249655dee2ebfc9da6ad65a42036dec7fbf9f6bd0614f84f2ae4ce411e036c63ddeffd5b41967
-  data.tar.gz: d3e557a70e7f9fd80c6be842b5fb392eee8e93f786638ae691b6ee7c4c9b5bbb7ea939b61f42d2854e19c2a075e247e49d9240085ca6d08484cac57f6d6a20b4
+  metadata.gz: 76ad75ea81fb427b72a0b214637427f44ecd54d5e4b80cc43f49ae4c13598816d3756144b4b1081f7c357169595d97d7c8ce4e1137533e54e99807c619f9d9d6
+  data.tar.gz: 9294cad6b4a5661321d4a0780652af3e31fbebfeabf6b3ebe5d62577d37aaf6cc685255edb50acc5fd8782f468bb05a0d8ba1e64bb40c346c6f10202e8103250

data/CONTRIBUTING.md

@@ -0,0 +1,113 @@
+# How to Contribute
+
+We want your help to make Project Hydra great.
+There are a few guidelines that we need contributors to follow so that we can have a chance of keeping on top of things.
+
+## Hydra Project Intellectual Property Licensing and Ownership
+
+All code contributors must have an Individual Contributor License Agreement (iCLA) on file with the Hydra Project Steering Group.
+If the contributor works for an institution, the institution must have a Corporate Contributor License Agreement (cCLA) on file.
+
+https://wiki.duraspace.org/display/hydra/Hydra+Project+Intellectual+Property+Licensing+and+Ownership
+
+You should also add yourself to the `CONTRIBUTORS.md` file in the root of the project.
+
+## Contribution Tasks
+
+* Reporting Issues
+* Making Changes
+* Submitting Changes
+* Merging Changes
+
+### Reporting Issues
+
+* Make sure you have a [GitHub account](https://github.com/signup/free)
+* Submit a [Github issue](./issues) by:
+  * Clearly describing the issue
+    * Provide a descriptive summary
+    * Explain the expected behavior
+    * Explain the actual behavior
+    * Provide steps to reproduce the actual behavior
+
+### Making Changes
+
+* Fork the repository on GitHub
+* Create a topic branch from where you want to base your work.
+  * This is usually the master branch.
+  * To quickly create a topic branch based on master; `git branch fix/master/my_contribution master`
+  * Then checkout the new branch with `git checkout fix/master/my_contribution`.
+  * Please avoid working directly on the `master` branch.
+  * You may find the [hub suite of commands](https://github.com/defunkt/hub) helpful
+* Make commits of logical units.
+  * Your commit should include a high level description of your work in HISTORY.textile 
+* Check for unnecessary whitespace with `git diff --check` before committing.
+* Make sure your commit messages are [well formed](http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html).
+* If you created an issue, you can close it by including "Closes #issue" in your commit message. See [Github's blog post for more details](https://github.com/blog/1386-closing-issues-via-commit-messages)
+
+```
+    Present tense short summary (50 characters or less)
+
+    More detailed description, if necessary. It should be wrapped to 72
+    characters. Try to be as descriptive as you can, even if you think that
+    the commit content is obvious, it may not be obvious to others. You
+    should add such description also if it's already present in bug tracker,
+    it should not be necessary to visit a webpage to check the history.
+
+    Include Closes #<issue-number> when relavent.
+
+    Description can have multiple paragraphs and you can use code examples
+    inside, just indent it with 4 spaces:
+
+        class PostsController
+          def index
+            respond_with Post.limit(10)
+          end
+        end
+
+    You can also add bullet points:
+
+    - you can use dashes or asterisks
+
+    - also, try to indent next line of a point for readability, if it's too
+      long to fit in 72 characters
+```
+
+* Make sure you have added the necessary tests for your changes.
+* Run _all_ the tests to assure nothing else was accidentally broken.
+* When you are ready to submit a pull request
+
+### Submitting Changes
+
+[Detailed Walkthrough of One Pull Request per Commit](http://ndlib.github.io/practices/one-commit-per-pull-request/)
+
+* Read the article ["Using Pull Requests"](https://help.github.com/articles/using-pull-requests) on GitHub.
+* Make sure your branch is up to date with its parent branch (i.e. master)
+  * `git checkout master`
+  * `git pull --rebase`
+  * `git checkout <your-branch>`
+  * `git rebase master`
+  * It is likely a good idea to run your tests again.
+* Squash the commits for your branch into one commit
+  * `git rebase --interactive HEAD~<number-of-commits>` ([See Github help](https://help.github.com/articles/interactive-rebase))
+  * To determine the number of commits on your branch: `git log master..<your-branch> --oneline | wc -l`
+  * Squashing your branch's changes into one commit is "good form" and helps the person merging your request to see everything that is going on.
+* Push your changes to a topic branch in your fork of the repository.
+* Submit a pull request from your fork to the project.
+
+### Merging Changes
+
+* It is considered "poor from" to merge your own request.
+* Please take the time to review the changes and get a sense of what is being changed. Things to consider:
+  * Does the commit message explain what is going on?
+  * Does the code changes have tests? _Not all changes need new tests, some changes are refactorings_
+  * Does the commit contain more than it should? Are two separate concerns being addressed in one commit?
+  * Did the Travis tests complete successfully?
+* If you are uncertain, bring other contributors into the conversation by creating a comment that includes their @username.
+* If you like the pull request, but want others to chime in, create a +1 comment and tag a user.
+
+# Additional Resources
+
+* [General GitHub documentation](http://help.github.com/)
+* [GitHub pull request documentation](http://help.github.com/send-pull-requests/)
+* [Pro Git](http://git-scm.com/book) is both a free and excellent book about Git.
+* [A Git Config for Contributing](http://ndlib.github.io/practices/my-typical-per-project-git-config/)

data/History.txt

@@ -1,3 +1,12 @@
+h2. 3.2.0
+  #25 Allow any field_value except nil to be inserted into a solr field
+  #24 Remove dependency on solrizer-fedora, use AF to update index by pid
+  #23 Enhance Suffix#config so it can be usefully overridden by downstream 
+
+h2. 3.1.1
+  #22 Support for boolean values
+  #21 Testing on Rails version 4
+
 h2. 3.1.0
   #16 Inserting non-multivalued fields should not create a solr error
   #20 Time fields should be formatted correctly when using active_support/core_ext/date_time/conversions

data/README.md

@@ -0,0 +1,252 @@
+# solrizer
+
+[![Build Status](https://travis-ci.org/projecthydra/solrizer.png?branch=master)](https://travis-ci.org/projecthydra/solrizer)
+[![Gem Version](https://badge.fury.io/rb/solrizer.png)](http://badge.fury.io/rb/solrizer)
+
+A lightweight, configurable tool for indexing metadata into solr.  Can be triggered from within your application, from
+the command line, or as a JMS listener.
+
+Solrizer provides the baseline and structures for the process of solrizing.  In order to actually read objects from a
+data source and write solr documents into a solr instance, you need to use an implementation specific gem, such as
+"solrizer-fedora":https://github.com/projecthydra/solrizer-fedora, which provides the mechanics for reading from a
+fedora repository and writing to a solr instance.
+
+
+## Installation
+
+The gem is hosted on [rubygems.org](http://rubygems.org/gems/solrizer). The best way to manage the gems for your project
+is to use bundler.  Create a Gemfile in the root of your application and include the following:
+
+
+    source "http://rubygems.org"
+    gem 'solrizer'
+
+Then:
+
+    bundle install
+
+## Usage
+
+### Fire up the console:
+
+The code snippets in the following sections can be cut/pasted into your console, giving you the opportunity to play with Solrizer.
+
+Start up a console and load solrizer:
+
+    > irb
+    > require "rubygems"
+    > require "solrizer"
+
+### Field Mapper
+
+The `FieldMapper` maps term names and values to Solr fields, based on the term's data type and any index_as options.
+Solrizer comes with default mappings to dynamic field types defined in the Hydra Solr 
+[schema.xml](https://github.com/projecthydra/hydra-head/blob/master/hydra-core/lib/generators/hydra/templates/solr_conf/conf/schema.xml).
+	
+More information on the conventions followed for the dynamic solr fields is on the 
+[wiki page](https://github.com/projecthydra/hydra-head/wiki/Solr-Schema).
+
+To examine all of Solrizer's field names, open up a ruby console:
+
+
+    > require 'solrizer'
+    => true
+    > default_mapper = Solrizer::FieldMapper.new
+    => #<Solrizer::FieldMapper:0x007fb47a273770 @id_field="id">
+    > default_mapper.solr_name("foo",:searchable, type: :string)
+    => "foo_teim"
+    > default_mapper.solr_name("foo",:searchable, type: :date)
+    => "foo_dtim"
+    > default_mapper.solr_name("foo",:searchable, type: :integer)
+    => "foo_iim"
+    > default_mapper.solr_name("foo",:facetable, type: :string)
+    => "foo_sim"
+    > default_mapper.solr_name("foo",:facetable, type: :integer)
+    => "foo_sim"
+    > default_mapper.solr_name("foo",:sortable, type: :string)
+    => "foo_si"
+    > default_mapper.solr_name("foo",:displayable, type: :string)
+    => "foo_ssm"
+
+### Default indexing strategies
+
+    > solr_doc = Hash.new
+    > Solrizer.insert_field(solr_doc, 'title', 'whatever', :stored_searchable) 
+    => {"title_tesim"=>["whatever"]}
+
+    > Solrizer.insert_field(solr_doc, 'pub_date', 'Nov 2012', :sortable, :displayable) 
+    => {"pub_date_si"=>"Nov 2012", "pub_date_ssm"=>["Nov 2012"]}
+
+### Indexing dates
+
+as a date:
+
+    > solr_doc = {}
+    > Solrizer.insert_field(solr_doc, 'pub_date', Date.parse('Nov 7th 2012'), :searchable)
+    => {"pub_date_dtim"=>["2012-11-07T00:00:00Z"]}
+
+or as a string:
+
+    > solr_doc = {}
+    > Solrizer.insert_field(solr_doc, 'pub_date', Date.parse('Nov 7th 2012'), :sortable, :displayable)
+    => {"pub_date_dti"=>"2012-11-07T00:00:00Z", "pub_date_ssm"=>["2012-11-07"]}
+
+or a string that is stored as a date:
+
+    > solr_doc = {}
+    > Solrizer.insert_field(solr_doc, 'pub_date', 'Jan 29th 2013', :dateable)
+    => {"pub_date_dtsim"=>["2013-01-29T00:00:00Z"]}
+
+### Custom indexing strategies
+
+#### Create your own index descriptor
+
+    > solr_doc = {}
+    > displearchable = Solrizer::Descriptor.new(:integer, :indexed, :stored)
+    > Solrizer.insert_field(solr_doc, 'some_count', 45, displearchable)
+    => {"some_count_isi"=>"45"}
+
+#### Override the defaults
+
+We can override the default indexing methods within `Solrizer::DefaultDescriptors`
+
+Here's the default behavior:
+
+    > solr_doc = {}
+    > Solrizer.insert_field(solr_doc, 'title', 'foobar', :facetable)
+    => {"title_sim"=>["foobar"]}
+
+But let's override that by redefining `:facetable`
+
+    module Solrizer
+      module DefaultDescriptors
+        def self.facetable
+          Descriptor.new(:string, :indexed, :stored)
+        end
+      end
+    end
+
+Now, `:facetable` will return something different:
+
+    > solr_doc = {}
+    > Solrizer.insert_field(solr_doc, 'title', 'foobar', :facetable)
+    => {"title_ssi"=>"foobar"}
+
+#### Creating your own indexers
+
+    module MyMappers
+      def self.mapper_one
+        Solrizer::Descriptor.new(:string, :indexed, :stored)
+      end
+    end
+
+Now, set Solrizer's field mapper to use our new module:
+
+    > solr_doc = {}
+    > Solrizer::FieldMapper.descriptors = [MyMappers]
+    => [MyMappers]
+    > Solrizer.insert_field(solr_doc, 'title', 'foobar', :mapper_one)
+    => {"title_ssi"=>"foobar"}
+
+### Using OM
+
+    t.main_title(:index_as=>[:facetable],:path=>"title", :label=>"title") { ... }
+
+But now you may also pass an Descriptor instance if that works for you:
+
+    indexer = Solrizer::Descriptor.new(:integer, :indexed, :stored)
+    t.main_title(:index_as=>[indexer],:path=>"title", :label=>"title") { ... }
+
+### Extractor and Extractor Mixins
+
+Solrizer::Extractor provides utilities for extracting solr fields from objects or inserting solr fields into documents:
+
+    > extractor = Solrizer::Extractor.new
+    > solr_doc = Hash.new
+    > extractor.format_node_value(["foo     ","\n      bar"])
+    => "foo bar"
+    > extractor.insert_solr_field_value(solr_doc, "foo","bar")
+    => {"foo"=>"bar"}
+    > extractor.insert_solr_field_value(solr_doc,"foo","baz")
+    => {"foo"=>["bar", "baz"]}
+    > extractor.insert_solr_field_value(solr_doc, "boo","hoo")
+    => {"foo"=>["bar", "baz"], "boo"=>"hoo"}
+
+#### Solrizer provides some default mixins:
+
+`Solrizer::HTML::Extractor` provides html_to_solr method and `Solrizer::XML::Extractor` provides xml_to_solr method:
+
+    > Solrizer::XML::Extractor
+    > extractor = Solrizer::Extractor.new
+    > xml = "<fields><foo>bar</foo><bar>baz</bar></fields>"
+    > extractor.xml_to_solr(xml)
+    => {:foo_tesim=>"bar", :bar_tesim=>"baz"}
+
+#### Solrizer::XML::TerminologyBasedSolrizer
+
+Another powerful mixin for use with classes that include the `OM::XML::Document` module is
+`Solrizer::XML::TerminologyBasedSolrizer`. The methods provided by this module map provides a robust way of mapping
+terms and solr fields via om terminologies. A notable example  can be found in `ActiveFedora::NokogiriDatatstream`.
+
+## JMS Listener for Hydra Rails Applications
+
+### The executables: solrizer and solrizerd
+
+The solrizer gem provides two executables:
+
+ * solrizer is a stomp consumer which listens for fedora.apim.updates and solrizes (or de-solrizes) objects accordingly. 
+ * solrizerd is a wrapper script that spawns a daemonized version of solrizer and handles start|stop|restart|status requests. 
+
+### Usage 
+
+The usage for solrizerd is as follows: 
+
+    solrizerd command --hydra_home PATH [options] 
+
+The commands are as follows:
+ *  start      start an instance of the application 
+ *  stop       stop all instances of the application 
+ *  restart    stop all instances and restart them afterwards 
+ *  status     show status (PID) of application instances 
+
+Required parameters:
+
+--hydra_home: this is the path to your hydra rails applications' root directory.  Solrizerd needs this in order to load all your models and corresponding terminoligies.
+
+The options:
+ *  -p, --port         Stomp port  61613 
+ *  -o, --host         Host to connect to  localhost 
+ *  -u, --user         User name for stomp listener  
+ *  -w, --password     Password for stomp listener  
+ *  -d, --destination  Topic to listen to (default: /topic/fedora.apim.update) 
+ *  -h, --help         Display this screen 
+
+Note:
+
+Since the solrizer script must fire up your hydra rails application, it must have all the gems installed that your hydra instance needs.
+
+## Note on Patches/Pull Requests
+ 
+* Fork the project.
+* Make your feature addition or bug fix.
+* Add tests for it. This is important so I don't break it in a
+  future version unintentionally.
+* Commit, do not mess with rake file, version, or history.
+  (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
+* Send me a pull request. Bonus points for topic branches.
+
+## Acknowledgments
+
+### Technical Lead
+
+Matt Zumwalt ("MediaShelf":http://yourmediashelf.com)
+
+### Thanks to 
+
+* Douglas Kim, who created the initial code base for Solrizer. 
+* Chris Fitzpatrick, who patiently ran the first prototype through its paces for weeks.
+* Bess Sadler, who created the JMS integration for Solrizer, generously served as a sounding board for numerous design issues around solr indexing, and pushes the technology forward with the skill of a true engineer.
+
+## Copyright
+
+Copyright (c) 2010 Matt Zumwalt. See LICENSE for details.

data/bin/solrizer

@@ -90,8 +90,7 @@ begin
       puts @msg.headers.inspect
       puts "\nPID: #{@msg.headers["pid"]}\n"
       if ["addDatastream", "addRelationship","ingest","modifyDatastreamByValue","modifyDatastreamByReference","modifyObject","purgeDatastream","purgeRelationship"].include? method
-        solrizer = Solrizer::Fedora::Solrizer.new
-        solrizer.solrize @msg.headers["pid"]
+        ActiveFedora::Base.find(@msg.headers["pid"], cast: true).update_index
       elsif method == "purgeObject"
         ActiveFedora::SolrService.instance.conn.delete_by_id(pid)
       else

data/lib/solrizer.rb

@@ -31,7 +31,7 @@ module Solrizer
 
   # @params [Hash] doc the hash to insert the value into
   # @params [String] name the name of the field (without the suffix)
-  # @params [String,Date] value the value to be inserted
+  # @params [String,Date,Array] value the value (or array of values) to be inserted
   # @params [Array,Hash] indexer_args the arguments that find the indexer
   # @returns [Hash] doc the document that was provided with the new field inserted
   def self.insert_field(doc, name, value, *indexer_args)

data/lib/solrizer/field_mapper.rb

@@ -181,7 +181,7 @@ module Solrizer
     # mapped names and values. The values in the hash are _arrays_, and may contain multiple values.
     
     def solr_names_and_values(field_name, field_value, index_types)
-      return {} unless field_value
+      return {} if field_value.nil?
       
       # Determine the set of index types
       index_types = Array(index_types)

data/lib/solrizer/suffix.rb

@@ -1,20 +1,26 @@
+require 'ostruct'
+
 module Solrizer
   class Suffix
 
-    def initialize(fields)
-      @fields = fields
+    def initialize(*fields)
+      @fields = fields.flatten
     end
 
     def multivalued?
-      @fields.include? :multivalued
+      has_field? :multivalued
     end
 
     def stored?
-      @fields.include? :stored
+      has_field? :stored
     end
 
     def indexed?
-      @fields.include? :indexed
+      has_field? :indexed
+    end
+
+    def has_field? f
+      f.to_sym == :type or @fields.include? f.to_sym
     end
 
     def data_type
@@ -22,40 +28,52 @@ module Solrizer
     end
 
     def to_s
-      stored_suffix = config[:stored_suffix] if stored?
-      index_suffix = config[:index_suffix] if indexed?
-      multivalued_suffix = config[:multivalued_suffix] if multivalued?
+
       raise Solrizer::InvalidIndexDescriptor, "Missing datatype for #{@fields}" unless data_type
-      type_suffix = config[:type_suffix].call(data_type)
-      raise Solrizer::InvalidIndexDescriptor, "Invalid datatype `#{data_type.inspect}'. Must be one of: :date, :time, :text, :text_en, :string, :integer" unless type_suffix
 
-      [config[:suffix_delimiter], type_suffix, stored_suffix, index_suffix, multivalued_suffix].join
+      field_suffix = [config.suffix_delimiter]
+
+      config.fields.select { |f| has_field? f }.each do |f|
+        key = :"#{f}_suffix"
+        field_suffix << if config.send(key).is_a? Proc
+          config.send(key).call(@fields)
+        else
+          config.send(key)
+        end
+      end
+      
+      field_suffix.join
     end
 
+    def self.config
+      @config ||= OpenStruct.new :fields => [:type, :stored, :indexed, :multivalued],
+        suffix_delimiter: '_',
+        type_suffix: (lambda do |fields|
+          type = fields.first
+          case type
+          when :string, :symbol # TODO `:symbol' usage ought to be deprecated
+            's'
+          when :text
+            't'
+          when :text_en
+            'te'
+          when :date, :time
+            'dt'
+          when :integer
+            'i'
+          when :boolean
+            'b'
+          else
+            raise Solrizer::InvalidIndexDescriptor, "Invalid datatype `#{type.inspect}'. Must be one of: :date, :time, :text, :text_en, :string, :symbol, :integer, :boolean"
+          end
+        end),
+        stored_suffix: 's',
+        indexed_suffix: 'i',
+        multivalued_suffix: 'm'
+    end
 
-    private
     def config
-      @config ||= 
-      {suffix_delimiter: '_',
-      type_suffix: lambda do |type|  
-        case type
-        when :string, :symbol # TODO `:symbol' usage ought to be deprecated
-          's'
-        when :text
-          't'
-        when :text_en
-          'te'
-        when :date, :time
-          'dt'
-        when :integer
-          'i'
-        when :boolean
-          'b'
-        end
-      end,
-      stored_suffix: 's', 
-      index_suffix: 'i',
-      multivalued_suffix: 'm'}
+      @config ||= self.class.config.dup
     end
   end
 end

data/lib/solrizer/version.rb

@@ -1,3 +1,3 @@
 module Solrizer
-  VERSION = "3.1.1"
+  VERSION = "3.2.0"
 end

data/solrizer.gemspec

@@ -28,7 +28,7 @@ Gem::Specification.new do |s|
   s.executables   = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
   s.extra_rdoc_files = [
     "LICENSE",
-    "README.textile"
+    "README.md"
   ]
   s.require_paths = ["lib"]
 end

data/spec/units/solrizer_spec.rb

@@ -25,15 +25,24 @@ describe Solrizer do
         Solrizer.insert_field(doc, 'foo', Time.parse('2013-01-13T22:45:56+06:00'))
         doc.should == {'foo_dtsim' => ["2013-01-13T16:45:56Z"]}
       end
-      it "should insert Booleans" do
+      it "should insert true Booleans" do
         Solrizer.insert_field(doc, 'foo', true)
         doc.should == {'foo_bsi' => true}
       end
+      it "should insert false Booleans" do
+        Solrizer.insert_field(doc, 'foo', false)
+        doc.should == {'foo_bsi' => false}
+      end
 
       it "should insert multiple values" do
         Solrizer.insert_field(doc, 'foo', ['A name', 'B name'], :sortable, :facetable)
         doc.should == {'foo_si' => 'B name', 'foo_sim' => ['A name', 'B name']}
       end
+
+      it 'should insert nothing when passed a nil value' do
+        Solrizer.insert_field(doc, 'foo', nil, :sortable, :facetable)
+        doc.should == {}
+      end
     end
 
     describe "on a document with values" do

data/spec/units/suffix_spec.rb

@@ -0,0 +1,80 @@
+require 'spec_helper'
+
+describe Solrizer::Suffix do
+
+  describe "#multivalued?" do
+    it "should be multivalued if :multivalued is among the field types" do
+      expect(Solrizer::Suffix.new(:multivalued)).to be_multivalued
+    end
+
+    it "should not be multivalued if :multivalued was not passed in a field type" do
+      expect(Solrizer::Suffix.new(:some_other_field_type)).to_not be_multivalued
+    end
+  end
+
+  describe "#stored?" do
+    it "should be stored if :stored is among the field types" do
+      expect(Solrizer::Suffix.new(:stored)).to be_stored
+    end
+
+    it "should not be stored if :stored was not passed in a field type" do
+      expect(Solrizer::Suffix.new(:some_other_field_type)).to_not be_stored
+    end
+  end
+
+  describe "#indexed?" do
+    it "should be indexed if :indexed is among the field types" do
+      expect(Solrizer::Suffix.new(:indexed)).to be_indexed
+    end
+
+    it "should not be indexed if :indexed was not passed in a field type" do
+      expect(Solrizer::Suffix.new(:some_other_field_type)).to_not be_indexed
+    end
+  end
+  describe "#has_field?" do  
+    subject do
+      Solrizer::Suffix.new(:type, :a, :b, :c)
+    end  
+    it "should be able to tell if a field is in the suffix or not" do
+      expect(subject).to have_field :a
+      expect(subject).to have_field :b
+      expect(subject).to have_field :c
+      expect(subject).to_not have_field :d
+    end
+  end
+
+  describe "#data_type" do
+    it "should always be the first argument to the suffix" do
+      expect(Solrizer::Suffix.new(:some_type, :a).data_type).to eq :some_type
+    end
+  end
+
+  describe "#to_s" do
+    it "should combine the fields into a suffix string" do
+      expect(Solrizer::Suffix.new(:string, :stored, :indexed).to_s).to eq '_ssi'
+      expect(Solrizer::Suffix.new(:integer, :stored, :multivalued).to_s).to eq '_ism'
+    end
+  end
+
+  describe "config" do
+    subject do
+      Solrizer::Suffix.new(:my_custom_type, :a, :b, :c)
+    end  
+
+    it "should let you mess with the suffix config" do
+      subject.config.fields += [:b]
+      subject.config.suffix_delimiter = "#"
+      subject.config.type_suffix = lambda do |fields|
+        type = fields.first
+
+        if type == :my_custom_type
+          "custom_suffix_"
+        else
+          "nope"
+        end
+      end
+      subject.config.b_suffix = 'now_with_more_b'
+      expect(subject.to_s).to eq "#custom_suffix_now_with_more_b"
+    end
+  end
+end

metadata

@@ -1,153 +1,153 @@
 --- !ruby/object:Gem::Specification
 name: solrizer
 version: !ruby/object:Gem::Version
-  version: 3.1.1
+  version: 3.2.0
 platform: ruby
 authors:
 - Matt Zumwalt
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-06-25 00:00:00.000000000 Z
+date: 2014-05-28 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: nokogiri
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
   name: xml-simple
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
   name: mediashelf-loggable
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - ~>
+    - - "~>"
       - !ruby/object:Gem::Version
         version: 0.4.7
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - ~>
+    - - "~>"
       - !ruby/object:Gem::Version
         version: 0.4.7
 - !ruby/object:Gem::Dependency
   name: stomp
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
   name: daemons
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
   name: activesupport
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
   name: rspec
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
   name: rake
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
   name: yard
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
   name: RedCloth
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - '>='
+    - - ">="
       - !ruby/object:Gem::Version
         version: '0'
 description: Use solrizer to populate solr indexes.  You can run solrizer from within
@@ -159,14 +159,15 @@ executables:
 extensions: []
 extra_rdoc_files:
 - LICENSE
-- README.textile
+- README.md
 files:
-- .gitignore
-- .travis.yml
+- ".gitignore"
+- ".travis.yml"
+- CONTRIBUTING.md
 - Gemfile
 - History.txt
 - LICENSE
-- README.textile
+- README.md
 - Rakefile
 - bin/solrizer
 - bin/solrizerd
@@ -193,6 +194,7 @@ files:
 - spec/units/extractor_spec.rb
 - spec/units/field_mapper_spec.rb
 - spec/units/solrizer_spec.rb
+- spec/units/suffix_spec.rb
 - spec/units/xml_extractor_spec.rb
 homepage: http://github.com/projecthydra/solrizer
 licenses: []
@@ -203,17 +205,17 @@ require_paths:
 - lib
 required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
-  - - '>='
+  - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
-  - - '>='
+  - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.0.3
+rubygems_version: 2.2.2
 signing_key: 
 specification_version: 4
 summary: A utility for building solr indexes, usually from Fedora repository content
@@ -225,5 +227,6 @@ test_files:
 - spec/units/extractor_spec.rb
 - spec/units/field_mapper_spec.rb
 - spec/units/solrizer_spec.rb
+- spec/units/suffix_spec.rb
 - spec/units/xml_extractor_spec.rb
 has_rdoc: 

data/README.textile

@@ -1,249 +0,0 @@
-h1. solrizer
-
-A lightweight, configurable tool for indexing metadata into solr.  Can be triggered from within your application, from the command line, or as a JMS listener.
-
-Solrizer provides the baseline and structures for the process of solrizing.  In order to actually read objects from a 
-datasource and write solr documents into a solr instance, you need to use an implementation specific gem, such as 
-"solrizer-fedora":https://github.com/projecthydra/solrizer-fedora, which provides the mechanics for reading from a fedora repository and writing to a solr instance. 
-
-
-h2. Installation
-
-The gem is hosted on rubygems.org. The best way to manage the gems for your project is to use bundler.  Create a Gemfile in the root of your application and include the following:
-
-<pre>
-source "http://rubygems.org"
-
-gem 'solrizer'
-</pre>
-
-Then:
-
-<pre>bundle install</pre>
-
-h2. Usage
-
-h3. Fire up the console:
-
-The code snippets in the following sections can be cut/pasted into your console, giving you the opportunity to play with Solrizer.
-
-Start up a console and load solrizer:
-
-<pre>
-irb
-require "rubygems"
-require "solrizer"
-</pre>
-
-
-h3. Field Mapper
-
-The FieldMapper maps term names and values to Solr fields, based on the term’s data type and any index_as options.  Solrizer comes with default mappings to dynamic field types defined in the Hydra Solr schema.xml file.  A copy of that is available :
-  https://github.com/projecthydra/hydra-head/blob/master/hydra-core/lib/generators/hydra/templates/solr_conf/conf/schema.xml
-	
-More information on the conventions followed for the dynamic solr fields is here:
-  https://github.com/projecthydra/hydra-head/wiki/Solr-Schema
-
-<pre>
-default_mapper = Solrizer::FieldMapper::Default.new
-
-# some of the default mappings in solrizer
-default_mapper.solr_name("foo",:string,:searchable)     # returns foo_tesim
-default_mapper.solr_name("foo",:date,:searchable)       # returns foo_dtsim
-default_mapper.solr_name("foo",:integer,:searchable     # returns foo_isim
-default_mapper.solr_name("foo",:string,:facetable)      # returns foo_sim
-default_mapper.solr_name("foo",:integer,:facetable)     # returns foo_iim
-default_mapper.solr_name("foo",:string,:sortable)       # returns foo_si
-default_mapper.solr_name("foo",:string,:displayable)    # returns foo_ssm
-</pre>
-
-## Using default indexing strategies
-
-<pre>
-solr_doc = {}
-Solrizer.insert_field(solr_doc, 'title', 'whatever', :searchable) 
-=> {"title_tesim"=>["whatever"]}
-
-Solrizer.insert_field(solr_doc, 'pub_date', 'Nov 2012', :sortable, :displayable) 
-=> {"title_tesim"=>["whatever"], "pub_date_ssi"=>["Nov 2012"], "pub_date_ssm"=>["Nov 2012"]}
-</pre>
-
-#### You can also index dates
-<pre>
-# as a date
-solr_doc = {}
-Solrizer.insert_field(solr_doc, 'pub_date', Date.parse('Nov 7th 2012'), :searchable)
-=> {"pub_date_dtsi"=>["2012-11-07T00:00:00Z"]}
-
-# or as a string
-solr_doc = {}
-Solrizer.insert_field(solr_doc, 'pub_date', Date.parse('Nov 7th 2012'), :sortable, :displayable)
-=> {"pub_date_ssi"=>["2012-11-07"], "pub_date_ssm"=>["2012-11-07"]}
-
-# or a string that is stored as a date
-solr_doc = {}
-Solrizer.insert_field(solr_doc, 'pub_date', 'Jan 29th 2013', :dateable)
-=> {"pub_date_dtsi"=>["2013-01-29T00:00:00Z"]}
-</pre>
-
-
-## Using a custom indexing strategy
-All you have to do is create your own index descriptor:
-<pre>
-solr_doc = {}
-displearchable = Solrizer::Descriptor.new(:integer, :indexed, :stored)
-Solrizer.insert_field(solr_doc, 'some_count', 45, displearchable)
-{"some_count_isi"=>["45"]}
-</pre>
-
-## Changing the behavior of a default descriptor
-
-Simply override the methods within Solrizer::DefaultDescriptors
-<pre>
-# before
-solr_doc = {}
-Solrizer.insert_field(solr_doc, 'title', 'foobar', :facetable)
-=> {"title_sim"=>["foobar"]}
-
-# redefine facetable:
-module Solrizer
-  module DefaultDescriptors
-    def self.facetable
-      Descriptor.new(:string, :indexed, :stored)
-    end
-  end
-end
-
-# after
-solr_doc = {}
-Solrizer.insert_field(solr_doc, 'title', 'foobar', :facetable)
-=> {"title_ssi"=>["foobar"]}
-</pre>
-
-
-## Creating your own Indexers
-<pre>
-module MyMappers
-  def self.mapper_one
-    Solrizer::Descriptor.new(:string, :indexed, :stored)
-  end
-end
-
-solr_doc = {}
-
-Solrizer::FieldMapper.descriptors = [MyMappers]
-=> [MyMappers]
-
-Solrizer.insert_field(solr_doc, 'title', 'foobar', :mapper_one)
-=> {"title_ssi"=>["foobar"]}
-</pre>
-
-## Using OM
-Same as it ever was:
-<pre>
-    t.main_title(:index_as=>[:facetable],:path=>"title", :label=>"title") { ... }
-</pre>
-
-But now you may also pass an Descriptor instance if that works for you:
-<pre>
-    indexer = Solrizer::Descriptor.new(:integer, :indexed, :stored)
-    t.main_title(:index_as=>[indexer],:path=>"title", :label=>"title") { ... }
-
-</pre>
-
-h3. Extractor and Extractor Mixins
-
-Solrizer::Extractor provides utilities for extracting solr fields from objects or inserting solr fields into documents:
-
-<pre>
-extractor = Solrizer::Extractor.new
-
-extractor.format_node_value(["foo     ","\n      bar"])                     # returns "foo bar"
-
-solr_doc = Hash.new
-extractor.insert_solr_field_value(solr_doc, "foo","bar")         # solr_doc is now {"foo" => ["bar"]}
-extractor.insert_solr_field_value(solr_doc,"foo","baz")    # solr_doc is now {"foo" => ["bar","baz"]}
-extractor.insert_solr_field_value(solr_doc, "boo","hoo")         # solr_doc is now {"foo" => ["bar","baz"], "boo" => ["hoo"]}
-</pre>
-
-h4. Solrizer provides some default mixins:
-
-* Solrizer::HTML::Extractor     -=>  provides html_to_solr method
-* Solrizer::XML::Extractor      -=>  provides xml_to_solr method
-
-<pre>
-xml = "<fields><foo>bar</foo><bar>baz</bar></fields>"
-
-extractor.xml_to_solr(xml)      # returns {:foo_tesim=>"bar", :bar_tesim=>"baz"}
-</pre>
-
-h4. Solrizer::XML::TerminologyBasedSolrizer
-
-Another powerful mixin for use with classes that include the OM::XML::Document module is Solrizer::XML::TerminologyBasedSolrizer.  
-The methods provided by this module map provides a robust way of mapping terms and solr fields via om terminologies.  A notable example 
-can be found in ActiveFedora::NokogiriDatatstream.
-
-
-h2. JMS Listener for Hydra Rails Applications
-
-h3. The executables: solrizer and solrizerd
-
-The solrizer gem provides two executables:
-
- * solrizer is a stomp consumer which listens for fedora.apim.updates and solrizes (or de-solrizes) objects accordingly. 
- * solrizerd is a wrapper script that spawns a daemonized version of solrizer and handles start|stop|restart|status requests. 
-
-h3. Usage 
-
-The usage for solrizerd is as follows: 
-
-<pre>
- solrizerd command --hydra_home PATH [options] 
-</pre>
-
-The commands are as follows:
- *  start      start an instance of the application 
- *  stop       stop all instances of the application 
- *  restart    stop all instances and restart them afterwards 
- *  status     show status (PID) of application instances 
-
-Required parameters:
-
---hydra_home: this is the path to your hydra rails applications' root directory.  Solrizerd needs this in order to load all your models and corresponding terminoligies.
-
-The options:
- *  -p, --port         Stomp port  61613 
- *  -o, --host         Host to connect to  localhost 
- *  -u, --user         User name for stomp listener  
- *  -w, --password     Password for stomp listener  
- *  -d, --destination  Topic to listen to (default: /topic/fedora.apim.update) 
- *  -h, --help         Display this screen 
-
-Note:
-
-Since the solrizer script must fire up your hydra rails application, it must have all the gems installed that your hydra instance needs.
-
-
-h2. Note on Patches/Pull Requests
- 
-* Fork the project.
-* Make your feature addition or bug fix.
-* Add tests for it. This is important so I don't break it in a
-  future version unintentionally.
-* Commit, do not mess with rake file, version, or history.
-  (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
-* Send me a pull request. Bonus points for topic branches.
-
-h2. Acknowledgements
-
-Technical Lead: Matt Zumwalt ("MediaShelf":http://yourmediashelf.com)
-
-Thanks to 
-
-Douglas Kim, who created the initial code base for Solrizer. 
-Chris Fitzpatrick, who patiently ran the first prototype through its paces for weeks.
-Bess Sadler, who created the JMS integration for Solrizer, generously served as a sounding board for numerous design issues around solr indexing, and pushes the technology forward with the skill of a true engineer.
-
-h2. Copyright
-
-Copyright (c) 2010 Matt Zumwalt. See LICENSE for details.