bento_search 1.3.0 → 1.4.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 07eca03117fe99d9677efc390aaef6c62d3cdef5
-  data.tar.gz: e7eeaff771953510f2f08828410c2db297f4cb36
+  metadata.gz: ee7b23e2add7fc9cadf6223574615febac70af86
+  data.tar.gz: ebc2f96b07206c4b7a4e4de5083a8ef0a02576ea
 SHA512:
-  metadata.gz: 324fb0e242743f393ca3403e6612e51f92a9f704e01aecdba7d8f2eb4bc68b6df07740dbb33fb2ff9120093d91d99d209b16288a4859827d260be72272bd1c3a
-  data.tar.gz: 6bbc568ac0485540414a7e9a01ae5ade8f17d20fbbc9fa0199bca3dfac1e5739771c6e9ea980b1c8e3fe8593499cbf57c3dfc3368b88b1c13ec2c7facebed2bf
+  metadata.gz: 4e7e9044ff5b1bdc115d828c05ddaa7af4190bed36ec28d3994cd8a4b7092f58d7086b1ed322183c32419d29fc2e311172722495fcde7b1b2afa5b4ec70324cf
+  data.tar.gz: 86adb80a03ef49a5a2d55d56447f92decb412084d246b092c54558c3ef215ede30517d73438fe9e2cce1c7e3ce3037b6a2e389707f32ef0f69951b685a044c29

data/README.md

@@ -5,7 +5,7 @@
 
 bento_search provides an abstraction/normalization layer for querying and
 displaying results from external search engines, in Ruby on Rails. Works with
-Rails 3.x or 4.x.  Currently tested only under ruby 1.9.3. (todo, change this soon)
+Rails 3.x or 4.x.   ruby 1.9.3+
 
 ### Goals: To help you
 
@@ -87,7 +87,7 @@ may be required for certain engines.
 ~~~~
 
 `results` are a [BentoSearch::Results](./app/models/bento_search/results.rb) object, which acts like an array of
-[BentoSearch::Item](./app/models/bento_search/results.rb) objects, along with some meta-information about the
+[BentoSearch::ResultItem](./app/models/bento_search/result_item.rb) objects, along with some meta-information about the
 search itself (pagination keys, etc).  BentoSearch::Results and Item fields
 are standardized accross engines. BentoSearch::Items provide semantic
 values (title, author, etc.), as available from the particular engine.
@@ -350,6 +350,46 @@ end
 There are additional details that might matter to you, for more info see the
 [wiki page](https://github.com/jrochkind/bento_search/wiki/Machine-Readable-Serialization-With-Atom)
 
+### Round-Trip Serialization to JSON
+
+You can serialize BentoSearch::Results to a simple straightforward JSON structure, and de-serialize
+them back into BentoSearch::Results. 
+
+~~~ruby
+json_str          = results.dump_to_json
+copy_of_results   = BentoSearch::Results.load_json(json_str)
+~~~
+
+Search context (query, start, per_page) are not serialized, and will be lost
+on de-serialization. 
+
+Unlike the Atom serialization, **the JSON serialization is of internal data
+state, without decoration.** Configuration context is not serialized.  
+
+However, the engine_id is included in serialization if present, 
+and configuration from the specified engine
+will be re-assigned on de-serialization.  This means if the configuration
+changed between serialization and de-serialization, you get the new stuff
+assigned on de-serialization. 
+
+The use case guiding JSON serialization is storage somewhere, and
+round-trip de-serialization in the current app context. 
+
+If you want to take de-serialized results that did not have an engine_id,
+or set configuration on them to a different engine (registered or not) you can:
+
+~~~ruby
+  restored = BentoSearch::Results.load_json(json_str)
+  some_engine.fill_in_search_metadata_for(restored)
+
+  # restored Results will have configuration (engine_id, decorators, etc)
+  # set to those configured on some_engine
+~~~
+
+If you want a serialization to be consumed by something other than an
+app using the bento_search gem, as an API, we recommend the [Atom serialization](https://github.com/jrochkind/bento_search/wiki/Machine-Readable-Serialization-With-Atom)
+instead. 
+
 ## Planned Features
 
 I am trying to keep BentoSearch as simple as it can be to conveniently meet

data/app/controllers/bento_search/search_controller.rb

@@ -32,7 +32,7 @@ module BentoSearch
   # We may provide fancier/nicer API for this in the future, if there's
   # demand. 
   class SearchController < BentoSearchController
-    class AccessDenied < Exception ; end
+    class AccessDenied < BentoSearch::Error ; end
 
     
     rescue_from AccessDenied, :with => :deny_access

data/app/models/bento_search/author.rb

@@ -1,5 +1,7 @@
 module BentoSearch
   class Author
+    include ::BentoSearch::Results::Serialization
+
     def initialize(args ={})
       args.each_pair do |key, value|
         send("#{key}=", value)
@@ -7,15 +9,15 @@ module BentoSearch
     end
     
     # Can be first name or initial, whatever source provides
-    attr_accessor :first
+    serializable_attr_accessor :first
     # last name/surname/family name as provided by source
-    attr_accessor :last
+    serializable_attr_accessor :last
     # middle name or initial, as and if provided by source
-    attr_accessor :middle
+    serializable_attr_accessor :middle
     
     # if source doens't provide seperate first/last, 
     # source may only be able to provide one big string, author_display
-    attr_accessor :display
+    serializable_attr_accessor :display
     
     def empty?
       first.blank? && last.blank? && middle.blank? && display.blank?

data/app/models/bento_search/link.rb

@@ -3,27 +3,30 @@ module BentoSearch
   #
   # label, url, other metadata about the link. 
   class Link
+    include BentoSearch::Results::Serialization
+
+    serializable_attr_accessor :label
     # url is normally a string, but can also be a Hash passed
     # to url_for in the local app. 
-    attr_accessor :label, :url
+    serializable_attr_accessor :url
     
     # Used both for HTML links and possibly later Atom. 
     # Must be a string, EITHER a complete URL (representing
     # vocab term), OR a legal short name from 
     # http://www.whatwg.org/specs/web-apps/current-work/multipage/links.html#linkTypes    
-    attr_accessor :rel
+    serializable_attr_accessor :rel
     
     # MIME content type may be used for both HMTL links and Atom
     # link 'type' attribute
-    attr_accessor :type
+    serializable_attr_accessor :type
     
     # Array of strings, used for CSS classes on this link, possibly
     # for custom styles/images etc. May be used in non-html link
     # contexts too. 
-    attr_accessor :style_classes
+    serializable_attr_accessor :style_classes
     
     # Suggested `target` attribute to render link with as html <a> 
-    attr_accessor :target
+    serializable_attr_accessor :target
     
     def initialize(hash = {})
       self.style_classes = []

data/app/models/bento_search/multi_searcher.rb

@@ -105,7 +105,7 @@ begin
       def start(*search_args)
         begin
           @results = self.engine.search(*search_args)
-        rescue Exception => e
+        rescue StandardError => e
           Rails.logger.error("\nBentoSearch:MultiSearcher caught exception: #{e}\n#{e.backtrace.join("   \n")}")
           # Make a fake results with caught exception. 
           @results = BentoSearch::Results.new

data/app/models/bento_search/registrar.rb

@@ -1,9 +1,10 @@
+require 'bento_search'
 
 # Holds a list of registered search engines with configuration. 
 # There's one global one referened by BentoSearch module, but one
 # might want to create multiple. 
 class BentoSearch::Registrar
-  class ::BentoSearch::NoSuchEngine < Exception ; end
+  class ::BentoSearch::NoSuchEngine < ::BentoSearch::Error ; end
   
   def initialize    
     @registered_engine_confs = {}

data/app/models/bento_search/result_item.rb

@@ -1,4 +1,7 @@
 require 'language_list'
+require 'bento_search/author'
+require 'bento_search/link'
+
 
 module BentoSearch
   # Data object representing a single hit from a search, normalized
@@ -14,6 +17,8 @@ module BentoSearch
     include ERB::Util # for html_escape for our presentational stuff
     include ActionView::Helpers::OutputSafetyHelper # for safe_join
 
+    include ::BentoSearch::Results::Serialization
+
     # Can initialize with a hash of key/values
     def initialize(args = {})
       args.each_pair do |key, value|
@@ -30,21 +35,24 @@ module BentoSearch
     # internal unique id for the document, from the particular
     # search service it came from. May be alphanumeric. May be nil
     # for engines that don't support it.
-    attr_accessor :unique_id
+    serializable_attr_accessor :unique_id
+    
 
     # If set to true, item will refuse to generate an openurl,
     # returning nil from #to_openurl or #openurl_kev
-    attr_accessor :openurl_disabled
+    serializable_attr_accessor :openurl_disabled
+    
 
     # Array (possibly empty) of BentoSearch::Link objects
     # representing additional links. Often SearchEngine's themselves
     # won't include any of these, but Decorators will be used
     # to add them in.
     attr_accessor :other_links
+    serializable_attr :other_links, :collection_of => "BentoSearch::Link"
 
     # * dc.title
     # * schema.org CreativeWork: 'name'
-    attr_accessor :title
+    serializable_attr_accessor :title    
     # backwards compat, we used to have separate titles and subtitles
     alias_method :complete_title, :title
 
@@ -52,6 +60,7 @@ module BentoSearch
     # Can be changed in actual presentation with a Decorator.
     # * schema.org CreativeWork: 'url'
     attr_accessor :link
+    serializable_attr :link
 
     # does the #link correspond to fulltext?  true or false -- or nil
     # for unknown/non-applicable. Not all engines will set.
@@ -61,6 +70,7 @@ module BentoSearch
     def link_is_fulltext=(v)
       @link_is_fulltext = v
     end
+    serializable_attr :link_is_fulltext
 
     # Our own INTERNAL controlled vocab for 'format'.
     #
@@ -102,7 +112,7 @@ module BentoSearch
     #
     # Note: We're re-thinking this, might allow uncontrolled
     # in here instead.
-    attr_accessor :format
+    serializable_attr_accessor :format    
 
     # Translated from internal format vocab at #format. Outputs
     # eg http://schema.org/Book
@@ -127,7 +137,7 @@ module BentoSearch
     # uncontrolled presumably english-language format string.
     # if supplied will be used in display in place of controlled
     # format.
-    attr_accessor :format_str
+    serializable_attr_accessor :format_str    
 
     # Language of materials. Producer can set language_code to an ISO 639-1 (two
     # letter) or 639-3 (three letter) language code. If you do this, you don't
@@ -143,7 +153,7 @@ module BentoSearch
     # #language_iso_639_2 (either may be null), or #language_str for uncontrolled
     # string. If engine just sets one of these, internals take care of filling
     # out the others. r
-    attr_accessor :language_code
+    serializable_attr_accessor :language_code    
     attr_writer :language_str
     def language_str
       @language_str || language_code.try do |code|
@@ -152,6 +162,7 @@ module BentoSearch
         end
       end
     end
+    serializable_attr :language_str
     # Returns a LanguageList gem language object-- from #language_code
     # if available, otherwise from direct language_str if available and
     # possible.
@@ -176,45 +187,46 @@ module BentoSearch
     # * prism:coverDate, year portion
     #
     # See also publication_date when you have a complete date
-    attr_accessor :year
+    serializable_attr_accessor :year
     # ruby stdlib Date object.
     attr_accessor :publication_date
+    serializable_attr :publication_date, :serializer => "Date"
 
-    attr_accessor :volume
-    attr_accessor :issue
-    attr_accessor :start_page
-    attr_accessor :end_page
+    serializable_attr_accessor :volume
+    serializable_attr_accessor :issue
+    serializable_attr_accessor :start_page
+    serializable_attr_accessor :end_page
 
     # source_title is often used for journal_title (and aliased
     # as #journal_title, although that may go away), but can
     # also be used for other 'container' titles. Book title for
     # a book chapter. Even web site or URL for a web page.
-    attr_accessor :source_title
+    serializable_attr_accessor :source_title
     alias_method :journal_title, :source_title
     alias_method :'journal_title=',  :'source_title='
 
 
-    attr_accessor :issn
-    attr_accessor :isbn
-    attr_accessor :oclcnum # OCLC accession number, WorldCat.
+    serializable_attr_accessor :issn
+    serializable_attr_accessor :isbn
+    serializable_attr_accessor :oclcnum # OCLC accession number, WorldCat.
 
-    attr_accessor :doi
-    attr_accessor :pmid
+    serializable_attr_accessor :doi
+    serializable_attr_accessor :pmid
 
     # usually used for books rather than articles
-    attr_accessor :publisher
+    serializable_attr_accessor :publisher
 
     # an openurl kev-encoded context object. optional,
     # only if source provides one that may be better
     # than can be constructed from individual elements above
-    attr_accessor :openurl_kev_co
+    serializable_attr_accessor :openurl_kev_co
 
     # Short summary of item.
     # Mark .html_safe if it includes html -- creator is responsible
     # for making sure html is safely sanitizied and/or stripped,
     # rails ActionView::Helpers::Sanistize #sanitize and #strip_tags
     # may be helpful.
-    attr_accessor :abstract
+    serializable_attr_accessor :abstract
 
     # An ARRAY of string query-in-context snippets. Will usually
     # have highlighting <b> tags in it. Creator is responsible
@@ -225,14 +237,16 @@ module BentoSearch
     # with same content formatted differently (array of multiple vs
     # one combined string), some engines they may be different. 
     attr_accessor :snippets
+    serializable_attr :snippets
 
     # An array (order matters) of BentoSearch::Author objects
     # add authors to it with results.authors << Author
     attr_accessor :authors
+    serializable_attr :authors, :collection_of => "BentoSearch::Author"
 
     # engine-specific data not suitable for abstract API, usually
     # for internal use.
-    attr_accessor :custom_data
+    serializable_attr_accessor :custom_data
 
     # Copied over from engine configuration usually, a string
     # qualified name of a decorator class. Can be nil for default.
@@ -248,6 +262,8 @@ module BentoSearch
     # into a backpointing reference instead? And user cover-methods
     # for it? Nice thing about the configuration has instead is it's
     # easily serializable, it's just data.
+    #
+    # Although we intentionally do NOT include these in JSON serialization, ha. 
     attr_accessor :display_configuration
     attr_accessor :engine_id
 

data/app/models/bento_search/results/serialization.rb

@@ -0,0 +1,136 @@
+require 'bento_search/results'
+require 'active_support/concern'
+require 'json'
+require 'date'
+
+# Call #dump_to_json on a BentoSearch value object (such as BentoSearch::Result or ::Author)
+# to get it in Json 
+#
+# Values marked with serializable_attr in BentoSearch::Result are
+# included in seralization. 
+#
+# At present metadata and configuration are NOT serialized: #decorator, #display_configuration,
+# and #engine_id are not included in the serialization, so when loaded from serialization,
+# ResultItems will not have such things set. 
+# 
+# * Works by getting and setting instance variables directly, ignores getters/setters
+#
+# * This means decorated values are NOT included in serialization, the raw
+#   values are what is serialized. This is intended, we serialize internal
+#   state, not decoration which can be recreated. You should make sure the decorators you
+#   want are applied after de-serialization. 
+#
+# * preserves html_safety status in serialization, by adding extra `_attr_htmlsafe: true` key/value
+#
+module BentoSearch::Results::Serialization
+  extend ActiveSupport::Concern
+
+  included do
+    class_attribute :_serializable_attrs, :_serializable_attr_options
+    self._serializable_attrs = []
+    self._serializable_attr_options = {}
+  end
+
+  class_methods do
+    # Just a macro to mark a property name serializable -- the name is
+    # of an instance method that will be included in our serializations
+    # and de-serializations. 
+    #
+    # Options:
+    #   * collection_of: String fully qualified name of a class that is
+    #       is also BentoSearch::Results::Serialization, the attribute
+    #       is an array of these. 
+    #   * serializer: String fully qualified class name of a serializer
+    #        class that has a `dump` and a `load` for individual values,
+    #        we just use it for Date now, see BentoSearch::Results::Serialization::Date
+    def serializable_attr(symbol, options = nil)
+      symbol = symbol.to_s
+      self._serializable_attrs << symbol
+      if options 
+        self._serializable_attr_options[symbol] = options
+      end
+    end
+
+    # convenience macro to do attr_accessor AND mark it
+    # serializable
+    def serializable_attr_accessor(symbol)
+      self.send(:attr_accessor, symbol)
+      self.serializable_attr symbol
+    end
+
+    def from_internal_state_hash(hash)
+      o = self.new
+      hash.each_pair do |key, value|
+        key = key.to_s
+
+        next if key =~ /\A_.*_htmlsafe\Z/
+
+
+        if _serializable_attr_options[key] && _serializable_attr_options[key][:collection_of]
+          klass = qualified_const_get(_serializable_attr_options[key][:collection_of])
+          value = value.collect do |item|
+            klass.from_internal_state_hash(item)
+          end
+        end
+
+        if _serializable_attr_options[key] && _serializable_attr_options[key][:serializer]
+          klass = qualified_const_get(_serializable_attr_options[key][:serializer])
+          value = klass.load(value)
+        end
+
+        if hash["_#{key}_htmlsafe"] == true && value.respond_to?(:html_safe)
+          value = value.html_safe
+        end
+
+        o.instance_variable_set("@#{key}", value)
+      end
+
+      return o
+    end
+
+    def load_json(json_str)
+      self.from_internal_state_hash( JSON.parse! json_str )
+    end
+
+  end
+
+  def internal_state_hash
+    hash = {}
+    self._serializable_attrs.each do |accessor|
+      accessor = accessor.to_s
+      value = self.instance_variable_get("@#{accessor}")
+
+      next if value.blank?
+
+      if _serializable_attr_options[accessor] && _serializable_attr_options[accessor][:serializer]
+        klass = self.class.qualified_const_get(_serializable_attr_options[accessor][:serializer])
+        value = klass.dump(value)
+      elsif value.respond_to?(:to_ary)
+        value = value.to_ary.collect do |item|
+          item.respond_to?(:internal_state_hash) ? item.internal_state_hash : item
+        end
+      end
+
+      hash[accessor] = value
+
+      if value.respond_to?(:html_safe?) && value.html_safe?
+        hash["_#{accessor}_htmlsafe"] = true
+      end
+    end
+    return hash
+  end
+
+  def dump_to_json
+    JSON.dump self.internal_state_hash
+  end
+
+  class Date
+    def self.dump(datetime)
+      datetime.iso8601
+    end
+    def self.load(str)
+      ::Date.iso8601(str)
+    end
+  end
+
+end

data/app/models/bento_search/results.rb

@@ -1,10 +1,20 @@
+require 'json'
+
 module BentoSearch
   # An array-like object (in fact it-subclasses Array) that holds
   # a page of search results. But also has some meta-data about the
   # search itself (query, paging, etc). 
-  
+  #
   # If #error is non-nil, object  may not have real results, but
   # be an error. You can use failed? to see. 
+  #
+  # Serializes as a Hash including ONLY the serialized results themselves,
+  # and the engine ID.  Search context (total items, start, etc) are not
+  # included in serialization. Configuration context is also not serialized,
+  # although since the engineID is, it can be reconstructed on de-serialization. 
+  #
+  # Serialization isn't actually implemented with the BentoSearch::Results::Serialization
+  # module, cause it didn't work out, just re-implemented here with same methods. 
   class Results < ::Array
     attr_accessor :total_items
     # 0-based index into total results, used for pagination
@@ -48,6 +58,7 @@ module BentoSearch
     # Registered id of engine used to create these results, 
     # may be nil if used with an unregistered engine.
     attr_accessor :engine_id
+
     
     # Returns a BentoSearch::Results::Pagination, that should be suitable
     # for passing right to Kaminari (although Kaminari isn't good about doc/specing
@@ -63,6 +74,40 @@ module BentoSearch
     def inspect
       "<BentoSearch::Results #{super} #{'FAILED' if self.failed?}>"
     end
+
+    # Serialization
+    def internal_state_hash
+      {
+        "engine_id" => self.engine_id,
+        "result_items" => self.collect {|i| i.internal_state_hash}
+      }
+    end
+
+    # Creates a Results object from an internal_state_hash, and restores
+    # it's configuration from engine_id
+    def self.from_internal_state_hash(hash)
+      results = BentoSearch::Results.new
+      results.engine_id = hash["engine_id"]
+      hash["result_items"].each do |item_hash|
+        results << BentoSearch::ResultItem.from_internal_state_hash(item_hash)
+      end
+
+      if results.engine_id
+        BentoSearch.get_engine(results.engine_id).fill_in_search_metadata_for(results, {})
+      end
+
+      return results
+    end
+
+    def dump_to_json
+      JSON.dump self.internal_state_hash
+    end
+
+    def self.load_json(json_str)
+      from_internal_state_hash JSON.parse(json_str)
+    end
+
+
     
   end
 end

data/app/models/bento_search/search_engine.rb

@@ -10,9 +10,13 @@ require 'nokogiri'
 module BentoSearch
   # Usually raised by #get on an engine, when result for specified identifier
   # can't be found. 
-  class NotFound < Exception ; end
+  class ::BentoSearch::NotFound < ::BentoSearch::Error ; end
   # Usually raised by #get when identifier results in more than one record. 
-  class TooManyFound < Exception ; end
+  class ::BentoSearch::TooManyFound < ::BentoSearch::Error ; end
+  # Raised for problem contacting or unexpected response from
+  # remote service. Not yet universally used. 
+  class ::BentoSearch::FetchError < ::BentoSearch::Error ; end
+
   
   # Module mix-in for bento_search search engines. 
   #
@@ -217,16 +221,7 @@ module BentoSearch
       fill_in_search_metadata_for(results, arguments)
             
       results.timing = (Time.now - start_t)
-            
-      results.each do |item| 
-        # We copy some configuraton info over to each Item, as a convenience
-        # to display logic that may have decide what to do given only an item,
-        # and may want to parameterize based on configuration.
-        item.engine_id              = results.engine_id 
-        item.decorator              = configuration.lookup!("for_display.decorator")
-        item.display_configuration  = configuration.for_display
-      end
-        
+                    
       return results
     rescue *auto_rescue_exceptions => e
       # Uncaught exception, log and turn into failed Results object. We
@@ -252,16 +247,26 @@ module BentoSearch
     
     # SOME of the elements of Results to be returned that SearchEngine implementation
     # fills in automatically post-search. Extracted into a method for DRY in
-    # error handling to try to fill these in even in errors. And *possible*
-    # experimental use in other classes for same thing is why method is
-    # public, see MultiSearcher.     
-    def fill_in_search_metadata_for(results, normalized_arguments)
+    # error handling to try to fill these in even in errors. Also can be used
+    # as public method for de-serialized or mock results. 
+    def fill_in_search_metadata_for(results, normalized_arguments = {})
       results.search_args           = normalized_arguments
       results.start = normalized_arguments[:start] || 0
       results.per_page = normalized_arguments[:per_page]
       
       results.engine_id             = configuration.id
-      results.display_configuration = configuration.for_display                        
+      results.display_configuration = configuration.for_display
+
+      # We copy some configuraton info over to each Item, as a convenience
+      # to display logic that may have decide what to do given only an item,
+      # and may want to parameterize based on configuration.
+      results.each do |item| 
+        item.engine_id              = configuration.id 
+        item.decorator              = configuration.lookup!("for_display.decorator")
+        item.display_configuration  = configuration.for_display
+      end
+
+      results
     end
         
 

data/app/search_engines/bento_search/eds_engine.rb

@@ -539,7 +539,7 @@ class BentoSearch::EdsEngine
   # there's a short reason in #message, but also
   # possibly an http_status and http_body copied
   # from error EDS response. 
-  class EdsCommException < Exception
+  class EdsCommException < ::BentoSearch::FetchError
     attr_accessor :http_status, :http_body
     def initialize(message, status = nil, body = nil)
       super(message)