bento_search 0.9.0 → 1.0.0

data/README.md

@@ -2,10 +2,6 @@
 
 [![Build Status](https://secure.travis-ci.org/jrochkind/bento_search.png)](http://travis-ci.org/jrochkind/bento_search)
 
-(Fairly robust and stable at this point, but still pre-1.0 release, may
-be some breaking api changes before 1.0, but probably not too many, it's
-looking pretty good). 
-
 bento_search provides an abstraction/normalization layer for querying and 
 displaying results from external search engines, in Ruby on Rails. Requires
 Rails3 and tested only under ruby 1.9.3. 
@@ -88,14 +84,14 @@ may be required for certain engines.
     results = engine.search("a query")
 ~~~~
 
-`results` are a BentoSearch::Results object, which acts like an array of
-BentoSearch::Item objects, along with some meta-information about the
+`results` are a [BentoSearch::Results](./app/models/bento_search/results.rb) object, which acts like an array of
+[BentoSearch::Item](./app/models/bento_search/results.rb) objects, along with some meta-information about the
 search itself (pagination keys, etc).  BentoSearch::Results and Item fields
 are standardized accross engines. BentoSearch::Items provide semantic
 values (title, author, etc.), as available from the particular engine. 
 
 To see which engines come bundled with BentoSearch, and any special 
-engine-specific instructions, look at BentoSearch source in `./app/search_engines`
+engine-specific instructions, look at BentoSearch source in [`./app/search_engines/bento_search`](./app/search_engines/bento_search)
 
 ### Register engines in global configuration
 
@@ -211,7 +207,7 @@ An engine instance advertises it's maximum per-page values.
 bento_search fixes the default per_page at 10.     
     
 For help creating your UI, you can ask a BentoSearch::Results for
-`results.pagination`, which returns a BentoSearch::Results::Pagination
+`results.pagination`, which returns a [BentoSearch::Results::Pagination](app/models/bento_search/results/pagination.rb)
 object which should be suitable for passing to [kaminari](https://github.com/amatsuda/kaminari)
 `paginate`, or else have convenient methods for roll your own pagination UI. 
 Kaminari's paginate method:
@@ -259,7 +255,7 @@ that Celluloid uses multi-threading in such a way that you might need
 to turn Rails config.cache_classes=true even in development.
  
 
-For more info, see BentoSearch::MultiSearcher. 
+For more info, see [BentoSearch::MultiSearcher](./app/models/bento_search/multi_searcher.rb). 
 
 ### Delayed results loading via AJAX (actually more like AJAHtml)
 
@@ -289,7 +285,8 @@ link resolver.
 BentoSearch::Items can have a main link associated with them (generally 
 hyperlinked from title), as well as a list of additional links. Most engines
 do not provide additional links by default, custom local Decorators would
-be used to add them. See wiki for more info on decorators, and BentoSearch::Link
+be used to add them. See [wiki on display cusotmization](https://github.com/jrochkind/bento_search/wiki/Customizing-Results-Display) 
+for more info on decorators, and [BentoSearch::Link](app/models/bento_search/link.rb)
 for fields. 
      
 ### OpenURL and metadata
@@ -307,8 +304,8 @@ documented at ResultItem#format). As well as how well the #to_openurl routine
 handles all edge cases (OpenURL can be weird). As edge cases are discovered, they
 can be solved. 
 
-See `./app/item_decorators/bento_search/openurl_add_other_link.rb` for an example
-of using item decorators to add a link to your openurl resover to an item when
+See [`./app/item_decorators/bento_search/openurl_add_other_link.rb`](./app/item_decorators/bento_search/openurl_add_other_link.rb) 
+for an example of using item decorators to add a link to your openurl resover to an item when
 displayed.
 
 ### Exporting (eg as RIS) and get by unique_id
@@ -322,8 +319,34 @@ the RIS format, suitable for import into EndNote, Refworks, etc.
 
 Accomodating actual exports into the transactional flow of a web app can be
 tricky, and often requires use of the `result_item#unique_id` and 
-`engine.get( unique_id )` features. See the wiki at 
+`engine.get( unique_id )` features. See the wiki on [exports and #unique_id](https://github.com/jrochkind/bento_search/wiki/Exports-and-the-get-by-unique_id-feature)
+
+### Machine-readable serialization in Atom
+
+Translation of any BentoSearch::Results to an Atom response that is enhanced to
+include nearly all the elements of each BentoSearch::ResultItem, so can serves
+well as machine-readable api response in general, not just for Atom feed readers. 
+
+You can use the  [`bento_search/atom_results`](./app/views/bento_search/atom_results.atom.builder) view template, perhaps 
+in your action method like so:
+
+~~~ruby
+# ...
+respond_to do |format|
+   format.html # default view
+   format.atom do
+      render( :template => "bento_search/atom_results",              
+              :locals   => {
+                 :atom_results     => @results,
+                 :feed_name        => "Acme results",
+                 :feed_author_name => "MyCorp"
+              }      
+      ) 
+end   
+~~~
 
+There are additional details that might matter to you, for more info see the 
+[wiki page](https://github.com/jrochkind/bento_search/wiki/Machine-Readable-Serialization-With-Atom) 
 
 ## Planned Features
 
@@ -340,7 +363,17 @@ Probably:
   a normalized cross-engine way. 
 
 Other needs or suggestions?
-  
+
+## Backwards compat
+
+We are going to try to be strictly backwards compatible with all post 1.0 
+releases that do not increment the major version number (semantic versioning). 
+
+As a general rule, we're going to let our tests enforce this -- if a test has
+to be changed to pass with new code, that's a very strong sign that it is
+not a backwards-compat change, and you should think _very_ carefully to 
+be sure it is an exception to this rule before changing any existing tests
+for new functionality. 
 
 ## Developing
 

data/app/item_decorators/bento_search/standard_decorator.rb

@@ -87,18 +87,14 @@ module BentoSearch
       self.any_present?(:source_title, :publisher, :start_page)
     end
     
-    # Put together title and subtitle if neccesary. 
+    # Mix-in a default missing title marker for empty titles
+    # (Used to combine title and subtitle when those were different fields)
     def complete_title
-      t = self.title
-      if self.subtitle
-        t = safe_join([t, ": ", self.subtitle], "")        
-      end
-      
-      if t.blank?
-        t = I18n.translate("bento_search.missing_title")
-      end
-      
-      return t
+      if self.title.present?
+        self.title
+      else
+        I18n.translate("bento_search.missing_title")
+      end      
     end
     
     
@@ -125,7 +121,7 @@ module BentoSearch
       return result_elements.join(", ").html_safe
     end
     
-        # A display method, this is like #langauge_str, but will be nil if
+    # A display method, this is like #langauge_str, but will be nil if
     # the language_code matches the current default locale, used
     # for printing language only when not "English" normally. 
     #
@@ -152,6 +148,28 @@ module BentoSearch
         
       return value.blank? ? nil : value        
     end
+    
+    # A unique opaque identifier for a record may sometimes be
+    # required, for instance in Atom. 
+    #
+    # We here provide a really dumb implementation, if and only if
+    # the result has an engine_id and unique_id available, (and
+    # a #root_url is available) by basically concatenating them to 
+    # app base url. 
+    #
+    # That's pretty lame, probably not resolvable, but best we
+    # can do without knowing details of host app. You may want
+    # to over-ride this in a decorator to do something more valid
+    # in an app-specific way. 
+    #
+    # yes uri_identifier is like PIN number, deal with it. 
+    def uri_identifier
+      if self.engine_id.present? && self.unique_id.present? && _h.respond_to?(:root_url)
+        "#{_h.root_url.chomp("/")}/bento_search_opaque_id/#{CGI.escape self.engine_id}/#{CGI.escape self.unique_id}"
+      else 
+        nil
+      end
+    end
 
     
     ###################

data/app/models/bento_search/link.rb

@@ -13,6 +13,10 @@ module BentoSearch
     # http://www.whatwg.org/specs/web-apps/current-work/multipage/links.html#linkTypes    
     attr_accessor :rel
     
+    # MIME content type may be used for both HMTL links and Atom
+    # link 'type' attribute
+    attr_accessor :type
+    
     # Array of strings, used for CSS classes on this link, possibly
     # for custom styles/images etc. May be used in non-html link
     # contexts too. 

data/app/models/bento_search/multi_searcher.rb

@@ -102,8 +102,10 @@ class BentoSearch::MultiSearcher
         Rails.logger.error("\nBentoSearch:MultiSearcher caught exception: #{e}\n#{e.backtrace.join("   \n")}")
         # Make a fake results with caught exception. 
         @results = BentoSearch::Results.new
+        self.engine.fill_in_search_metadata_for(@results, self.engine.normalized_search_arguments(search_args))
+        
         @results.error ||= {}
-        @results.error["exception"] = e        
+        @results.error["exception"] = e           
       end
     end
     

data/app/models/bento_search/registrar.rb

@@ -24,13 +24,42 @@ class BentoSearch::Registrar
   #
   # The first parameter identifier, eg "gbs", may be used in some
   # URLs, for AJAX etc. 
-  def register_engine(id, &block)
-    conf = Confstruct::Configuration.new(&block)
+  #
+  # You can also pass in a hash or hash-like object (including
+  # a configuration object returned by a prior register_engine)
+  # instead of or in addition to the block 'dsl' -- this can be used
+  # to base one configuration off another, with changes:
+  #
+  #     BentoSearch.register_engine("original", {
+  #       :engine => "Something",
+  #       :title => "Original",
+  #       :shared => "shared"
+  #     })
+  #
+  #     BentoSearch.register_engine("derived") do |conf|
+  #        conf.title = "Derived"
+  #     end
+  #
+  # Above would not change 'shared' in 'original', but would
+  # over-ride 'title' in 'derived', without changing 'title' in
+  # 'original'. 
+  def register_engine(id, conf_data = nil, &block)
+    conf = Confstruct::Configuration.new
+    
+    # Make sure we make a deep_copy so any changes don't mutate
+    # the original. Confstruct can be unpredictable. 
+    if conf_data.present?
+      conf_data = Confstruct::Configuration.new(conf_data).deep_copy
+    end
+    
+    conf.configure(conf_data, &block)
     conf.id = id.to_s
     
     raise ArgumentError.new("Must supply an `engine` class name") unless conf.engine
     
     @registered_engine_confs[id] = conf    
+    
+    return conf
   end
   
   # Get a configured SearchEngine, using configuration and engine

data/app/models/bento_search/result_item.rb

@@ -44,12 +44,8 @@ module BentoSearch
     # * dc.title 
     # * schema.org CreativeWork: 'name'    
     attr_accessor :title
-    
-    # When an individual seperate subtitle is available. 
-    # May also be nil with subtitle in "title" field after colon. 
-    # 
-    # * 
-    attr_accessor :subtitle
+    # backwards compat, we used to have separate titles and subtitles
+    alias_method :complete_title, :title
     
     # usually a direct link to the search provider's 'native' page. 
     # Can be changed in actual presentation with a Decorator.
@@ -65,12 +61,24 @@ module BentoSearch
       @link_is_fulltext = v
     end
     
-    # normalized controlled vocab title, important this is supplied
-    # if possible for OpenURL generation and other features. 
+    # Our own INTERNAL controlled vocab for 'format'. 
+    #
+    # Important that this be supplied by engine for maximum
+    # success of openurl, ris export, etc. 
+    #
+    # This vocab is based on schema.org CreativeWork 'types', 
+    # but supplemented with values we needed not present in schema.org. 
+    # String values are last part of schema.org URLs, symbol values are custom.
+    #
+    # However, for backwards compat, values that didn't exist in schema.org
+    # when we started but later came to exist -- we still use our string
+    # values. If you actually want a schema.org url, see #schema_org_type_url
+    # which translates as needed. 
     #
     # schema.org 'type' that's a sub-type of CreativeWork. 
     # should hold a string that, when appended to "http://schema.org/"
-    # is a valid schema.org type uri, that sub-types CreativeWork. Eg.
+    # is a valid schema.org type uri, that sub-types CreativeWork. Ones
+    # we have used:
     # * Article
     # * Book
     # * Movie
@@ -93,7 +101,27 @@ module BentoSearch
     #
     # Note: We're re-thinking this, might allow uncontrolled
     # in here instead. 
-    attr_accessor :format    
+    attr_accessor :format
+
+    # Translated from internal format vocab at #format. Outputs
+    # eg http://schema.org/Book
+    # Uses the @@format_to_schema_org hash for mapping from
+    # certain internal symbol values to schema org value, where
+    # possible. 
+    #
+    # Can return nil if we don't know a schema.org type
+    def schema_org_type_url    
+      if format.kind_of? String
+        "http://schema.org/#{format}"
+      elsif mapped = @@format_to_schema_org[format]
+        "http://schema.org/#{mapped}"
+      else
+        nil
+      end
+    end
+    @@format_to_schema_org = {
+      :report => "Article",
+    }
     
     # uncontrolled presumably english-language format string.
     # if supplied will be used in display in place of controlled
@@ -110,10 +138,10 @@ module BentoSearch
     # Manually set language_str will over-ride display string calculated from
     # language_code. 
     # 
-    # Consumers can look at language_code or language_str regardless (although
-    # either or both may be nil). You can get a language_list gem obj from
-    # language_obj, and use to normalize to a 
-    # 2- or 3-letter from language_code that could be either. 
+    # Consumers that want a language code can use #language_iso_639_1 or
+    # #language_iso_639_2 (either may be null), or #language_str for uncontrolled
+    # string. If engine just sets one of these, internals take care of filling
+    # out the others. r
     attr_accessor :language_code
     attr_writer :language_str
     def language_str
@@ -131,6 +159,16 @@ module BentoSearch
       @language_obj ||= LanguageList::LanguageInfo.find( self.language_code )
     end
     
+    # Two letter ISO language code, or nil
+    def language_iso_639_1
+      language_obj.try { |l| l.iso_639_1 }
+    end
+    
+    # Three letter ISO language code, or nil
+    def language_iso_639_3
+      language_obj.try {|l| l.iso_639_3 }
+    end
+    
     # year published. a ruby int
     # PART of:. 
     # * schema.org CreativeWork "datePublished", year portion

data/app/models/bento_search/search_engine.rb

@@ -213,18 +213,11 @@ module BentoSearch
       arguments = normalized_search_arguments(*arguments)
 
       results = search_implementation(arguments)
-            
-      
-      # standard result metadata
-      results.start = arguments[:start] || 0
-      results.per_page = arguments[:per_page]
-      
-      results.search_args   = arguments
-      results.engine_id     = configuration.id
       
+      fill_in_search_metadata_for(results, arguments)
+            
       results.timing = (Time.now - start_t)
             
-      results.display_configuration = configuration.for_display
       results.each do |item| 
         # We copy some configuraton info over to each Item, as a convenience
         # to display logic that may have decide what to do given only an item,
@@ -249,14 +242,27 @@ module BentoSearch
       failed.error ||= {}
       failed.error[:exception] = e
       
-      failed.search_args           = arguments
-      failed.engine_id             = configuration.id
-      failed.display_configuration = configuration.for_display
       failed.timing                = (Time.now - start_t)
+      
+      fill_in_search_metadata_for(failed, arguments)
 
       
       return failed
     end
+    
+    # SOME of the elements of Results to be returned that SearchEngine implementation
+    # fills in automatically post-search. Extracted into a method for DRY in
+    # error handling to try to fill these in even in errors. And *possible*
+    # experimental use in other classes for same thing is why method is
+    # public, see MultiSearcher.     
+    def fill_in_search_metadata_for(results, normalized_arguments)
+      results.search_args           = normalized_arguments
+      results.start = normalized_arguments[:start] || 0
+      results.per_page = normalized_arguments[:per_page]
+      
+      results.engine_id             = configuration.id
+      results.display_configuration = configuration.for_display                        
+    end
         
 
     # Take the arguments passed into #search, which can be flexibly given

data/app/search_engines/bento_search/google_books_engine.rb

@@ -113,8 +113,7 @@ module BentoSearch
                       
       item.unique_id             = item_response["id"]
       
-      item.title          = v_info["title"] 
-      item.subtitle       = v_info["subtitle"] 
+      item.title          = format_title(v_info)
       item.publisher      = v_info["publisher"]
       # previewLink gives you your search results highlighted, preferable
       # if it exists. 
@@ -255,6 +254,14 @@ module BentoSearch
       end
       return nil            
     end
+    
+    def format_title(v_info)
+      title = v_info["title"]
+      if v_info["subtitle"]
+        title = "#{title}: #{v_info["subtitle"]}"
+      end
+      return title
+    end
         
   end
 end

data/app/search_engines/bento_search/google_site_search_engine.rb

@@ -99,7 +99,7 @@ class BentoSearch::GoogleSiteSearchEngine
     10
   end
   
-  def self.required_configuation
+  def self.required_configuration
     [:api_key, :cx]
   end
   

data/app/search_engines/bento_search/summon_engine.rb

@@ -121,11 +121,7 @@ class BentoSearch::SummonEngine
       
       item.unique_id      = first_if_present doc_hash["ID"]
       
-      item.title          = handle_highlighting( first_if_present doc_hash["Title"] )
-      item.custom_data["raw_title"] = handle_highlighting( first_if_present(doc_hash["Title"]) , :strip => true)
-      
-      item.subtitle       = handle_highlighting( first_if_present doc_hash["Subtitle"] )# TODO is this right?
-      item.custom_data["raw_subtitle"] = handle_highlighting( first_if_present(doc_hash["Subtitle"]), :strip => true )
+      item.title          = format_title(doc_hash)
       
       item.link           = doc_hash["link"]
       # Don't understand difference between hasFullText and
@@ -356,6 +352,18 @@ class BentoSearch::SummonEngine
       :strip => options[:strip]
       )          
   end
+  
+  # combine title and subtitle into one title, 
+  def format_title(doc_hash)
+    title          = first_if_present  doc_hash["Title"]  
+    subtitle       = first_if_present doc_hash["Subtitle"] 
+    
+    if subtitle.present?
+      title = "#{title}: #{subtitle}"
+    end
+    
+    return handle_highlighting( title ) 
+  end
     
   def self.required_configuration
     [:access_id, :secret_key]