bento_search 0.6.0 → 0.7.0

data/README.md

@@ -7,16 +7,46 @@ be some breaking api changes before 1.0, but probably not too many, it's
 looking pretty good). 
 
 bento_search provides an abstraction/normalization layer for querying and 
-displaying results for external search engines, in Ruby on Rails. Requires
+displaying results from external search engines, in Ruby on Rails. Requires
 Rails3 and tested only under ruby 1.9.3. 
 
-* It is focused on use cases for academic libraries, but may be useful in generic
-cases too. Initially, engine adapters are planned to be provided for: 
-Google Books, Scopus, SerialSolutions Summon, Ex Libris Primo, 
-EBSCO Discovery Service, EBSCO traditional 'EIT' api, Google Site Search.  Most
-of these search engines require a vendor license to use. 
+### Goals: To help you
 
-* bento_search could be considered building blocks for a type of 'federated
+* **Get up and running as quickly as possible** with searching and displaying
+  results from a  third-party service. Solutions to idiosyncracies and
+  undocumented workarounds are encoded in a shared codebase, which abstracts
+  everything to a good, simple code API giving you building blocks to focus
+  on your needs, not the search service's problems. 
+* Let you switch out one search service for another in an already built
+  application with as little code rewriting as possible. **Avoid vendor lock-in**. 
+* Give you the harness to **write adapters for new search services**, without
+  having to rewrite common general functionality, just focus on the interface
+  with the new API you want to support. 
+
+bento_search is focused on use cases for academic libraries, which is mainly
+evidenced by the search engine adapters currently included, and by the
+generalized domain models including fields that matter in our domain (issn,
+vol/issue/page, etc), and some targetted functionality (OpenURL generation). 
+But it ought to be useful for more general basic use
+cases too (we include a google site search adapter for instance). 
+
+Adapters currently included in bento_search
+
+* Google Books (requires free api key)
+* Scopus (requires license)
+* Serial Solution Summon (requires license)
+* Ex Libris Primo (requires license)
+* EBSCO Discovery Service (requires license)
+* EBSCOHost 'traditional' API (requires license)
+* WorldCat Search (requires OCLC membership to get api key)
+* Google Site Search (requires sign-up for more than 100 searches/day)
+
+
+
+
+### Scope of functionality
+
+bento_search could be considered building blocks for a type of 'federated
 search' functionality, but it does not and will never support merging results
 from multiple engines into one result set. It is meant to support displaying the
 first few results from multiple engines on one page, "bento box" style (as
@@ -24,18 +54,18 @@ named by Tito Sierra@NCSU), as well as more expanded single-search-on-a-page
 uses. 
 
 * bento_search provides abstract functionality for pagination, sorting, 
-and single-field-specified queries. Faceting, limiting, and 'advanced'
-multi-field searches are not yet supported, but planned. Not all
-search engine adapters support all features.  Search engine adapters can
+and single-field-specified queries. Faceting, generalized limiting, and 'advanced'
+multi-field searches are not yet supported, but possibly will be built
+out in the future. 
+
+Not all search engine adapters support all features.  Some engines offer
+engine-specific features, such as limiting. Search engine adapters can
 declare search fields and sort options with 'semantics', so you can for
 instance search or sort by 'title' across search engines without regard
 to internal engine-specific field names. 
 
 bento_search is designed to allow code to be written agnostic of the search
-provider, so you can switch out the search provider, minimizing dependent
-code in your app that needs to be rewritten. As well as letting you get
-started quick without reinventing the wheel and figuring out poorly
-documented vendor API's yourself. 
+provider, so you can switch out the search provider. 
 
 See code-level api documentation for more details, especially at
 BentoSearch::SearchEngine. http://rubydoc.info/gems/bento_search/frames/
@@ -53,9 +83,11 @@ are a few standard keys (see BentoSearch::SearchEngine), and others that
 may be engine-specific. Some engine-specific keys (such as api auth keys) 
 may be required for certain engines. 
 
+~~~~ruby
     engine = BentoSearch::GoogleBooksEngine.new(:api_key => "my_gbs_api_key")
     results = engine.search("a query")
-    
+~~~~
+
 `results` are a BentoSearch::Results object, which acts like an array of
 BentoSearch::Item objects, along with some meta-information about the
 search itself (pagination keys, etc).  BentoSearch::Results and Item fields
@@ -72,16 +104,20 @@ required for certain functionality (like out-of-the-box AJAX loading).
 
 In an initializer in your app, like say `./config/initializers/bento_search.rb`:
 
+~~~~ruby
     BentoSearch.register_engine("gbs") do |conf|
        conf.engine = "BentoSearch::GoogleBooksEngine"
        conf.api_key = "my_google_api_key"
        # any other configuration
     end
+~~~~
     
 Then you can refer to it, for instance in a controller, by the id you registered:
 
+~~~~ruby
     @results = BentoSearch.get_engine("gbs").search("my query")
-    
+~~~~
+
 ### Display results
 
 You can of course write your own code to display a BentoSearch::Results object
@@ -89,47 +125,70 @@ however you like. But BentoSearch comes with a helper method for displaying
 a list of BentoSearch::Results in a standard way, using the bento_search
 helper method. 
 
-    <%= bento_search(@results) %>
+~~~~ruby
+    <%= bento_search @results %>
+~~~~
+
+See also the [Customizing Results Display wiki page](https://github.com/jrochkind/bento_search/wiki/Customizing-Results-Display). 
     
 ### Fielded searching.
 
 You can search by an internal engine-specific field name:
 
+~~~~ruby
     google_books_engine.search("smith", :search_field => "inauthor")
-    
+~~~~
+
 Or, if the engine provides it, you can search by normalized semantic search
 field type names:
 
-    google_books_engine.search("smith", :semantic_earch_field => :title)
-    
-This will raise if an engine doesn't support that semantic search field. 
+~~~~ruby
+    google_books_engine.search("smith", :semantic_search_field => :title)
+~~~~
+
 You can find out what fields a particular engine supports.
 
+~~~~ruby
     google_books_engine.search_keys # => internal keys
     google_books_engine.semantic_search_keys 
+~~~~
+
+A helper method for generating an html select of search field options is
+available in `bento_field_hash_for`, check it out. 
 
 You can also provide all arguments in a single hash when it's convenient
 to do so:
 
+~~~~ruby
     google_books_engine.search(:query => "smith", :search_field => "inauthor")
+~~~~
+
+Search fields that are not recognized (semantic or internal) will normally
+be ignored, but set `:unrecognized_search_field => :raise` in configuration
+or search arg to get an ArgumentError instead. 
     
 ### Sorting
 
 An engine advertises what sort types it supports:
 
+~~~~ruby
     google_books_engine.sort_keys
-   
+~~~~
+
 An array of sort identifiers, where possible
-chosen from a standard list of semantics. (See list in config/i18n/en.yml,
-bento_search.sort_keys). 
+chosen from a standard list of semantics. (See list in `./config/i18n/en.yml`,
+`bento_search.sort_keys`). 
 
+~~~~ruby
     google_books_engine.search("my query", :sort => "date_desc")
-    
-For help creating your UI, you can use built-in helper method:
+~~~~
 
-    bento_sort_hash_for(engine)
-    #=> returns a Hash suitable as first argument for rails
-    # options_for_select helper, with sort options and labels from I18n. 
+For help creating your UI, you can use built-in helper method, perhaps with Rails helper
+options_for_select:
+
+~~~~ruby
+    <%= options_for_select( bento_sort_hash_for(engine), params[:sort] ) %>
+~~~~    
         
     
 ### Pagination
@@ -138,13 +197,17 @@ You can tell the search engine how many items you want per-page, and
 use _either_ `:start` (0-based item offset) or `:page` (1-based page
 offset) keys to paginate into the results. 
 
+~~~~ruby
     results = google_books_engine.search("my query", :per_page => 20, :start => 40)
     results = google_books_engine.search("my query", :per_page => 20, :page => 2) # means same as above
-    
+~~~~
+
 An engine instance advertises it's maximum per-page values. 
 
+~~~~ruby
     google_books_engine.max_per_page
-    
+~~~~
+
 bento_search fixes the default per_page at 10.     
     
 For help creating your UI, you can ask a BentoSearch::Results for
@@ -153,8 +216,10 @@ object which should be suitable for passing to [kaminari](https://github.com/ama
 `paginate`, or else have convenient methods for roll your own pagination UI. 
 Kaminari's paginate method:
 
+~~~~ruby
     <%= paginate results.pagination %> 
-    
+~~~~
+
 ### Concurrent searching
 
 If you're going to search 2 or more search engines at once, you'll want to execute
@@ -172,8 +237,8 @@ to help you do this easily. Say, in a controller:
     # constructor takes id's registered with BentoSearch.register_engine
     searcher = BentoSearch::MultiSearcher.new(:gbs, :scopus, :summon)
     
-    # Call 'start' with any parameters you would give to an_engine.search
-    searcher.start("my query", :semantic_search_field => :author, :sort => "title")
+    # Call 'search' with any parameters you would give to an_engine.search
+    searcher.search("my query", :semantic_search_field => :author, :sort => "title")
     
     # At this point, all searches are executing asynchronously in seperate threads.
     # To get the results, blocking until all complete:
@@ -189,8 +254,10 @@ in the main thread (like search a local store of some kind outside of
 bento_search)
 
 You will need to add the 'celluloid' gem to your app to use this feature, 
-BentoSearch doesn't automatically include the celluloid dependency right now
-(should it?). 
+BentoSearch doesn't automatically include the celluloid dependency. Note
+that Celluloid uses multi-threading in such a way that you might need
+to turn Rails config.cache_classes=true even in development.
+ 
 
 For more info, see BentoSearch::MultiSearcher. 
 
@@ -199,38 +266,16 @@ For more info, see BentoSearch::MultiSearcher.
 BentoSearch provides some basic support for initially displaying a placeholder
 progress spinner, and having Javascript call back to get the actual results. 
 
-* **Setup Pre-requisites** 
-    * In your `./config/routes.rb`, you need `BentoSearch::Routes.new(self).draw` in order
-      to route to the ajax loader. 
-    * In your asset pipeline, you must have `//= require 'bento_search/ajax_load` 
-      to get JS for ajax loading. (or require 'bento_search' to get all bento_search JS)
-* **Note** that this is not a panacea for a very slow search engine -- if the
-search results take 20 seconds to come in, when the AJAX call back happens,
-your Rails process _will_ be blocked from serving any other requests for that 20
-seconds. In fact, that makes this feature of very limited applicability in general,
-think carefully about what this will do for you. 
-* **Beware** that there are some authorization considerations if your search
-engine is not publically configurable, see BentoSearch::SearchController
-for more details. 
-
-You have have registered a configured engine globally, and given it the special
-`:allow_routable_results` key. 
-
-    BentoSearch.register_engine("gbs") do |conf|
-      conf.api_key = "x"
-      conf.allow_routable_results = true
-    end
-    
-Now you can use the `bento_search` helper method with the registered id
-and query, instead of with results as before, and with an option for
-ajax auto-load. 
-
-    <%= bento_search("gbs", :query => "my query", 
-                     :semantic_search_field => :title,
-                     :load => :ajax_auto) %>
-
+It's not a panacea for pathologically slow search results, and can be tricky
+for results that need access controls. But it can be useful
+in some situations, both for automatic on-page-load ajax loading, and triggered
+ajax loading. 
 
+See the [wiki page](https://github.com/jrochkind/bento_search/wiki/AJAX-results-loading)
+for more info. 
 
+                     
+                     
 ### Item Decorators, and Links
 
 You can configure Decorators, in the form of plain old ruby modules, to be
@@ -244,15 +289,27 @@ link resolver.
 BentoSearch::Items can have a main link associated with them (generally 
 hyperlinked from title), as well as a list of additional links. Most engines
 do not provide additional links by default, custom local Decorators would
-be used to add them.
+be used to add them. See wiki for more info on decorators, and BentoSearch::Link
+for fields. 
+     
+## OpenURL and metadata
 
-    BentoSearch.register_engine("something") do |conf|
-       conf.engine = SomeEngine
-       conf.item_decorators = [ SomeModule, OtherModule]
-    end
+Academic library uses often need openurl links from scholarly citations. One of
+the design goals of bento_search is to produce standardized normalized BentoSearch::ResultItem
+models, with sufficient semantics for translation to other formats. 
+
+See ResultItem#to_openurl_kev (string URL query encoding of OpenURL), and 
+ResultItem#to_openurl (a [ruby OpenURL gem](https://github.com/openurl/openurl) object). 
 
-See BentoSearch::Link for more info on links. (TODO: Better docs/examples
-on decorators). 
+Quality may vary, depending on how well the particular engine adapter captures semantics,
+especially the format/type of results (See bento_search's internal format/type vocabulary
+documented at ResultItem#format). As well as how well the #to_openurl routine
+handles all edge cases (OpenURL can be weird). As edge cases are discovered, they
+can be solved. 
+
+See `./app/item_decorators/bento_search/openurl_add_other_link.rb` for an example
+of using item decorators to add a link to your openurl resover to an item when
+displayed. 
 
 ## Planned Features
 
@@ -267,7 +324,6 @@ Probably:
   also may be supported by some engines that do not support facetting). 
 * Support for multi-field, multi-entry-box 'advanced search' UI's, in
   a normalized cross-engine way. 
-* More mindless support for displaying pagination UI with kaminari.
 
 Other needs or suggestions?
   
@@ -304,3 +360,4 @@ welcome. See more info on writing a BentoSearch::SearchEngine in the inline
 docs in that file. 
 
 
+

data/app/assets/javascripts/bento_search/ajax_load.js

@@ -4,7 +4,7 @@ var BentoSearch = BentoSearch || {}
 // Will AJAX load bento search results inside that node.
 // optional second arg success callback function. 
 BentoSearch.ajax_load = function(node, success_callback) {
-  div = $(node);
+  var div = $(node);
   
   if (div.length == 0) {
     //we've got nothing
@@ -22,11 +22,19 @@ BentoSearch.ajax_load = function(node, success_callback) {
   // Now load the actual external content from html5 data-bento-ajax-url
   $.ajax({
       url: div.data("bentoAjaxUrl"), 
-      success: function(response, status, xhr) {        
+      success: function(response, status, xhr) {
+        var do_replace = true;         
         if (success_callback) {
-           success_callback.apply(div, response);
+          // We need to make the response into a DOM so the callback
+          // can deal with it better. Wrapped in a div, so it makes
+          // jquery happy even if there isn't a single parent element. 
+          response = $("<div>" + response + "</div>");
+          
+          do_replace = success_callback.apply(div, [response]);                    
         }
-        div.replaceWith(response);          
+        if (do_replace != false) { 
+          div.replaceWith(response);
+        }          
       },
       error: function(xhr, status, errorThrown) {
         var msg = "Sorry but there was an error: ";

data/app/assets/stylesheets/bento_search/suggested_styles.css

@@ -3,10 +3,6 @@
    or just use this as documentation for suggestions to implement yourself. */
    
    
-/* year in bold in citations */
-.bento_item_row.published_in .year {
-  font-weight: bold;
-}
 
 /* highlighted element in title is already in a <b> tag, but
    title is likely to be bold already. italisize it too. */   
@@ -29,3 +25,7 @@
 .bento_item_title a .bento_search_highlight {
     font-style: inherit;
 }
+
+.bento_item_body .source_title {
+  font-style: italic;
+}

data/app/helpers/bento_search_helper.rb

@@ -1,3 +1,7 @@
+# encoding: UTF-8
+
+require 'nokogiri'
+
 # Rails helper module provided by BentoSearch meant to be included
 # in host app's helpers. 
 module BentoSearchHelper
@@ -25,6 +29,7 @@ module BentoSearchHelper
   #     bento_results("google_books", :query => "cancer", :load => :ajax_auto)
   #  
   def bento_search(search_arg, options = {})
+    
     results = search_arg if search_arg.kind_of? BentoSearch::Results
     
     load_mode = options.delete(:load) 
@@ -35,6 +40,7 @@ module BentoSearchHelper
       engine = (if search_arg.kind_of? BentoSearch::SearchEngine
         search_arg
       else
+        raise ArgumentError.new("Need Results, engine, or registered engine_id as first argument to #bento_search") unless search_arg
         BentoSearch.get_engine(search_arg.to_s)
       end)
       
@@ -66,16 +72,32 @@ module BentoSearchHelper
       results = engine.search(options) unless results
 
       if results.failed?
-        render :partial => "bento_search/search_error", :locals => {:results => results}
-      elsif results.length > 0      
-        render :partial => "bento_search/std_item", :collection => results, :as => :item
+        partial = (results.display_configuration.error_partial if results.display_configuration) || "bento_search/search_error"         
+        render :partial => partial, :locals => {:results => results}
+      elsif results.length > 0   
+        partial = (results.display_configuration.item_partial if results.display_configuration) || "bento_search/std_item"        
+        render :partial => partial, :collection => results, :as => :item, :locals => {:results => results}
       else
         content_tag(:div, :class=> "bento_search_no_results") do
-          I18n.translate("bento_search.no_results")
+          partial = (results.display_configuration.no_results_partial if results.display_configuration) || "bento_search/no_results"                   
+          render :partial => partial, :locals => {:results => results}
         end
       end
     end                          
   end
+  
+  # Wrap a ResultItem in a decorator! For now hard-coded to
+  # BentoSearch::StandardDecorator
+  def bento_decorate(result_item)
+    # What decorator class? If specified as string in #decorator,
+    # look it up as a class object, else default. 
+    decorator_class = result_item.decorator.try {|name| BentoSearch::Util.constantize(name) } || BentoSearch::StandardDecorator  
+    
+    # in a helper method, 'self' is a view_context already I think?
+    decorated = decorator_class.new(result_item, self)    
+    yield(decorated) if block_given?    
+    return decorated    
+  end
     
   
   ##
@@ -89,33 +111,53 @@ module BentoSearchHelper
   #
   ##
   
-  def bento_abstract_truncate(str)
-    # if it's html safe, we can't truncate it, we don't have an HTML-aware
-    # truncation routine right now, that avoids leaving tags open etc. 
-    return str if str.html_safe?
+  # Like rails truncate helper, and taking the same options, but html_safe.
+  # 
+  # If input string is NOT marked html_safe?, simply passes to rails truncate helper. 
+  # If a string IS marked html_safe?, uses nokogiri to parse it, and truncate 
+  # actual displayed text to max_length, while keeping html structure valid.
+  #
+  # Default omission marker is unicode elipsis
+  #
+  # :length option will also default to 280, what we think is a good
+  # length for abstract/snippet display
+  def bento_truncate(str, options = {})
+    options.reverse_merge!(:omission => "…", :length => 280, :separator => ' ')       
+    
+    # works for non-html of course, but for html a quick check
+    # to avoid expensive nokogiri parse if the whole string, even
+    # with tags, is still less than max length. 
+    return str if str.length < options[:length]
     
-    truncate(str, :length => 280, :separator => " ")    
+    if str.html_safe? 
+      noko = Nokogiri::HTML::DocumentFragment.parse(str)
+      BentoSearch::Util.nokogiri_truncate(noko, options[:length], options[:omission], options[:separator]).inner_html.html_safe
+    else
+      return truncate(str, options)
+    end
   end
+    
+
   
-  # Prepare a title in an H4, with formats in parens in a <small> (for
-  # bootstrap), linked, etc. 
+  # Deprecated, made more sense to put this in a partial. Just
+  # call partial directly:
+  #  <%= render :partial => "bento_search/item_title", :object => item, :as => 'item'  %>
   def bento_item_title(item)
-    content_tag("h4", :class => "bento_item_title") do
-      safe_join([
-        link_to_unless( item.link.blank?, item.complete_title, item.link ),
-        if item.format.present? || item.format_str.present?
-          content_tag("small", :class => "bento_item_about") do
-            " (" +
-              if item.format_str
-                item.format_str
-              else
-                t(item.format, :scope => [:bento_search, :format], :default => item.format.to_s.titleize)
-              end + ")"
-          end
-        end
-      ], '')
-    end
+    render :partial => "bento_search/item_title", :object => item, :as => 'item'
   end
+  deprecate :bento_item_title
+  
+  # pass in 0-based rails current collection counter and a BentoSearch::Results,
+  # calculates a user-displayable result set index label. 
+  #
+  # Only non-trivial thing is both inputs are allowed to be nil; if either
+  # is nil, nil is returned.  
+  def bento_item_counter(counter, results)
+    return nil if counter.nil? || results.nil? || results.start.nil?
+    
+    return counter + results.start + 1
+  end
+    
   
   # first 3 authors, each in a <span>, using item.author_display, seperated by
   # semi-colons. 
@@ -125,7 +167,7 @@ module BentoSearchHelper
     first_three = item.authors.slice(0,3) 
         
     first_three.each_with_index do |author, index|
-      parts << content_tag("span", :class => "bento_item_author") do
+      parts << content_tag("span", :class => "authors") do
           item.author_display(author)
       end
       if (index + 1) < first_three.length
@@ -133,6 +175,10 @@ module BentoSearchHelper
       end      
     end
     
+    if item.authors.length > 3
+      parts << I18n.t("bento_search.authors_et_al")
+    end
+    
     return safe_join(parts, "")
   end
   
@@ -150,5 +196,46 @@ module BentoSearchHelper
     ]    
   end
   
+  # Returns a hash of label => key suitable for passing to rails
+  # options_for_select. ONLY includes fields with :semantics set at
+  # present. Key will be _semantic_ key name. 
+  # For engine-specific fields, you're on you're own, sorry!
+  #
+  # If first arg is an engine instance, will
+  # be search fields supported by that engine. If first arg is nil,
+  # will be any field in our i18n lists for search fields. 
+  #
+  # Can pass in options :only or :except to customize list. Values
+  # in :only and :except will match on internal field names OR
+  # semantic field names (hopefully this convenience ambiguity
+  # won't cause problems)
+  #
+  # eg:
+  #     <%= select_tag 'search_field', options_for_select(bento_field_hash_for(@engine), params[:search_field]) %>
+  def bento_field_hash_for(engine, options = {})
+    if engine.nil?
+      hash = I18n.t("bento_search.search_fields").invert
+    else
+      hash = Hash[ engine.search_field_definitions.collect do |k, defn|
+        if defn[:semantic] && (label = I18n.t(defn[:semantic], :scope => "bento_search.search_fields", :default => defn[:semantic].to_s.titlecase )) 
+          [label,  defn[:semantic].to_s]
+        end
+      end]
+    end
+  
+    # :only/:except    
+    if options[:only]      
+      keys = [options[:only]].flatten.collect(&:to_s)
+      hash.delete_if {|key, value|  ! keys.include?(value) }  
+    end
+    
+    if options[:except]
+      keys = [options[:except]].flatten.collect(&:to_s)
+      hash.delete_if {|key, value|  keys.include?(value) }
+    end
+    
+    return hash
+  end
+  
   
 end