bento_search 0.0.1

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright 2012 Jonathan Rochkind/Johns Hopkins Universities
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,299 @@
+# BentoSearch
+
+[![Build Status](https://secure.travis-ci.org/jrochkind/bento_search.png)](http://travis-ci.org/jrochkind/bento_search)
+
+
+(**in progress*, not yet ready for use, mainly because we need more
+out of the box search engines supported). 
+
+bento_search provides an abstraction/normalization layer for querying and 
+displaying results for external search engines, in Ruby on Rails. Requires
+Rails3 and tested only under ruby 1.9.3. 
+
+* It is focused on use cases for academic libraries, but may be useful in generic
+cases too. Initially, engine adapters are planned to be provided for: 
+Google Books, Scopus, SerialSolutions Summon, Ex Libris Primo, 
+EBSCO Discovery Service, and EBSCO traditional 'EIT' api. Most
+of these search engines require a vendor license to use. 
+
+* bento_search could be considered building blocks for a type of 'federated
+search' functionality, but it does not and will never support merging results
+from multiple engines into one result set. It is meant to support displaying the
+first few results from multiple engines on one page, "bento box" style (as
+named by Tito Sierra@NCSU), as well as more expanded single-search-on-a-page
+uses. 
+
+* bento_search provides abstract functionality for pagination, sorting, 
+and single-field-specified queries. Faceting, limiting, and 'advanced'
+multi-field searches are not yet supported, but planned. Not all
+search engine adapters support all features.  Search engine adapters can
+declare search fields and sort options with 'semantics', so you can for
+instance search or sort by 'title' across search engines without regard
+to internal engine-specific field names. 
+
+bento_search is designed to allow code to be written agnostic of the search
+provider, so you can switch out the search provider, minimizing dependent
+code in your app that needs to be rewritten. As well as letting you get
+started quick without reinventing the wheel and figuring out poorly
+documented vendor API's yourself. 
+
+
+## Usage
+
+### Instantiate an engine, and search
+
+When you instantiate an engine, you can provide configuration keys. There
+are a few standard keys (see BentoSearch::SearchEngine), and others that
+may be engine-specific. Some engine-specific keys (such as api auth keys) 
+may be required for certain engines. 
+
+    engine = BentoSearch::GoogleBooksEngine.new(:api_key => "my_gbs_api_key")
+    results = engine.search("a query")
+    
+`results` are a BentoSearch::Results object, which acts like an array of
+BentoSearch::Item objects, along with some meta-information about the
+search itself (pagination keys, etc).  BentoSearch::Results and Item fields
+are standardized accross engines. BentoSearch::Items provide semantic
+values (title, author, etc.), as available from the particular engine. 
+
+To see which engines come bundled with BentoSearch, and any special 
+engine-specific instructions, look at BentoSearch source in `./app/search_engines`
+
+### Register engines in global configuration
+
+It can be convenient to register an engine in global configuration, and is 
+required for certain functionality (like out-of-the-box AJAX loading). 
+
+In an initializer in your app, like say `./config/initializers/bento_search.rb`:
+
+    BentoSearch.register_engine("gbs") do |conf|
+       conf.engine = "BentoSearch::GoogleBooksEngine"
+       conf.api_key = "my_google_api_key"
+       # any other configuration
+    end
+    
+Then you can refer to it, for instance in a controller, by the id you registered:
+
+    @results = BentoSearch.get_engine("gbs").search("my query")
+    
+### Display results
+
+You can of course write your own code to display a BentoSearch::Results object
+however you like. But BentoSearch comes with a helper method for displaying
+a list of BentoSearch::Results in a standard way, using the bento_search
+helper method. 
+
+    <%= bento_search(@results) %>
+    
+### Fielded searching.
+
+You can search by an internal engine-specific field name:
+
+    google_books_engine.search("smith", :search_field => "inauthor")
+    
+Or, if the engine provides it, you can search by normalized semantic search
+field type names:
+
+    google_books_engine.search("smith", :semantic_earch_field => :title)
+    
+This will raise if an engine doesn't support that semantic search field. 
+You can find out what fields a particular engine supports.
+
+    BentoSearch::GoogleBooksEngine.search_keys # => internal keys
+    BentoSearch::GoogleBooksEngine.semantic_search_keys 
+
+You can also provide all arguments in a single hash when it's convenient
+to do so:
+
+    google_books_engine.search(:query => "smith", :search_field => "inauthor")
+    
+### Sorting
+
+An engine advertises what sort types it supports:
+
+    BentoSearch::GoogleBooksEngine.sort_definitions
+   
+That returns a hash, where the keys are sort identifiers, where possible
+chosen from a standard list of semantics. (See list in config/i18n/en.yml,
+bento_search.sort_keys). 
+
+    google_books_engine.search("my query", :sort => "date_desc")
+    
+For help creating your UI, you can use built-in helper method:
+
+    bento_sort_hash_for(engine)
+    #=> returns a Hash suitable as first argument for rails
+    # options_for_select helper, with sort options and labels from I18n. 
+        
+    
+### Pagination
+
+You can tell the search engine how many items you want per-page, and 
+use _either_ `:start` (0-based item offset) or `:page` (1-based page
+offset) keys to paginate into the results. 
+
+    results = google_books_engine.search("my query", :per_page => 20, :start => 40)
+    results = google_books_engine.search("my query", :per_page => 20, :page => 2) # means same as above
+    
+An engine instance advertises it's maximum per-page values. 
+
+    google_books_engine.max_per_page
+    
+bento_search fixes the default per_page at 10.     
+    
+For help creating your UI, you can ask a BentoSearch::Results for
+`results.pagination`, which returns a BentoSearch::Results::Pagination
+object which should be suitable for passing to [kaminari](https://github.com/amatsuda/kaminari)
+`paginate`, or else have convenient methods for roll your own pagination UI. 
+Kaminari's paginate method:
+
+    <%= paginate results.pagination %> 
+    
+### Concurrent searching
+
+If you're going to search 2 or more search engines at once, you'll want to execute
+those searches concurrently. For instance, if GoogleBooks results take 2 second
+to come in, and Scopus results take 3 seconds -- you don't want to first wait
+the 2 second then wait the 3 seconds for a total of 5 -- you instead want
+to execute concurrently in seperate threads, so the total wait time is the slowest
+engine, not the sum of the engines. 
+
+You can write your own logic using ruby threads to do this, but 
+BentoSearch provides a multi-searching helper using [Celluloid](https://github.com/celluloid/celluloid)
+to help you do this easily. Say, in a controller:
+
+~~~~ruby
+    # constructor takes id's registered with BentoSearch.register_engine
+    searcher = BentoSearch::MultiSearcher.new(:gbs, :scopus, :summon)
+    
+    # Call 'start' with any parameters you would give to an_engine.search
+    searcher.start("my query", :semantic_search_field => :author, :sort => "title")
+    
+    # At this point, all searches are executing asynchronously in seperate threads.
+    # To get the results, blocking until all complete:
+    @results = searcher.results
+    
+    # @results will be a hash, keyed by registered engine id, values
+    # are BentoSearch::Results
+~~~~
+
+Even if you are only searching one engine, this may be useful to have the
+search execute in a seperate thread, so you can continue doing other work
+in the main thread (like search a local store of some kind outside of
+bento_search)
+
+You will need to add the 'celluloid' gem to your app to use this feature, 
+BentoSearch doesn't automatically include the celluloid dependency right now
+(should it?). 
+
+For more info, see BentoSearch::MultiSearcher. 
+
+### Delayed results loading via AJAX (actually more like AJAHtml)
+
+BentoSearch provides some basic support for initially displaying a placeholder
+progress spinner, and having Javascript call back to get the actual results. 
+
+* **Setup Pre-requisites** 
+    * In your `./config/routes.rb`, you need `BentoSearch::Routes.new(self).draw` in order
+      to route to the ajax loader. 
+    * In your asset pipeline, you must have `//= require 'bento_search/ajax_load` 
+      to get JS for ajax loading. (or require 'bento_search' to get all bento_search JS)
+* **Note** that this is not a panacea for a very slow search engine -- if the
+search results take 20 seconds to come in, when the AJAX call back happens,
+your Rails process _will_ be blocked from serving any other requests for that 20
+seconds. In fact, that makes this feature of very limited applicability in general,
+think carefully about what this will do for you. 
+* **Beware** that there are some authorization considerations if your search
+engine is not publically configurable, see BentoSearch::SearchController
+for more details. 
+
+You have have registered a configured engine globally, and given it the special
+`:allow_routable_results` key. 
+
+    BentoSearch.register_engine("gbs") do |conf|
+      conf.api_key = "x"
+      conf.allow_routable_results = true
+    end
+    
+Now you can use the `bento_search` helper method with the registered id
+and query, instead of with results as before, and with an option for
+ajax auto-load. 
+
+    <%= bento_search("gbs", :query => "my query", 
+                     :semantic_search_field => :title,
+                     :load => :ajax_auto) %>
+
+
+
+### Item Decorators, and Links
+
+You can configure Decorators, in the form of plain old ruby modules, to be
+applied to BentoSearch::Items, on an engine-by-engine basis. These can modify,
+add, or remove Item data, as well as over-ride some presentational methods.  
+
+One common use for these Decorators is changing, adding, or removing links
+associated with an item. For instance, to link to your local OpenURL 
+link resolver.
+
+BentoSearch::Items can have a main link associated with them (generally 
+hyperlinked from title), as well as a list of additional links. Most engines
+do not provide additional links by default, custom local Decorators would
+be used to add them.
+
+    BentoSearch.register_engine("something") do |conf|
+       conf.engine = SomeEngine
+       conf.item_decorators = [ SomeModule, OtherModule]
+    end
+
+See BentoSearch::Item for more information on decorators, and BentoSearch::Link
+on links. 
+
+## Planned Features
+
+I am trying to keep BentoSearch as simple as it can be to conveniently meet
+actual use cases.  Trying to avoid premature over-engineering, and pave
+the cowpaths as needed. 
+
+Probably:
+
+* Support for display facets for engines that support such, as well as 
+  search with limits from controlled vocabulary (ie, selected facet, but
+  also may be supported by some engines that do not support facetting). 
+* Support for multi-field, multi-entry-box 'advanced search' UI's, in
+  a normalized cross-engine way. 
+* More mindless support for displaying pagination UI with kaminari.
+
+Other needs or suggestions?
+  
+
+## Developing
+
+BentoSearch is fairly well covered by automated tests. We simply use Test::Unit.
+Run tests with `rake test`. 
+
+The testing environment was generated with `rails plugin new`, and includes
+a dummy app used when testing at `./test/dummy`. 
+
+For integration tests against live external search API's, we use the awesome
+[VCR](https://github.com/myronmarston/vcr) gem to cache responses. 
+To write your own Test::Unit tests using VCR, take note of the 
+`test_with_cassette` method provided in `./test/support/test_with_cassette.rb`. 
+
+Also note use of VCR.filter_sensitive_data to make sure your API keys
+do not get saved in cached response in the repo, while still allowing
+tests to be run against cached responses even for engines that require
+auth. 
+
+To re-generate cached responses, delete the relevant files in 
+`./test/vcr_cassettes` and re-run tests. You may have to set an ENV
+variable with your own API keys to re-run tests without cached response
+like this. 
+
+Also note `./test/support/mock_engine.rb`, a simple mock/dummy SearchEngine
+implementation that can be used in other tests. 
+
+Pull requests welcome.  Pull requests with additional search engine implementations
+welcome. See more info on writing a BentoSearch::SearchEngine in the inline
+docs in that file. 
+
+

data/Rakefile

@@ -0,0 +1,40 @@
+#!/usr/bin/env rake
+begin
+  require 'bundler/setup'
+rescue LoadError
+  puts 'You must `gem install bundler` and `bundle install` to run rake tasks'
+end
+begin
+  require 'rdoc/task'
+rescue LoadError
+  require 'rdoc/rdoc'
+  require 'rake/rdoctask'
+  RDoc::Task = Rake::RDocTask
+end
+
+RDoc::Task.new(:rdoc) do |rdoc|
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.title    = 'BentoSearch'
+  rdoc.options << '--line-numbers'
+  rdoc.rdoc_files.include('README.rdoc')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+end
+
+APP_RAKEFILE = File.expand_path("../test/dummy/Rakefile", __FILE__)
+load 'rails/tasks/engine.rake'
+
+
+
+Bundler::GemHelper.install_tasks
+
+require 'rake/testtask'
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << 'lib'
+  t.libs << 'test'
+  t.pattern = 'test/**/*_test.rb'
+  t.verbose = false
+end
+
+
+task :default => :test

data/app/assets/javascripts/bento_search.js

@@ -0,0 +1,3 @@
+// require all bento_search javascript. 
+//
+//= require_tree ./bento_search

data/app/assets/javascripts/bento_search/ajax_load.js

@@ -0,0 +1,22 @@
+jQuery(document).ready(function($) {
+    //Intentionally wait for window.load, not just onready, to
+    //prevent interfering with rest of page load. 
+    $(window).bind("load", function() {   
+      $(".bento_search_ajax_wait").each(function(i, div) {
+         div = $(div);
+         // from html5 data-bento-ajax-url
+         $.ajax({
+           url: div.data("bentoAjaxUrl"), 
+           success: function(response, status, xhr) {
+            div.replaceWith(response);   
+           },
+           error: function(xhr, status, errorThrown) {
+             var msg = "Sorry but there was an error: ";
+             div.html(msg + xhr.status + " " + xhr.statusText + ", " + status);
+           }
+         });
+                              
+      });
+    });
+    
+});

data/app/assets/stylesheets/bento_search/bento.css

@@ -0,0 +1,4 @@
+/*
+  Place all the styles related to the matching controller here.
+  They will automatically be included in application.css.
+*/

data/app/controllers/bento_search/bento_search_controller.rb

@@ -0,0 +1,7 @@
+module BentoSearch
+  # Master superclass for all controllers included in
+  # bento_search, note it inherits from host app's 
+  # ApplicationController
+  class BentoSearchController < ::ApplicationController
+  end
+end

data/app/controllers/bento_search/search_controller.rb

@@ -0,0 +1,72 @@
+module BentoSearch
+  # This is a controller that provides stand-alone search results
+  # for registered engines. Right now, this is only for automatic
+  # AJAX delayed loading. In the future it may be used for atom results,
+  # or other such. 
+  #
+  # You need to make sure to include routing for this controller in your
+  # app to use it, for instance with `BentoSearch::Routes.new(self).draw`
+  # in your ./config/routes.rb
+  #
+  # # Authorization Issues
+  # 
+  # You may have some engines which should not be publically searchable,
+  # they should only be searchable by certain auth'd users. This controller
+  # could accidentally provide a non-protected endpoint to get results if
+  # nothing were done to prevent it. 
+  #
+  # Only engines which have a :allow_routable_results => true key
+  # in their config will be served by this controller. 
+  #
+  # If you need routable results on an engine which ALSO needs to 
+  # be protected by auth, you can add your own Rails before_filter
+  # to provide auth. Say, in an initializer in your app:
+  #
+  #     SearchController.before_filter do |controller|
+  #       unless controller.current_user
+  #          raise BentoSearch::SearchController::AccessDenied
+  #       end
+  #     end
+  #    
+  # 
+  # We may provide fancier/nicer API for this in the future, if there's
+  # demand. 
+  class SearchController < BentoSearchController
+    class AccessDenied < Exception ; end
+
+    
+    rescue_from AccessDenied, :with => :deny_access
+    rescue_from NoSuchEngine, :with => :render_404
+      
+    # returns partial HTML results, suitable for
+    # AJAX to insert into DOM. 
+    # arguments for engine.search are taken from URI request params.
+    # (TODO: Is this a security issue, do we need to whitelist em? )
+    def search                  
+      engine = BentoSearch.get_engine(params[:engine_id])
+
+      
+      unless engine.configuration.allow_routable_results == true
+        raise AccessDenied.new("engine needs to be registered with :allow_routable_results => true")
+      end
+
+      @results = engine.search(params.to_hash.symbolize_keys)
+      
+      render :layout => false # partial HTML results     
+    end
+    
+
+    
+    protected 
+    
+    def deny_access(exception)
+      render :text => exception.message, :status => 403
+    end
+    
+    def render_404(exception)      
+      render :text => exception.message, :status => 404
+    end
+    
+    
+  end
+end