umlaut 3.0.0alpha1

data/app/models/service_response.rb

@@ -0,0 +1,179 @@
+=begin rdoc
+
+A ServiceResponse is a piece of data generated by a Service. It usually will
+be displayed on the resolve menu.
+
+ServiceResponses have a service type, represented by a string. When displaying, ServiceResponses are typically grouped into lists by service type. ServiceResponses are tied to the Service that created them, with the #service accessor.
+
+ServiceResponses have a few basic attributes stored in columns in the db: 'display_text' is the text to put in the hyperlink. 'notes' is available for longer explanatory text (\n in notes will be converted to <br> by view). 'url' can be used to store the url to link to (but see below on linking mechanism). 
+
+[The legacy columns response_key, value_string, value_alt_string and value_text are deprecated and should not be used, but some legacy Services still use them, so they're still there for now].
+
+In addition, there's a Hash (automatically serialized by ActiveRecord) that's stored in service_data, for arbitrary additional data that a Service can store--whatever you want, just put it in. However, there are conventions that Views expect, see below.  You can access ALL the arbitrary key/values in a ServiceResponse, built-in in attributes or from the serialized Hash, by the proxy object returned from #data_values.
+
+ServiceResponse is connected to a Request via the ServiceType join table. The data architecture allows a ServiceResponse to be tied to multiple requests, perhaps to support some kind of cacheing re-use in the future. But at present, the code doesn't do this, a ServiceResponse will really only be related to one request. However, a ServiceResponse can be related to a single Request more than once--once per each type of service response. ServiceType is really a three way join, representing a ServiceResponse, attached to a particular Request, with a particular ServiceTypeValue.  
+
+
+== View Display of ServiceResponse
+
+The resolve menu View expects a Hash (or Hash-like) object with certain conventional keys, to display a particular ServiceResponse. You can provide code in your Service to translate a ServiceResponse to a Hash. But you often don't need to, you can use the proxy object returned by #data_values instead, which provides hash-like access to all arbitrary key/values stored in ServiceResponse. If the Service stores properties in there using conventional keys (see below), no further translation is needed.
+
+However, if you need to do further translation you can implement methods on the Service, of the form: "to_[service type string](response)", for instance "to_fulltext". Umlaut will give it a ServiceResponse object, method should return a hash (or hash-like obj).  Service can also implement a method response_to_view_data(response), as a 'default' translation. This mechanism of various possible 'translations' is implemented by Service#view_data_from_service_type.
+
+== Url generation
+
+At the point the user clicks on a ServiceResponse, Umlaut will attempt to find a url for the ServiceResponse, by calling response_url(response) on the relevant Service. The default implementation in service.rb just returns service_response['url'], so the easiest way to do this is just to put the url in service_response['url'].  However, your Service can over-ride this method to provide it's own implementation to generate to generate the url on demand in any way it wants.  If it does this, technically service_response['url'] doesn't need to include anything. But if you have a URL, you may still want to put it there, for Umlaut to use in guessing something about the destination, for de-duplication and possibly other future purposes.  
+
+= Conventional keys:
+
+ Absolute minimum: 
+ [:display_text]   Text that will be used 
+
+ Basic set (used by fulltext and often others)
+ [:display_text]
+ [:notes]          (newlines converted to <br>)
+ [:coverage]
+ [:authentication]
+ [:match_reliability] => One of MatchExact or MatchUnsure (maybe more later), for whether there's a chance this is an alternate Edition or the wrong work entirely. These are fuzzy of neccisity -- if it MIGHT be an alt edition, use MatchAltEdition even if you can't be sure it's NOT an exact match. 
+ :edition_str => String statement of edition or work to let the user disambiguate and see if it's what they want. Can be taken for instance from Marc 260. Generally only displayed when match_reliabilty is not MatchExact. If no value, Umlaut treats as MatchExact.
+
+== Full text specific
+These are applicable only when the incoming OpenURL is an article-level citation. Umlaut uses Request#title_level_citation? to estimate this.
+
+  [:coverage_checked]  boolean, default true.  False for links from, eg, the catalog, where we weren't able to pre-check if the particular citation is included at this link.
+  [:can_link_to_article] boolean, default true. False if the links is _known_ not to deliver user to actual article requested, but just to a title-level page. Even though SFX links sometimes incorrectly do this, they are still not set to false here.  
+ 
+== highlighted_link (see also)
+ [:source]   (optional, otherwise service's display_name is used)
+
+== Holdings set adds:
+ [:source_name]
+ [:call_number]
+ [:status]
+ [:request_url]     a url to request the item. optional. 
+ [:coverage_array] (Array of coverage strings.)
+ [:due_date]
+ [:collection_str]
+ [:location_str]
+
+== search_inside
+ Has no additional conventional keys, but when calling it's url handling functionality, send it a url param query= with the users query. In the API, this means using the umlaut_passthrough_url, but adding a url parameter query on to it. This will redirect to the search results. 
+
+== Cover images:
+ [:display_text] set to desired alt text
+ [:url]    src url to img
+ [:size]  => 'small', 'medium', 'large' or 'extra-large'. Also set in :key
+
+== Anything from amazon:
+ [:asin]
+
+== Abstracts/Tocs:
+   Can be a link to, or actual content. Either way, should be set
+   up to link to source of content if possible. Basic set, plus:
+   [:content]           actual content, if available. 
+
+=end
+class ServiceResponse < ActiveRecord::Base  
+  @@built_in_fields = [:display_text, :url, :notes, :response_key, :value_string, :value_alt_string, :value_text]
+  belongs_to :request
+  serialize :service_data  
+  # This value is not stored in db, but is set temporarily so
+  # the http request params can easily be passed around with a response
+  # object.
+  attr_accessor :http_request_params
+
+  # Constants for 'match_reliability' value.
+  MatchExact = 'exact'
+  MatchUnsure = 'unsure'
+  #MatchAltEdition = 'edition'
+  #MatchAltWork = 'work'
+
+  def initialize(params = nil)
+    super(params)
+    self.service_data = {} unless self.service_data
+  end
+
+  # Instantiates and returns a new Service associated with this response.
+  def service
+    @service ||= ServiceStore.instantiate_service!( self.service_id, nil )
+  end
+  
+  # Returns a hash or hash-like object with properties for the service response. 
+  def view_data
+    self.service.view_data_from_service_type(self)
+  end
+  
+    # Should take a ServiceTypeValue object, or symbol name of
+  # ServiceTypeValue object. 
+  def service_type_value=(value)
+    value = ServiceTypeValue[value] unless value.kind_of?(ServiceTypeValue)        
+    self.service_type_value_name = value.name   
+  end
+  def service_type_value
+    ServiceTypeValue[self.service_type_value_name]
+  end
+  
+  
+
+  def take_key_values(hash)    
+    # copy it, cause we're gonna modify it
+    hash = hash.clone
+    hash.each_pair do |key, value|
+      if ( self.class.built_in_fields.include?(key))
+        self.send(key.to_s + '=', value)
+        hash.delete(key)
+      end
+    end
+    # What's left is arbitrary key/values that go in service_data
+    init_service_data(hash)
+  end
+
+  def init_service_data(hash)
+    hash.each {|key, value| data_values[key] = value} if hash
+  end
+
+  def data_values    
+    # Lazy load, and store a reference. Don't worry, ruby
+    # GC handles circular references no problem. 
+    unless (@data_values_proxy)  
+      @data_values_proxy = ServiceResponseDataValues.new(self)
+    end
+    return @data_values_proxy;
+  end
+
+  def self.built_in_fields
+    @@built_in_fields
+  end
+  
+
+  
+end
+
+# A proxy-like class, to provide hash-like access to all arbitrary
+# key/value pairs stored in a ServiceResponse, whether they key/value
+# is stored in an ActiveRecord attribute (#built_in_fields) or in
+# the serialized hash in the service_data attribute. 
+# Symbols passed in will be 'normalized' to strings before being used as keys.
+# So symbols and strings are interchangeable. Normally, keys should be symbols.
+class ServiceResponseDataValues
+  def initialize(arg_service_response)
+    @service_response = arg_service_response
+  end
+
+  def [](key)        
+    if ServiceResponse.built_in_fields.include?(key)
+      return @service_response.send(key)
+    else
+      return @service_response.service_data[key]
+    end
+  end
+
+  def []=(key, value)
+    if(ServiceResponse.built_in_fields.include?(key))
+      @service_response.send(key.to_s+'=', value)
+    else
+      @service_response.service_data[key] = value
+    end
+  end
+  
+end

data/app/models/service_store.rb

@@ -0,0 +1,59 @@
+# Loads Service definitions from Rails.root/config/umlaut_services.yml
+# instantiates services from definitions, by id.
+#
+# It's terrible we need to do this globally like this, but
+# too hard to back out of legacy design now. 
+class ServiceStore
+  
+  # Returns complete hash loaded from services.yml
+  def self.config
+    # cache hash loaded from YAML, ensure it has the keys we expect. 
+    unless defined? @@services_config_list
+      yaml_path = File.expand_path("config/umlaut_services.yml", Rails.root)
+      if File.exists? yaml_path
+        @@services_config_list = YAML::load(File.open( yaml_path ))        
+      else
+        @@services_config_list = {}
+      end
+      @@services_config_list["default"] ||= {}
+      @@services_config_list["default"]["services"] ||= {}
+    end
+    return @@services_config_list
+  end
+  
+  # Returns hash keyed by unique service name, value service definition
+  # hash. 
+  def self.service_definitions
+    unless defined? @@service_definitions
+      @@service_definitions = {}
+      config.each_pair do |group_name, group|
+        @@service_definitions.merge!(  group["services"]  ) if group["services"]
+      end
+    end
+    return @@service_definitions
+  end
+  
+  def self.service_definition_for(service_id)
+    return service_definitions[service_id]
+  end
+  
+  # pass in string unique key OR a service definition hash, 
+  # and a current UmlautRequest.
+  # get back instantiated Service object. 
+  def self.instantiate_service!(service, request)
+    definition = service.kind_of?(Hash) ? service : service_definition_for(service.to_s)
+        
+    if definition.nil?
+      raise "Service '#{service}'' does not exist in umlaut-services.yml"
+    end
+    
+    className = definition["type"] || definition["service_id"]
+    classConst = Kernel.const_get(className)
+    service = classConst.new(definition)
+    service.request = request
+
+    return service    
+  end
+  
+  
+end

data/app/models/service_type_value.rb

@@ -0,0 +1,58 @@
+require 'yaml'
+
+
+# This model is the actual list of valid service types. Not ActiveRecord,
+# just load from config files into memory (loaded on file load). 
+#
+# ServiceTypeValue's also have displable strings, stored in the
+# display_name attribute, and possibly custom display_name_plural
+class ServiceTypeValue   
+  attr_accessor :name, :id, :display_name, :display_name_plural
+  
+  class << self; attr_accessor :values; end  
+  
+  def initialize(hash)
+    hash.each_pair do |key, value|
+      self.send(key.to_s+"=", value)
+    end
+  end
+  
+  def self.find(name)
+    load_values! if values.nil?
+    values[name.to_sym] or raise ArgumentError.new("No ServiceTypeValue found for #{name}")
+  end
+  def self.[](name)
+    find(name)
+  end
+  
+  def display_name_pluralize
+    return self.display_name_plural || self.display_name.pluralize
+  end
+  
+  @@distro_conf_file = File.join(Umlaut::Engine.root, "db", "orig_fixed_data", "service_type_values.yml")
+  @@local_conf_file = File.join(Rails.root, "config", "umlaut_service_type_values.yml")
+  
+  
+  # Loads from config files, distro and local, into memory.   
+  def self.load_values!
+      # Load in starting set of ServiceTypeValue, merge in local defines. 
+      
+      service_type_values = YAML.load_file( @@distro_conf_file )
+      local_overrides = File.exists?( @@local_conf_file ) ?
+            YAML.load_file(@@local_conf_file) :
+            nil
+      # Merge in the params for each service type with possible
+      # existing params.
+      if ( local_overrides )
+        local_overrides.each do |name, params|
+          service_type_values[name] ||= {}          
+          service_type_values[name].merge!( params )          
+        end
+      end
+      
+      self.values = {}
+      service_type_values.each_pair do |name, hash|
+        self.values[name.to_sym] = ServiceTypeValue.new(hash.merge(:name => name))                              
+      end            
+  end
+end

data/app/models/service_wave.rb

@@ -0,0 +1,150 @@
+# ServiceWave is basically responsible for multi-threaded execution of
+# a list of services, all in the same priority. Generally it's only
+# called by Collection, nobody else needs it directly. 
+#
+class ServiceWave
+  attr_accessor :services
+  attr_accessor :priority_level
+
+  # Priority level is purely information, used for debug output. 
+  def initialize(service_objects, priority_level = nil, config = UmlautController.umlaut_config)
+    @services = service_objects
+    @priority_level = priority_level
+
+    @log_timing = config.lookup!("log_service_timing", true)
+
+    # Don't forward exceptions, that'll interrupt other service processing.
+    # Catch the exception, record it in the dispatch table, done. May want
+    # to set this to true for debugging/development, but NOT for production.
+    @forward_exceptions = false
+  end
+  
+  # Safe to call in a thread. Returns true or false depending on
+  # whether dispatch should proceed. 
+  def prepare_dispatch!(request, service, session_id)
+    can_dispatch = false
+    
+    ActiveRecord::Base.connection_pool.with_connection do
+      if request.can_dispatch?( service)
+        # Mark this service as in progress in the dispatch table.
+        request.dispatched( service, DispatchedService::InProgress )
+        # remember the rails session id. 
+        service.session_id = session_id
+        
+        can_dispatch = true      
+      end
+    end
+    return can_dispatch
+  end
+
+  def handle(request, session_id)
+    return if (@services.nil? || @services.empty?)
+
+    bundle_start = Time.now
+    Rails.logger.info("Launching servicelevel #{@priority_level}, request #{request.id}") if @log_timing
+
+    
+    threads = []
+    some_service_executed = false
+    @services.each do | service |
+      some_service_executed = true
+      local_request = nil
+        
+      threads << Thread.new(request.id, service.clone) do | request_id, local_service |
+        # Deal with ruby's brain dead thread scheduling by setting
+        # bg threads to a lower priority so they don't interfere with fg
+        # threads.
+        Thread.current.priority = -1
+
+        # Save some things in thread local hash useful for debugging
+        Thread.current[:debug_name] = local_service.class.name
+        Thread.current[:service] = service
+
+        # Tell our AR extension not to allow implicit checkouts
+        ActiveRecord::Base.forbid_implicit_checkout_for_thread! if ActiveRecord::Base.respond_to?("forbid_implicit_checkout_for_thread!")
+        begin
+          service_start = Time.now
+          
+          local_request = ActiveRecord::Base.connection_pool.with_connection do
+            # pre-load all relationships so no ActiveRecord activity will be
+            # needed later to see em. 
+            Request.includes(:referent, :service_responses, :dispatched_services).find(request_id)
+          end
+          
+
+          if prepare_dispatch!(local_request, local_service, session_id)                          
+            local_service.handle_wrapper(local_request)
+          else
+            Rails.logger.info("NOT launching service #{local_service.service_id},  level #{@priority_level}, request #{local_request.id}: not in runnable state") if @log_timing
+          end
+          
+         
+        rescue Exception => e
+          # We may not be able to access ActiveRecord because it may
+          # have been an AR connection error, perhaps out of connections
+          # in the pool. So log and record in non-AR ways. 
+          # the code waiting on our thread will see exception
+          # reported in Thread local var, and log it AR if possible. 
+          
+          
+          # Log it too, although experience shows it may never make it to the 
+          # log for mysterious reasons. 
+          Rails.logger.error("Threaded service raised exception. Service: #{service.service_id}, #{e}\n  #{clean_backtrace(e).join("\n  ")}")
+          
+          # And stick it in a thread variable too
+          Thread.current[:exception] = e                      
+        ensure
+          Rails.logger.info("Completed service #{local_service.service_id},  level #{@priority_level}, request #{local_request && local_request.id}: in #{Time.now - service_start}.") if @log_timing
+        end
+      end                    
+    end
+
+    # Wait for all the threads to complete, if any. 
+    threads.each do |aThread|
+      aThread.join
+      #aThread.kill # shouldn't be neccesary, but I'm paranoid
+      
+     
+      
+      if aThread[:exception] 
+        debugger
+        # mark it finished in the db, we should have a connection already, we're
+        # in main request thread. 
+        begin
+          request.dispatched(aThread[:service], DispatchedService::FailedFatal, aThread[:exception])
+        rescue Exception => e
+          debugger
+          raise e
+        end
+      end
+      
+      # Okay, raise if exception, if desired.
+      if ( aThread[:exception] && self.forward_exceptions? )        
+        raise aThread[:exception]
+      end
+    end
+    
+    threads.clear # more paranoia
+    
+    # Clean up any leftover connections left open by threads.
+    #ActiveRecord::Base.clear_active_connections!
+    ActiveRecord::Base.connection_pool.clear_stale_cached_connections!
+    Rails.logger.info("Completed services level #{@priority_level}, request #{request.id}: in #{Time.now - bundle_start}") if some_service_executed && @log_timing
+  end
+
+  def forward_exceptions?
+    return @forward_exceptions
+  end
+  def forward_exceptions=(f)
+    @foward_excpetions = f    
+  end
+  
+  protected
+    def clean_backtrace(exception, *args)
+      defined?(Rails) && Rails.respond_to?(:backtrace_cleaner) ?
+        Rails.backtrace_cleaner.clean(exception.backtrace, *args) :
+        exception.backtrace
+    end
+
+  
+end