bento_search 1.7.0.beta.1 → 1.7.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: ac054c4415c845492a1e7e9c472264d242bfb0cf
4
- data.tar.gz: 0f32e4789d3ad0ae264f5cbb28f448cee18c11c9
3
+ metadata.gz: 65bfa7da3d1e0896569d44cb7875931b92876186
4
+ data.tar.gz: 222147474c34a761bfc483abd5185a3b4712b6fa
5
5
  SHA512:
6
- metadata.gz: 14ad5a420be09d2a01e68834e2508a93ae62eebceea84b2077e49494a6ff942b0aca46050f395bb9e3fee9f27ea400c37446c2cb89919ff44222d1dab3c955cb
7
- data.tar.gz: fbeb0bc33bcac41521f642aced391f21d1e04251d0ab05417fe3ae2adb0ada71384d57956a02800a0157c47c8c597dc20028997e8f6fb84abfc6d7ca6bc3870a
6
+ metadata.gz: ee8d316144f84e0469c37d0784d894cda427dc8606ad736af9a0172223c5ace2a61db026fb3d44d2b586791894160dbf5fdce368ae1c5e510314b067ca66bbd0
7
+ data.tar.gz: 3b31f327569b5e9de6fce19da85b7d6f653dae97c9b3ace77139be4c0048a3dff992a3579b4da9b54a553870c23c4fcc681ce193f2dfe3465b7f5487be2ba3e2
data/README.md CHANGED
@@ -250,12 +250,12 @@ to execute concurrently in seperate threads, so the total wait time is the slowe
250
250
  engine, not the sum of the engines.
251
251
 
252
252
  You can write your own logic using ruby threads to do this, but
253
- BentoSearch provides a multi-searching helper using [Celluloid](https://github.com/celluloid/celluloid)
253
+ BentoSearch provides a multi-searching helper using [concurrent-ruby](https://github.com/ruby-concurrency/concurrent-ruby)
254
254
  to help you do this easily. Say, in a controller:
255
255
 
256
256
  ~~~~ruby
257
257
  # constructor takes id's registered with BentoSearch.register_engine
258
- searcher = BentoSearch::MultiSearcher.new(:gbs, :scopus, :summon)
258
+ searcher = BentoSearch::ConcurentSearcher.new(:gbs, :scopus, :summon)
259
259
 
260
260
  # Call 'search' with any parameters you would give to an_engine.search
261
261
  searcher.search("my query", :semantic_search_field => :author, :sort => "title")
@@ -273,13 +273,22 @@ search execute in a seperate thread, so you can continue doing other work
273
273
  in the main thread (like search a local store of some kind outside of
274
274
  bento_search)
275
275
 
276
- You will need to add the 'celluloid' gem to your app to use this feature,
277
- BentoSearch doesn't automatically include the celluloid dependency. Note
278
- that Celluloid uses multi-threading in such a way that you might need
279
- to turn Rails config.cache_classes=true even in development.
276
+ If you are using a Rails previous to 5.x, you will have to add the
277
+ `concurrent-ruby` gem to your `Gemfile` (It's already a dependency of
278
+ Rails5).
279
+
280
+ If you are using Rails5, ConcurrentSearcher uses new Rails API that
281
+ should make development-mode class reloading work fine even with
282
+ the ConcurrentSearcher's concurrency.
283
+
284
+ For more info, see [BentoSearch::ConcurrentSearcher](./app/models/bento_search/concurrent_searcher.rb).
285
+
286
+ The previous **MultiSearcher** class is now deprecated, ConcurrentSearcher
287
+ is the replacement, and will likely work as a drop-in replacement.
288
+ See [CHANGES](./CHANGES.md#17) for more info.
289
+
280
290
 
281
291
 
282
- For more info, see [BentoSearch::MultiSearcher](./app/models/bento_search/multi_searcher.rb).
283
292
 
284
293
  ### Delayed results loading via AJAX (actually more like AJAHtml)
285
294
 
@@ -1,56 +1,72 @@
1
1
  var BentoSearch = BentoSearch || {}
2
2
 
3
- // Pass in a DOM node that has a data-ajax-url attribute.
3
+ // Pass in a DOM node that has a data-ajax-url attribute.
4
4
  // Will AJAX load bento search results inside that node.
5
- // optional second arg success callback function.
5
+ //
6
+ // optional success_callback function.
7
+ //
8
+ // success_callback: will have the container div as 'this', and new content
9
+ // as first argument (in JQuery wrapper). Called before DOM update happens, so
10
+ // first arg div is not yet placed in the DOM. Callback can
11
+ // return `false` to prevent automatic placement in the DOM, if you want
12
+ // to handle it yourself.
13
+ //
14
+ // You can set default success_callback function for all calls with:
15
+ //
16
+ // BentoSearch.ajax_load.default_success_callback = function(div) { ...
6
17
  BentoSearch.ajax_load = function(node, success_callback) {
18
+ // default success_callback
19
+ if (success_callback === undefined) {
20
+ success_callback = BentoSearch.ajax_load.default_success_callback;
21
+ }
22
+
7
23
  var div = $(node);
8
-
24
+
9
25
  if (div.length == 0) {
10
26
  //we've got nothing
11
27
  return
12
28
  }
13
-
29
+
14
30
 
15
31
  // We find the "waiting"/spinner section already rendered,
16
32
  // and show it. We experimented with generating the spinner/waiting
17
33
  // purely in JS, instead of rendering a hidden one server-side. But
18
- // it was too weird and unreliable to do that, sorry.
34
+ // it was too weird and unreliable to do that, sorry.
19
35
  div.find(".bento_search_ajax_loading").show();
20
-
21
-
36
+
37
+
22
38
  // Now load the actual external content from html5 data-bento-ajax-url
23
39
  $.ajax({
24
- url: div.data("bentoAjaxUrl"),
40
+ url: div.data("bentoAjaxUrl"),
25
41
  success: function(response, status, xhr) {
26
- var do_replace = true;
42
+ var do_replace = true;
27
43
  if (success_callback) {
28
44
  // We need to make the response into a DOM so the callback
29
45
  // can deal with it better. Wrapped in a div, so it makes
30
- // jquery happy even if there isn't a single parent element.
46
+ // jquery happy even if there isn't a single parent element.
31
47
  response = $("<div>" + response + "</div>");
32
-
33
- do_replace = success_callback.apply(div, [response]);
48
+
49
+ do_replace = success_callback.apply(div, [response]);
34
50
  }
35
- if (do_replace != false) {
51
+ if (do_replace != false) {
36
52
  div.replaceWith(response);
37
- }
53
+ }
38
54
  },
39
55
  error: function(xhr, status, errorThrown) {
40
56
  var msg = "Sorry but there was an error: ";
41
57
  div.html(msg + xhr.status + " " + xhr.statusText + ", " + status);
42
58
  }
43
59
  });
44
-
45
-
60
+
61
+
46
62
  }
47
63
 
48
64
  jQuery(document).ready(function($) {
49
65
  //Intentionally wait for window.load, not just onready, to
50
- //prevent interfering with rest of page load.
51
- $(window).bind("load", function() {
66
+ //prevent interfering with rest of page load.
67
+ $(window).bind("load", function() {
52
68
  $("*[data-bento-search-load=ajax_auto]").each(function(i, div) {
53
- BentoSearch.ajax_load(div);
69
+ BentoSearch.ajax_load(div);
54
70
  });
55
- });
56
- });
71
+ });
72
+ });
@@ -3,112 +3,110 @@
3
3
  require 'nokogiri'
4
4
 
5
5
  # Rails helper module provided by BentoSearch meant to be included
6
- # in host app's helpers.
6
+ # in host app's helpers.
7
7
  module BentoSearchHelper
8
-
8
+
9
9
  # Renders bento search results on page, or an AJAX loader, etc, as appropriate.
10
10
  # Pass in:
11
11
  # * BentoSearch::SearchResults => will render
12
12
  # * an instantiated BentoSearch::SearchEngine => Will do search and render
13
- # * an id that a search engine was registered under with
14
- # BentoSearch.register_engine => will do search and render.
13
+ # * an id that a search engine was registered under with
14
+ # BentoSearch.register_engine => will do search and render.
15
15
  #
16
16
  # Second arg options hash includes options for bento_search helper,
17
17
  # as well as other options pased on to BentoSearch::Engine.search(options)
18
18
  #
19
+ # Partial used for display can be configured on engine with
20
+ # * for_display.error_partial => gets `results` local
21
+ # * for_display.no_results_partial => gets `results` local
22
+ # * for_display.item_partial => `:collection => results, :as => :item, :locals => {:results => results}`
23
+ # * for_display.ajax_loading_partial => local `engine`
24
+ #
25
+ # If not specified for a particular engine, the partials listed in BentoSearch.defaults will be used.
26
+ #
19
27
  # == Options
20
28
  #
21
29
  # load: :ajax_auto, :immediate. :ajax_auto will put a spinner up there,
22
30
  # and load actual results via AJAX request. :immediate preloads
23
- # results.
31
+ # results.
24
32
  #
25
33
  # == Examples
26
34
  #
27
35
  # bento_results( results_obj )
28
36
  # bento_results( engine_obj, :query => "cancer")
29
37
  # bento_results("google_books", :query => "cancer", :load => :ajax_auto)
30
- #
38
+ #
31
39
  def bento_search(search_arg, options = {})
32
-
40
+
33
41
  results = search_arg if search_arg.kind_of? BentoSearch::Results
34
-
35
- load_mode = options.delete(:load)
36
-
42
+
43
+ load_mode = options.delete(:load)
44
+
37
45
  engine = nil
38
46
  unless results
39
- # need to load an engine and do a search, or ajax, etc.
47
+ # need to load an engine and do a search, or ajax, etc.
40
48
  engine = (if search_arg.kind_of? BentoSearch::SearchEngine
41
49
  search_arg
42
50
  else
43
51
  raise ArgumentError.new("Need Results, engine, or registered engine_id as first argument to #bento_search") unless search_arg
44
52
  BentoSearch.get_engine(search_arg.to_s)
45
53
  end)
46
-
54
+
47
55
  end
48
56
 
49
57
  if (!results && [:ajax_auto, :ajax_triggered].include?(load_mode))
50
58
  raise ArgumentError.new("`:load => :ajax` requires a registered engine with an id") unless engine.configuration.id
51
59
  content_tag(:div,
52
60
  :class => "bento_search_ajax_wait",
53
- :"data-bento-search-load" => load_mode.to_s,
61
+ :"data-bento-search-load" => load_mode.to_s,
54
62
  :"data-bento-ajax-url" => to_bento_search_url( {:engine_id => engine.configuration.id}.merge(options) )) do
55
-
56
- # An initially hidden div with loading msg/spinner that will be shown
57
- # by js on ajax load
58
- content_tag("noscript") do
59
- I18n.t("bento_search.ajax_noscript")
60
- end +
61
- content_tag(:div,
62
- :class => "bento_search_ajax_loading",
63
- :style => "display:none") do
64
- image_tag("bento_search/large_loader.gif",
65
- :alt => I18n.translate("bento_search.ajax_loading")
66
- )
63
+
64
+ partial = (engine.configuration.for_display.ajax_loading_partial if engine.configuration.for_display) || BentoSearch.defaults.ajax_loading_partial
65
+ render :partial => partial, locals: { engine: engine }
67
66
  end
68
- end
69
67
  else
70
68
  results = engine.search(options) unless results
71
69
 
72
70
  if results.failed?
73
- partial = (results.display_configuration.error_partial if results.display_configuration) || "bento_search/search_error"
71
+ partial = (results.display_configuration.error_partial if results.display_configuration) || BentoSearch.defaults.error_partial
74
72
  render :partial => partial, :locals => {:results => results}
75
- elsif results.length > 0
76
- partial = (results.display_configuration.item_partial if results.display_configuration) || "bento_search/std_item"
73
+ elsif results.length > 0
74
+ partial = (results.display_configuration.item_partial if results.display_configuration) || BentoSearch.defaults.item_partial
77
75
  render :partial => partial, :collection => results, :as => :item, :locals => {:results => results}
78
76
  else
79
77
  content_tag(:div, :class=> "bento_search_no_results") do
80
- partial = (results.display_configuration.no_results_partial if results.display_configuration) || "bento_search/no_results"
78
+ partial = (results.display_configuration.no_results_partial if results.display_configuration) || BentoSearch.defaults.no_results_partial
81
79
  render :partial => partial, :locals => {:results => results}
82
80
  end
83
81
  end
84
- end
82
+ end
85
83
  end
86
-
84
+
87
85
  # Wrap a ResultItem in a decorator! For now hard-coded to
88
86
  # BentoSearch::StandardDecorator
89
- def bento_decorate(result_item)
87
+ def bento_decorate(result_item)
90
88
  # in a helper method, 'self' is a view_context already I think?
91
- decorated = BentoSearch::DecoratorBase.decorate(result_item, self)
92
- yield(decorated) if block_given?
93
- return decorated
89
+ decorated = BentoSearch::DecoratorBase.decorate(result_item, self)
90
+ yield(decorated) if block_given?
91
+ return decorated
94
92
  end
95
-
96
-
93
+
94
+
97
95
  ##
98
96
  # More methods used by bento standard views, namespaced with bento_, sorry
99
97
  # no great way to take logic out of views into helper methods without
100
- # namespacey hack.
98
+ # namespacey hack.
101
99
  #
102
100
  # You can use these methods in your own custom views, you also should be
103
101
  # able to over-ride them (including calling super) in local helpers to
104
- # change behavior in standard views.
102
+ # change behavior in standard views.
105
103
  #
106
104
  ##
107
-
105
+
108
106
  # Like rails truncate helper, and taking the same options, but html_safe.
109
- #
110
- # If input string is NOT marked html_safe?, simply passes to rails truncate helper.
111
- # If a string IS marked html_safe?, uses nokogiri to parse it, and truncate
107
+ #
108
+ # If input string is NOT marked html_safe?, simply passes to rails truncate helper.
109
+ # If a string IS marked html_safe?, uses nokogiri to parse it, and truncate
112
110
  # actual displayed text to max_length, while keeping html structure valid.
113
111
  #
114
112
  # Default omission marker is unicode elipsis
@@ -118,23 +116,23 @@ module BentoSearchHelper
118
116
  def bento_truncate(str, options = {})
119
117
  return str if str.nil? || str.empty?
120
118
 
121
- options.reverse_merge!(:omission => "…", :length => 280, :separator => ' ')
122
-
119
+ options.reverse_merge!(:omission => "…", :length => 280, :separator => ' ')
120
+
123
121
  # works for non-html of course, but for html a quick check
124
122
  # to avoid expensive nokogiri parse if the whole string, even
125
- # with tags, is still less than max length.
123
+ # with tags, is still less than max length.
126
124
  return str if str.length < options[:length]
127
-
128
- if str.html_safe?
125
+
126
+ if str.html_safe?
129
127
  noko = Nokogiri::HTML::DocumentFragment.parse(str)
130
128
  BentoSearch::Util.nokogiri_truncate(noko, options[:length], options[:omission], options[:separator]).inner_html.html_safe
131
129
  else
132
130
  return truncate(str, options)
133
131
  end
134
132
  end
135
-
136
133
 
137
-
134
+
135
+
138
136
  # Deprecated, made more sense to put this in a partial. Just
139
137
  # call partial directly:
140
138
  # <%= render :partial => "bento_search/item_title", :object => item, :as => 'item' %>
@@ -142,41 +140,41 @@ module BentoSearchHelper
142
140
  render :partial => "bento_search/item_title", :object => item, :as => 'item'
143
141
  end
144
142
  deprecate :bento_item_title
145
-
143
+
146
144
  # pass in 0-based rails current collection counter and a BentoSearch::Results,
147
- # calculates a user-displayable result set index label.
145
+ # calculates a user-displayable result set index label.
148
146
  #
149
147
  # Only non-trivial thing is both inputs are allowed to be nil; if either
150
- # is nil, nil is returned.
148
+ # is nil, nil is returned.
151
149
  def bento_item_counter(counter, results)
152
150
  return nil if counter.nil? || results.nil? || results.start.nil?
153
-
151
+
154
152
  return counter + results.start + 1
155
153
  end
156
-
157
-
154
+
155
+
158
156
  # returns a hash of label => key suitable for passing to rails
159
- # options_for_select. (Yes, it works backwards from how you'd expect).
157
+ # options_for_select. (Yes, it works backwards from how you'd expect).
160
158
  # Label is looked up using I18n, at bento_search.sort_keys.*
161
159
  #
162
160
  # If no i18n is found, titleized version of key itself is used as somewhat
163
- # reasonable default.
161
+ # reasonable default.
164
162
  def bento_sort_hash_for(engine)
165
- Hash[
163
+ Hash[
166
164
  engine.sort_definitions.keys.collect do |key|
167
165
  [I18n.translate(key.to_s, :scope => "bento_search.sort_keys", :default => key.to_s.titleize), key.to_s]
168
- end
169
- ]
166
+ end
167
+ ]
170
168
  end
171
-
169
+
172
170
  # Returns a hash of label => key suitable for passing to rails
173
171
  # options_for_select. ONLY includes fields with :semantics set at
174
- # present. Key will be _semantic_ key name.
172
+ # present. Key will be _semantic_ key name.
175
173
  # For engine-specific fields, you're on you're own, sorry!
176
174
  #
177
175
  # If first arg is an engine instance, will
178
176
  # be search fields supported by that engine. If first arg is nil,
179
- # will be any field in our i18n lists for search fields.
177
+ # will be any field in our i18n lists for search fields.
180
178
  #
181
179
  # Can pass in options :only or :except to customize list. Values
182
180
  # in :only and :except will match on internal field names OR
@@ -190,25 +188,25 @@ module BentoSearchHelper
190
188
  hash = I18n.t("bento_search.search_fields").invert
191
189
  else
192
190
  hash = Hash[ engine.search_field_definitions.collect do |k, defn|
193
- if defn[:semantic] && (label = I18n.t(defn[:semantic], :scope => "bento_search.search_fields", :default => defn[:semantic].to_s.titlecase ))
191
+ if defn[:semantic] && (label = I18n.t(defn[:semantic], :scope => "bento_search.search_fields", :default => defn[:semantic].to_s.titlecase ))
194
192
  [label, defn[:semantic].to_s]
195
193
  end
196
194
  end.compact]
197
195
  end
198
-
199
- # :only/:except
200
- if options[:only]
196
+
197
+ # :only/:except
198
+ if options[:only]
201
199
  keys = [options[:only]].flatten.collect(&:to_s)
202
- hash.delete_if {|key, value| ! keys.include?(value) }
200
+ hash.delete_if {|key, value| ! keys.include?(value) }
203
201
  end
204
-
202
+
205
203
  if options[:except]
206
204
  keys = [options[:except]].flatten.collect(&:to_s)
207
205
  hash.delete_if {|key, value| keys.include?(value) }
208
206
  end
209
-
207
+
210
208
  return hash
211
209
  end
212
-
213
-
210
+
211
+
214
212
  end
@@ -0,0 +1,136 @@
1
+ begin
2
+ require 'concurrent'
3
+
4
+ # Concurrently runs multiple searches in separate threads. Since a search
5
+ # generally spends most of it's time waiting on foreign API, this is
6
+ # useful to significantly reduce total latency of running multiple searches,
7
+ # even in MRI.
8
+ #
9
+ # Uses [concurrent-ruby](https://github.com/ruby-concurrency/concurrent-ruby),
10
+ # already a dependency of Rails 5.x. To use with Rails previous to 5.x,
11
+ # just add concurrent-ruby to your `Gemfile`:
12
+ #
13
+ # gem 'concurrent-ruby', '~> 1.0'
14
+ #
15
+ # # Usage
16
+ #
17
+ # initialize with id's of registered engines:
18
+ #
19
+ # searcher = BentoBox::MultiSearcher.new(:gbs, :scopus)
20
+ #
21
+ # start the concurrent searches, params same as engine.search
22
+ #
23
+ # searcher.search( query_params )
24
+ #
25
+ # retrieve results, blocking until all are completed:
26
+ #
27
+ # results = searcher.results
28
+ #
29
+ # returns a Hash keyed by engine id, values BentoSearch::Results objects.
30
+ #
31
+ # results # => { "gbs" => <BentoSearch::Results ...>, "scopus" => <BentoSearch::Results ...>}
32
+ #
33
+ # Calling results more than once will just return the initial results again
34
+ # (cached), it won't run a search again.
35
+ #
36
+ # ## Dev-mode autoloading and concurrency
37
+ #
38
+ # In Rails previous to Rails5, you may have to set config.cache_classes=true
39
+ # even in development to avoid problems. In Rails 5.x, we take advantage of
40
+ # new api that should allow concurrency-save autoloading. But if you run into
41
+ # any weird problems (such as a deadlock), `cache_classes = true` and
42
+ # `eager_load = true` should eliminate them, at the cost of dev-mode
43
+ # auto-reloading.
44
+ #
45
+ #
46
+ # TODO: have a method that returns Futures instead of only supplying the blocking
47
+ # results method? Several tricks, including making sure to properly terminate actors.
48
+ class BentoSearch::ConcurrentSearcher
49
+ def initialize(*engine_ids)
50
+ auto_rescued_exceptions = [StandardError]
51
+
52
+ @engines = []
53
+ engine_ids.each do |id|
54
+ add_engine( BentoSearch.get_engine(id).tap { |e| e.auto_rescued_exceptions = auto_rescued_exceptions + e.auto_rescued_exceptions })
55
+ end
56
+ @extra_auto_rescue_exceptions = [StandardError]
57
+ end
58
+
59
+ # Adds an instantiated engine directly, rather than by id from global
60
+ # registry.
61
+ def add_engine(engine)
62
+ unless engine.configuration.id.present?
63
+ raise ArgumentError.new("ConcurrentSearcher engines need `configuration.id`, this one didn't have one: #{engine}")
64
+ end
65
+ @engines << engine
66
+ end
67
+
68
+ # Starts all searches, returns self so you can chain method calls if you like.
69
+ def search(*search_args)
70
+ search_args.freeze
71
+ @futures = @engines.collect do |engine|
72
+ Concurrent::Future.execute { rails_future_wrap { engine.search(*search_args) } }
73
+ end
74
+ return self
75
+ end
76
+
77
+ # Have you called #search yet? You can only call #results if you have.
78
+ # Will stay true forever, it doesn't tell you if the search is done or not.
79
+ def search_started?
80
+ !! @futures
81
+ end
82
+
83
+ # Call after #search. Blocks until each included engine is finished
84
+ # then returns a Hash keyed by engine registered id, value is a
85
+ # BentoSearch::Results object.
86
+ #
87
+ # If called multiple times, returns the same results each time, does
88
+ # not re-run searches.
89
+ #
90
+ # It is an error to invoke without having previously called #search
91
+ def results
92
+ unless search_started?
93
+ raise ArgumentError, "Can't call ConcurrentSearcher#results before you have executed a #search"
94
+ end
95
+
96
+ @results ||= begin
97
+ pairs = rails_wait_wrap do
98
+ @futures.collect { |future| [future.value!.engine_id, future.value!] }
99
+ end
100
+ Hash[ pairs ].freeze
101
+ end
102
+ end
103
+
104
+ protected
105
+
106
+ # In Rails5, future body's need to be wrapped in an executor,
107
+ # to handle auto-loading right in dev-mode, among other things.
108
+ # Rails docs coming, see https://github.com/rails/rails/issues/26847
109
+ @@rails_has_executor = Rails.application.respond_to?(:executor)
110
+ def rails_future_wrap
111
+ if @@rails_has_executor
112
+ Rails.application.executor.wrap { yield }
113
+ else
114
+ yield
115
+ end
116
+ end
117
+
118
+ # In Rails5, if we are collecting from within an action method
119
+ # (ie the 'request loop'), as we usually will be, we need to
120
+ # give up the autoload lock. Rails docs coming, see https://github.com/rails/rails/issues/26847
121
+ @@rails_needs_interlock_permit = ActiveSupport::Dependencies.respond_to?(:interlock) &&
122
+ !(Rails.application.config.eager_load && Rails.application.config.cache_classes)
123
+ def rails_wait_wrap
124
+ if @@rails_needs_interlock_permit
125
+ ActiveSupport::Dependencies.interlock.permit_concurrent_loads { yield }
126
+ else
127
+ yield
128
+ end
129
+ end
130
+
131
+ end
132
+ rescue LoadError
133
+ # you can use bento_search without celluloid, just not
134
+ # this class.
135
+ $stderr.puts "Tried but could not load BentoSearch::ConcurrentSearcher, concurrent-ruby not available!"
136
+ end